Research · Published · Updated
A Protocol for Judging Whether an Escalation Was Appropriate
Escalation volume says little on its own. This method separates correct stops from unnecessary transfers and missed boundaries.
On this page
Headline signal
A decision-quality protocol, not a target escalation rate. (https://csrc.nist.gov/pubs/sp/800/61/r3/final). Treat this as context for planning, not a guarantee about a particular assistant or operating result.
Analysis and operating context
A rising escalation count can mean the assistant is cautious, the brief is unclear, or a boundary is working exactly as designed. A falling count can mean the team learned the work or that risky requests are being handled without authority. Volume alone cannot tell these stories apart. This protocol classifies escalations by whether the stop was required, reasonable, or avoidable, and by whether the receiving owner could act on the packet.
The unit is one escalation event from the assistant's queue to a named receiving owner. Record the task, the stated reason, the boundary that applied, the evidence included, the time sent, the time accepted or returned, and the final disposition. Keep the original reason text. Do not rewrite it later to match the outcome, because the reasoning at the time is the object of study.
Define the boundary categories before sampling: identity or access, money or payment, legal or contractual, security, policy exception, technical change, and missing authority. A boundary may be written in the procedure, in the client contract, or in a standing instruction. If the boundary is not written, the case belongs in a separate category called undocumented boundary, which is itself a finding.
Classify each event on two dimensions. The first is necessity: required, reasonable but optional, or avoidable. Required means a written rule reserved the decision to another owner. Reasonable means the assistant lacked a fact or authority that the brief did not cover. Avoidable means the procedure or the evidence was available and the assistant transferred work that could have been completed. The second dimension is packet quality: actionable, incomplete, or unusable.
Use two reviewers and a calibration set. Give them the written boundaries and the task records, but not the outcome, so they judge the decision rather than the result. Compare classifications and discuss disagreements. A disagreement about necessity usually points to a boundary that exists only in someone's head. A disagreement about packet quality points to an unclear evidence standard. Preserve the initial ratings.
Analyze the combinations. Required and actionable is the healthy case, even if volume is high. Avoidable and actionable suggests training or procedure drift. Required and incomplete suggests the escalation template is missing fields. Avoidable and unusable is the most expensive pattern because it consumes two queues and still returns. Report the cells as counts, not as a single quality score.
Track what happens to the packet. Measure acceptance time, return rate, and the reason for return. A returned escalation is not automatically a failure; it may show that the receiving owner needed one missing fact. Group returns by missing field so the template can be fixed at the source. Do not ask the assistant to remember a longer list of fields without changing the form.
Interpret trends with the work mix. A new client with stricter access rules may raise required escalations without any change in assistant performance. A tool migration may create avoidable escalations for a few weeks. Compare like periods and disclose the operational changes. A target that rewards fewer escalations can suppress correct stops, so avoid using the count as a performance target.
Limitations include hindsight bias, undocumented boundaries, and the difficulty of judging necessity when the brief is ambiguous. The protocol also cannot determine whether the receiving owner made the right decision; it only examines the handoff. Keep that boundary explicit so the study is not mistaken for a review of outcomes.
The practical output is a short list: boundaries to write down, fields to add to the escalation form, procedures to clarify, and cases where the assistant should have completed the work. Review the list with the people who handle both sides of the handoff, then retest after the changes.
Apply this to a research support lane
Classify one month of escalations before changing any target.
Review an escalation sampleSources
Frequently asked questions
Is a lower escalation rate better?
Not necessarily. The useful question is whether each stop was required or avoidable and whether the packet let the receiver act.
Who should review the sample?
One operations owner and one person from the receiving queue, calibrated on the same definitions before the main review.