← Back to the atlas
Testing & agents

Testing the decision to stop

In draft Draft date

Completion, escalation, and unnecessary continued action are distinct behaviors that need distinct evaluations

Working outline — full essay forthcoming

Rough outline

Define the stopping states

  • Separate success, blocked execution, uncertainty, and budget exhaustion
  • Specify what the user should receive in each case

Construct adversarial cases

  • Include tasks that are complete before the agent believes they are
  • Include tasks that look complete but lack necessary verification

Measure the tradeoffs

  • Track premature stops alongside unnecessary actions
  • Account for the consequences of side effects, latency, and cost

Test the handoff

  • Check whether an escalation explains what is known and what is missing
  • Verify that retrying a stopped task does not duplicate completed actions