Abstract:
The vocabulary of Human-Computer Interaction was established in an era when humans executed and computers assisted. Norman's gulfs, GOMS, Cognitive Work Analysis - these frameworks share an assumption: the human perceives and acts, the system responds, and good design closes the gap between intention and outcome. Forty years on, AI and Agentic systems have started challenging that relationship. The human is no longer the sole executor but the supervisor, or the approver, or, more often than designers care to admit, the bystander. Yet we continue to reach for the same conceptual toolkit, dressing supervision-era problems in execution-era language. This flaw in categorization goes beyond the conceptual: it produces systems that cannot name what their human operators are actually doing, and therefore cannot design for it.
The transformation we’re witnessing in software development tools is not an incremental evolution but a category shift in what it means to be the human on the loop. We argue that the most common human-agent configuration now being deployed, the developer passively monitoring a capable autonomous system, labelled Human-on-the-Loop (HOTL), is not a guarantee of safe performance. In fact, it is the most cognitively dishonest position of all. It promises supervision while structurally rendering it quite difficult. We call this pattern the supervision theatre. In this performance, the human acts more like the spear carrier - the non-speaking background actor whose presence implies a scene they cannot affect.
Our argument rests on Rasmussen's Decision-Ladder and his distinction between skill, rule, and knowledge-based behaviour. HOTL assumes the human can ascend on demand (perceiving, interpreting, evaluating, reformulating intention), in time to intervene meaningfully. But when the system is fast, opaque, and confidence-inducing, the human is held at the skill-based rung, reacting to salient surface signals rather than reasoning about them. Agentic coding tools optimised for frictionless flow create exactly these conditions, and then call it oversight.
Kahneman's account of System 1 and System 2 thinking compounds the problem. System 2 - the slow, deliberate, effortful reasoning that meaningful intervention requires - does not simply activate because the situation demands it. It activates when friction signals that something is wrong. Agentic interfaces optimised for fluency are, in this precise sense, System 2 suppressants: the confident output stream, the seamless commit-and-deploy removes the very friction that would trigger deliberate evaluation. The human watches the agent act, nothing feels wrong, and System 2 never engages.
Aviation and automotive automation are instructive here. Those fields demonstrate what genuine supervisory infrastructure requires: moving yokes, mode awareness displays, and recurrent simulation training. What made them a benchmark for human-automation partnership was not a commitment to HOTL throughout. It was the investment in making supervisory positions cognitively honest, and the discipline to insert Human-in-the-Loop (HITL) gates at moments of genuine consequence, such as the pre-flight checklist or the mandatory handover protocol.
Effective Human-Agent Interaction (HAI) requires knowing where the human actually is in the loop at any given moment, and designing for that position, not the one we assume or the one for which is easier to design. We propose four dimensions (accuracy required, latency tolerated, risk exposure, and cognitive feasibility) for locating the human in the HAI space, and derive corresponding interface design commitments. Patching a vulnerability identified in a production codebase is not the same kind of task as drafting a commit message, and current interfaces should surface the appropriate differences. The framework is not a taxonomy for its own sake: it is a diagnostic instrument. Each dimension answers one of the questions that current tools and design discourse cannot answer, and we show that inability using the concrete failure patterns already endemic to today's agentic coding tools. Naming the position is the precondition for designing for it.
Software designers are choosing right now which patterns become the default for an industry still forming new Human-Agent interaction habits. These choices should be grounded in the cognitive-engineering tradition that tackled these problems in industries that faced somewhat similar transformations. We argue for the conditions it must meet: design that names the loop position the human inhabits, that treats every transition as a critical surface for design, and that measures transparency by what the human can do with what is shown, not by how much is shown.
The transformation we’re witnessing in software development tools is not an incremental evolution but a category shift in what it means to be the human on the loop. We argue that the most common human-agent configuration now being deployed, the developer passively monitoring a capable autonomous system, labelled Human-on-the-Loop (HOTL), is not a guarantee of safe performance. In fact, it is the most cognitively dishonest position of all. It promises supervision while structurally rendering it quite difficult. We call this pattern the supervision theatre. In this performance, the human acts more like the spear carrier - the non-speaking background actor whose presence implies a scene they cannot affect.
Our argument rests on Rasmussen's Decision-Ladder and his distinction between skill, rule, and knowledge-based behaviour. HOTL assumes the human can ascend on demand (perceiving, interpreting, evaluating, reformulating intention), in time to intervene meaningfully. But when the system is fast, opaque, and confidence-inducing, the human is held at the skill-based rung, reacting to salient surface signals rather than reasoning about them. Agentic coding tools optimised for frictionless flow create exactly these conditions, and then call it oversight.
Kahneman's account of System 1 and System 2 thinking compounds the problem. System 2 - the slow, deliberate, effortful reasoning that meaningful intervention requires - does not simply activate because the situation demands it. It activates when friction signals that something is wrong. Agentic interfaces optimised for fluency are, in this precise sense, System 2 suppressants: the confident output stream, the seamless commit-and-deploy removes the very friction that would trigger deliberate evaluation. The human watches the agent act, nothing feels wrong, and System 2 never engages.
Aviation and automotive automation are instructive here. Those fields demonstrate what genuine supervisory infrastructure requires: moving yokes, mode awareness displays, and recurrent simulation training. What made them a benchmark for human-automation partnership was not a commitment to HOTL throughout. It was the investment in making supervisory positions cognitively honest, and the discipline to insert Human-in-the-Loop (HITL) gates at moments of genuine consequence, such as the pre-flight checklist or the mandatory handover protocol.
Effective Human-Agent Interaction (HAI) requires knowing where the human actually is in the loop at any given moment, and designing for that position, not the one we assume or the one for which is easier to design. We propose four dimensions (accuracy required, latency tolerated, risk exposure, and cognitive feasibility) for locating the human in the HAI space, and derive corresponding interface design commitments. Patching a vulnerability identified in a production codebase is not the same kind of task as drafting a commit message, and current interfaces should surface the appropriate differences. The framework is not a taxonomy for its own sake: it is a diagnostic instrument. Each dimension answers one of the questions that current tools and design discourse cannot answer, and we show that inability using the concrete failure patterns already endemic to today's agentic coding tools. Naming the position is the precondition for designing for it.
Software designers are choosing right now which patterns become the default for an industry still forming new Human-Agent interaction habits. These choices should be grounded in the cognitive-engineering tradition that tackled these problems in industries that faced somewhat similar transformations. We argue for the conditions it must meet: design that names the loop position the human inhabits, that treats every transition as a critical surface for design, and that measures transparency by what the human can do with what is shown, not by how much is shown.