Digital Workers and Bounded Delegation

C11.1 — Delegate a bounded production role, not a vague intention

Delegation begins with a named task and intended outcome. “Improve this service” is not a production role. “Identify a class of code-efficiency opportunities, propose a bounded source change, run specified validation, open the change for independent review, and monitor the accepted change under a defined rollback policy” is closer to one.

The difference is operational. A named role allows the factory to specify which repositories, branches, environments, tools, credentials, data, actions, recipients, time windows, and spending limits are available. It also permits explicit exclusions. The actor may propose but not approve; change a branch but not merge; invoke a test service but not alter its policy; read a production signal but not query protected customer content; or stop when a task crosses a declared consequence boundary.

This is why autonomy is an incomplete design variable. Two systems described as “autonomous” may have radically different authority. One can prepare a draft pull request in an isolated branch. Another can merge, deploy, change data, contact users, or purchase services. The second propagates consequence farther and needs stronger verification, monitoring, intervention, and recovery. The relevant question is not how independent the software appears. It is what it is permitted to change, for whom, with what evidence, and under whose responsibility.

The NIST AI Risk Management Framework and its Generative AI Profile emphasize context, roles, limits, testing, monitoring, incident processes, fallback, override, and documented responsibility. NIST SP 800-53 provides established control families for identification, least privilege, audit, monitoring, separation, contingency, and recovery. These sources support the control families used here. They do not prescribe the book’s exact envelope, and their inclusion of a control does not prove that a particular implementation is effective.S11C11-S01C11-S02

GitHub’s documentation for its cloud coding agent supplies a concrete first-party control design. Initiation is tied to authorized users; work is attributed; branch scope, merge authority, rulesets, checks, logging, and review paths are constrained. Dependabot illustrates that even narrower service automation needs a named identity, credential boundary, reviewable output, and restricted access to secrets. Product documentation describes intended behavior, not independent evidence of safe operation, false-acceptance rates, review burden, incidents, rollback, or value.C11-R01C11-R02

The first manufacturing rule is therefore:

Delegate the smallest coherent role whose inputs, authority, output, evidence, and consequence can be governed together.

A tiny task is not always the correct boundary. Excessive fragmentation can destroy context and create coordination errors. The goal is a coherent bounded role: large enough to produce a meaningful outcome, small enough that authority and consequence remain inspectable.

C11.2 — Put the actor inside one accountability envelope

The Digital Worker delegation and accountability envelope has eight connected elements. They must be designed as one system because a weakness in one can invalidate the others.

Identity

The factory must distinguish the acting service, model or rule version, the initiating person or system, and the accountable owner. Identity supports attribution and control; it does not make software accountable in the human or legal sense. A shared token, generic bot account, or overwritten session record makes it difficult to reconstruct who initiated work, which mechanism acted, and which policy applied.

Authority

Authority names the allowed tools, credentials, data, repositories, branches, environments, actions, duration, and resource limits. It should be enforced technically where practical, not left as prompt text or team custom. Least privilege remains useful, but “least” is contextual: a role needs enough authority to complete its bounded outcome without acquiring unrelated power.

Inputs and outputs

Permitted input sources, data boundaries, expected artifacts or actions, and intended recipients must be explicit. A prompt is not a complete specification. Inputs may be stale, malicious, ambiguous, private, or outside the actor’s approved purpose. Outputs can include more than code: decisions, messages, configuration, evidence, requests, or side effects.

Evidence

The envelope must retain the evidence needed to reconstruct the action and decide whether it may continue. That can include session and action records, input provenance, tool use, produced artifacts, test results, review comments, approval or rejection, deployment state, and observed outcome. Logging is not correctness. A complete record of an invalid action remains an invalid action.

Monitoring

Monitoring watches the actor, its environment, and the human-machine configuration. Signals may concern behavior, quality, security, latency, cost, burden, drift, policy exceptions, or unexpected tool use. There is no universal metric set or alert threshold. Monitoring should be derived from the role, failure modes, affected parties, and intervention needs.

Escalation

Ambiguity, policy exceptions, failed verification, anomalous behavior, time or budget limits, and affected-party recourse need explicit paths. The actor cannot be the sole judge of whether escalation is necessary. Escalation must reach someone with competence, context, time, and real authority to decide, stop, or reframe the work.

Reversibility and recovery

Reversibility has at least three forms: discard before acceptance, rollback after a change, and recovery from external consequence. They are not equivalent. A source commit may be reverted while leaked information, corrupted data, a customer communication, a purchased resource, or physical consequence persists. The envelope must match recovery to the actual effects the actor can create.

Accountable human ownership

A named person or organizational role owns the delegated system, its authority, its monitoring, and its consequences. Ownership includes maintaining the role, responding to incidents, reviewing evidence quality, changing controls, and retiring the delegation when it is no longer justified. “Human in the loop” is not enough if the human lacks evidence, independence, attention, competence, time, or reject-and-stop authority.C11-A03C11-H01

These elements form a control relationship, not a score. An implementation does not become safer merely by counting more controls. The factory must show that the elements connect: identity binds to authority; actions create evidence; evidence reaches verification and review; monitoring can trigger intervention; and ownership persists through rollback, recovery, and learning.

Open figure

Figure F11.1 production specification: Digital Worker delegation and accountability envelope

Figure F11.1 — A Digital Worker operates inside one enforceable envelope connecting identity, authority, inputs and outputs, evidence, monitoring, escalation, reversibility, and accountable human ownership. Uncertainty, consequence, irreversibility, verification difficulty, weak evidence, or an incident can narrow authority or stop delegation. The envelope enables governed decisions; it does not guarantee a safe or useful outcome.

C11.3 — Contract authority as consequence becomes harder to verify or reverse

Delegation is not a one-way ladder toward greater autonomy. Authority may expand after new evidence, contract when operating conditions change, or be removed after an incident, control failure, task shift, or loss of confidence.

Six conditions should make the factory reconsider or narrow authority:

1. uncertainty about the task, environment, or actor rises;
2. the possible consequence or number of affected parties increases;
3. effects become harder to reverse;
4. verification becomes more difficult or less independent;
5. evidence becomes weak, stale, incomplete, or unavailable; or
6. an anomaly, incident, or policy exception occurs.

The decision can end in four states:

• delegate within the current envelope;
• delegate with stronger verification or narrower authority;
• escalate for a human decision; or
• do not delegate, or deactivate the delegation.

These states are not universal autonomy tiers. A task can move among them as context changes. A low-consequence documentation correction may be accepted automatically after deterministic checks. The same actor encountering an authentication change, protected data, a production migration, or a weak test environment may be limited to preparing evidence for a human decision.

The European Union AI Act provides one jurisdiction- and classification-bounded example of requirements concerning record-keeping, human oversight, accuracy, robustness, and cybersecurity for covered high-risk systems. It does not make every development assistant a high-risk system, and this chapter does not offer legal advice. Its useful contribution here is narrower: oversight must be designed around context, capability, consequence, and the ability to intervene, not declared through a generic label.C11-L01

Proportionality also prevents control from consuming the value of delegation. Maximum review and maximum isolation are not sensible defaults for every task. Controlled evidence shows that AI assistance can accelerate some bounded tasks, while other controlled evidence found slower completion for a small group of experienced open-source developers using early-2025 tools. The studies used different people, work, tools, measures, and conditions; their headline effects must not be compared as if they estimate one universal productivity effect.A11A12

The factory should instead measure the chosen role in its context. What work was eligible? What was proposed, rejected, revised, accepted, deployed, retained, or reversed? What human time and displaced work were required? Which consequences were observed, and which could not be observed? Proportional control and task-specific evidence are compatible: the controls define what may happen; the evidence tests whether the role remains justified.

Generated work is not yet organizational value

Agent evidence is frequently reported at the first observable output. A benchmark task passes. Code is generated. A pull request is opened or merged. A developer reports faster work. Each can be useful, but none alone establishes dependable production performance.

The hierarchy must remain explicit:

Code generated
≠
Change accepted
≠
Change safely deployed
≠
Capability improved
≠
Total organizational value established

Generation says an artifact exists. Acceptance says an authorized decision accepted it under a particular evidence set. Deployment says it entered an operating context; “safely” additionally requires declared outcome and consequence evidence. Capability improvement requires evidence that the change improved a relevant ability or outcome over a stated horizon. Organizational value must include benefits, costs, burdens, risks, displaced work, affected parties, and alternatives.

SWE-bench demonstrates that issue-resolution capability can be evaluated against repository tasks and executable tests. SWE-bench-Live improves freshness and repository diversity. Both remain benchmarks: they do not supply production authority, security review, deployment, monitoring, recovery, cost, or downstream value.A10A14

Public pull-request cohorts add workflow evidence. Studies of agent-authored pull requests show that merge and rejection labels can conceal agent failure, workflow constraints, absent rationale, reviewer intervention, and incomplete visible interaction. A 33,000-pull-request cohort broadens observation across agents and repositories, but public traces still do not expose hidden organizational authority, intervention, incidents, recovery, full cost, or maintenance.C11-A01C11-A06

Organization-scale studies reach farther but retain the same hierarchy. A Microsoft-authored preprint reported a merged-pull-request increase among adopters across tens of thousands of engineers during an early rollout, while explicitly leaving quality unresolved. An NBER working paper using telemetry for more than 100,000 developers reported that changes attenuated from coding activity toward projects, releases, and app use. These sources support separating activity, accepted work, shipped outcomes, use, quality, and value. They do not provide a universal productivity result, false-acceptance rate, incident denominator, review cost, or total lifecycle account.C11-A04C11-A05

The production system should therefore retain evidence at multiple levels rather than turn the most convenient measure into the conclusion. A healthy system may reject many generated proposals because its verification works. A high merge count may reflect trivial tasks, changed task mix, or insufficient challenge. A deployment may produce no immediate alert while creating maintenance debt. A capability improvement may be real but too costly to sustain. The hierarchy keeps each question open until evidence appropriate to that question exists.

Open figure

Figure F11.2 production specification: From generated output to organizational value

Figure F11.2 — Generated, accepted, safely deployed, capability-improving, and organization-valued outcomes require different evidence. The stages are related but no stage automatically proves the next.

Human review is a control, not a guarantee

Human review is necessary for many consequential delegations, but the phrase “human in the loop” can hide more than it reveals. Which human? At what point? With what information, competence, time, independence, and authority? Can the reviewer inspect the actor’s inputs and tool use? Can they reject, stop, roll back, or recover? Is review performance itself monitored?

A systematic review of automation bias found evidence of biased reliance across heterogeneous human-factors and healthcare tasks and associated verification complexity with problematic settings. It does not measure software-development review. Its transferable caution is that nominal human presence does not guarantee effective verification.C11-A03

The NTSB investigation of the 2018 Tempe automated-driving fatality is a historical human-factors boundary, not a software-production case. The investigation found inadequate countermeasures for automation complacency and weak operator monitoring among contributing conditions. It must not be used to infer coding-agent incident frequency or to dramatize ordinary development work. It demonstrates a narrower point: assigning a person to supervise automation does not create effective oversight if the socio-technical conditions make sustained, informed intervention unlikely.C11-H01

Review must be designed as production work. The reviewer needs a clear decision, a bounded evidence packet, sufficient context, independence appropriate to consequence, and a real stop path. The factory should observe review load, latency, rework, override, escalation, and outcomes rather than assume that an approval field closes the risk.

Automation may also challenge the reviewer. Independent tests, policy checks, adversarial evaluation, staged exposure, or a second mechanism can reduce reliance on one actor or one person. Independence is contextual: two agents using the same model, data, prompt pattern, and evaluator may produce the appearance of challenge while sharing the same failure mode.

The accountable owner remains responsible for the whole arrangement. If review demand becomes too high, evidence quality declines, or overrides become routine, the response is not to preserve throughput by making approval ceremonial. The factory can narrow task eligibility, improve verification, reduce authority, add capacity, redesign the role, or stop delegation.

The bounded delegation test

Before authorizing or renewing a Digital Worker role, leaders can ask ten questions.

1. What coherent production outcome is delegated? Name the task, intended result, exclusions, and affected parties.
2. Who is acting, who initiated the work, and who remains accountable? Keep actor identity, initiator, approver, operator, and owner distinct.
3. What authority is technically available? List tools, credentials, data, environments, actions, time, and resource limits.
4. What evidence is required before action or acceptance? Include provenance, tests, policy checks, review, and uncertainty.
5. What can the actor change without a human decision? Match the decision path to consequence, reversibility, and verification difficulty.
6. What conditions force escalation, narrowing, or deactivation? Do not leave escalation solely to the actor.
7. How can effects be discarded, rolled back, contained, or recovered? Follow external effects, not only source changes.
8. How is the human-machine configuration monitored? Observe actor behavior, control performance, reviewer burden, and operating outcomes.
9. Which output level has actually been established? Do not turn generated, merged, or deployed work into an unsupported value claim.
10. What evidence would cause the factory to change its decision? Define renewal, expansion, contraction, and retirement conditions.

This is a decision framework, not an autonomy maturity model, certification, universal permission set, or guarantee of safety. It applies to deterministic automation and agentic systems while allowing their evaluation methods and control strength to differ. The role, failure modes, uncertainty, attack surface, and consequence determine the implementation.

For executives, the governance obligation is not to approve every prompt or pull request. It is to require that consequential delegated roles have explicit boundaries, evidence, intervention, recovery, and ownership; to prevent output proxies from becoming value claims; and to ensure that the organization can stop a system whose evidence no longer justifies its authority.

For production teams, the practical obligation is equally concrete. Treat the Digital Worker as an operated production capability. Version its role and controls. Test the control path. Observe review and exception burden. Rehearse deactivation and recovery. Retain evidence long enough to answer consequential questions, while respecting security, privacy, and proportionality.

What to remember

A Digital Worker is a governed software-based production actor, not a metaphorical employee and not an escape from human accountability.

Delegation begins with a bounded role. Identity, authority, inputs and outputs, evidence, monitoring, escalation, reversibility, and accountable human ownership form one enforceable envelope.

Authority should contract when uncertainty, consequence, irreversibility, verification difficulty, evidence weakness, or incidents increase. Delegation can expand, narrow, escalate, stop, or be retired.

Google ECO demonstrates a bounded production agent system operating at meaningful scale with disclosed verification, review, monitoring, and rollback observation. It does not establish universal agent effectiveness, total cost reduction, incident frequency, complete review burden, maintenance economics, or industry-wide productivity.

Generated, accepted, safely deployed, capability-improving, and organization-valued outcomes require different evidence. Human review and logging are controls, not guarantees.

Delegate bounded work. Keep authority, evidence, intervention, recovery, and accountability connected.