Part VIII — Human Agency and AI-Native Autonomy
Human Agency in the Software Factory
What kind of human work and organization should Software Manufacturing create?
Faster output can still make worse work
A Software Factory changes more than delivery mechanics. It changes who frames a problem, who performs a task, who checks the result, who is watched, who learns, who is interrupted, and who can say no.
Automation is often introduced with a simple story: machines remove toil so people can exercise judgment. That outcome is possible. It is not automatic.
A tool can shorten code production while increasing review, correction, coordination, or recovery. It can expand access for one person while making navigation or context harder for another. It can help an experienced worker on an unfamiliar task and slow the same worker inside a mature codebase they know deeply. It can raise measured output while narrowing entry-level opportunity or moving invisible work to reviewers, maintainers, contractors, and data labor.
The central question is therefore not:
What work can the machine do?
It is:
What work system are we creating, for whom, with which authority, burden, learning, protection, evidence, and recourse?
The bounded proposition is:
Human agency must be designed and governed as part of the production system: specify the work, allocate authority, involve affected people, protect learning and access, constrain monitoring, measure burdens and benefits, and retain the power to adapt or stop.
This chapter does not claim that AI adoption will augment jobs, displace workers, improve productivity, preserve skill, or create better work. Those are outcomes to investigate, not premises to insert into the design.
C22.1 — Describe the work before allocating it
“Coding” is too large a unit for a human–machine decision.
A change may involve interpreting user need, locating relevant context, proposing an approach, generating code, changing tests, evaluating security, reviewing behavior, explaining a decision, coordinating a release, monitoring production, and responding to failure. Different parts carry different ambiguity, consequence, evidence, and accessibility requirements.
Begin with a bounded task-and-work record:
- the user or operational outcome;
- the work as performed now;
- the recurring burden or failure to address;
- the proposed human and machine activities;
- the information and tools each activity requires;
- the consequence of error or delay;
- the person accountable for the decision;
- the people affected by the change; and
- the conditions that should pause or end the arrangement.
This prevents “the agent writes the code” from hiding a larger redistribution. Prompting, waiting, reading, cross-checking, correcting, explaining, switching views, resolving conflicting suggestions, recovering state, and reassuring downstream reviewers are work. If those activities move to somebody else, the task has not disappeared.
Software experiments make the context problem visible. Three field experiments involving 4,867 developers reported an aggregate increase in completed tasks for developers given access to a coding assistant, with higher adoption and larger gains among less-experienced developers.C22-A03 A separate randomized study of 16 experienced open-source developers working on 246 real tasks in mature projects they knew well found that early-2025 AI tools slowed that group, despite their expectation that the tools would help.C22-A04
These results need not be reconciled into one productivity estimate. The population, task, codebase, organization, measure, tool vintage, and observation window differ. Their combined lesson is operational: import the method of measurement, not the headline number.
Authority is more precise than “human in the loop”
A human presence does not establish human control.
The person may be expected to approve a result without time, context, access, or permission to reject it. They may carry liability for a decision the system effectively made. They may be able to stop one action while having no influence over the metric, model, data, queue, or policy that produced it.
For each material activity, state:
- what the system may propose;
- what it may execute;
- what the person must inspect;
- what evidence is available;
- what the person may change, reject, or reverse;
- what happens when evidence is missing or conflicting;
- who can escalate;
- who owns harm and recovery; and
- who can suspend the capability.
NIST's AI Risk Management Framework calls for differentiated human–AI roles and oversight, lifecycle accountability, attention to affected communities, and accessible, transparent risk management.C22-S01 It is normative guidance, not proof that an oversight design works.
The practical test is whether authority is usable under production conditions. A reviewer with a full queue, unfamiliar code, weak evidence, and a release deadline is not a control simply because their name appears in an approval field.
Verification therefore needs its own capacity plan. Estimate the time, skill, tooling, independence, and evidence required to detect a consequential error. Measure correction and recovery, not merely acceptance. When machine output grows faster than credible verification capacity, reduce authority, narrow the task, add containment, or stop.
Chapter 23 will apply this rule to autonomous Digital Workers. Here the point is human: oversight without the means to exercise judgment transfers accountability without transferring power.
C22.2 — Measure the whole work loop
Output is one result of a work system. It is not the work system.
A Chapter 22 evaluation should keep separate measures for:
- task completion and elapsed time;
- product and service quality;
- review, correction, and recovery burden;
- interruption and coordination;
- skills exercised, displaced, and newly required;
- learning time and access to mentoring;
- autonomy, intensity, safety, and well-being;
- accessibility and accommodation burden;
- participation, challenge, and recourse;
- opportunity, role movement, and employment signals; and
- disconfirming or dissenting accounts.
Do not collapse these into a human-agency score.
Butler and colleagues combined a randomized trial with surveys and a three-week workplace diary. Participants reported changing practices and increased usefulness and enjoyment, while views of AI-generated code's trustworthiness did not change.C22-A01 That is evidence of short-window practice and perception, not durable skill, satisfaction, or career change.
DORA's 2025 research treats AI as an amplifier of the surrounding organizational system.C22-R04 Its reported associations can help an organization formulate hypotheses, but observational relationships do not establish that a tool caused the result.
The evaluation record should therefore name:
- baseline and comparison;
- affected population;
- task and consequence;
- tool and model version;
- adoption and actual use;
- observation window;
- missing people and missing work;
- uncertainty and alternative explanations;
- review date; and
- the decision the evidence can support.
A Production Line may continue, adapt, constrain, or stop. Continuing is not the default, and stopping is not failure. Both are evidence-bearing decisions.
C22.3 — Protect learning during role transition
Automation can remove repetitive work. It can also remove the practice through which judgment was formed.
If a junior engineer no longer traces a failure, reads unfamiliar code, constructs a test, or receives feedback on a first attempt, the organization may save time today while weakening tomorrow's verification capacity. If a senior engineer becomes a permanent reviewer of machine output, “higher-level work” may become an intensified queue of context reconstruction and exception handling.
A transition record should name:
- tasks removed, retained, changed, and introduced;
- skills exercised before and after;
- new verification and orchestration capability required;
- protected learning and practice time;
- mentoring and feedback routes;
- workload and role-boundary changes;
- support, redeployment, and opportunity;
- evidence of capability change; and
- a date to reconsider the allocation.
Developer preferences offer useful design input. A mixed-methods study of 860 Microsoft developers found demand for support with routine work, alongside limits involving reliability, security, transparency, steerability, fairness, mentoring, and professional identity.C22-A02 Preferences do not establish realized benefit or consent, but they warn against defining “toil” without the people who perform and learn through the work.
The ILO's 2026 review finds uneven and sometimes unverified productivity gains, limited observed large-scale displacement so far, and risks involving coordination, autonomy, job quality, inequality, and younger workers.C22-R03 The evidence is heterogeneous and rapidly changing. The defensible response is not a prediction. It is a transition discipline that keeps opportunity and capability visible while the evidence develops.
Accessibility is part of the operating design
An accessible tool is not necessarily an accessible work loop.
A peer-reviewed study of developers with visual impairments found both perceived benefits and new barriers from coding assistants, including excessive suggestions and harder context switching.C22-A05 A two-week study of 16 blind and low-vision programmers found capability expansion alongside difficulty conveying intent, interpreting output, managing multiple views, maintaining situational awareness, and learning the tools. Participants expressed different preferences and a need to balance automation with control.C22-A06
These studies do not represent every disability or establish long-term outcomes. They support a strong design rule: test the real workflow with affected people.
The record should examine:
- keyboard and screen-reader paths;
- audio, visual, and textual cues;
- focus and navigation across editor, chat, terminal, review, and evidence;
- the ability to locate and understand changes;
- suggestion volume and cognitive load;
- status, progress, and next-action clarity;
- configuration of detail, playback, interruption, and automation;
- recovery and fallback without the tool; and
- whether documentation and training are accessible.
Do not create one “accessibility persona” and treat it as coverage. Different people need different levels of detail, cue types, control, and pacing. Configuration is part of the capability, and affected-worker evidence is part of its release gate.
C22.4 — Give worker voice decision rights and recourse
Feedback after deployment is too late if the purpose, data, metric, or authority is already fixed.
Affected people and their representatives need routes before, during, and after a work-system change:
- participate in problem framing;
- challenge the definition of toil or productivity;
- identify hidden work and unequal effects;
- influence task and interface design;
- inspect monitoring and decision use;
- test accessibility and workload;
- report harm without retaliation;
- contest a decision or record;
- trigger correction, pause, or review; and
- see what changed as a result.
ILO case studies across five continents describe worker representatives influencing employment and skill questions, algorithmic management, working conditions, and labor in AI value chains.C22-R05 An OECD laboratory experiment in three German manufacturing firms found that consultation could change a simulated algorithmic-management design in ways stakeholders judged capable of preserving productivity and improving job quality.C22-R06
Neither source proves a universal consultation effect, and neither is a software-factory case. They establish that participation can be made concrete. “We consulted the team” is insufficient. Record who was represented, when they entered, what information and protection they had, what they could change, dissent, unresolved issues, and recourse.
Constrain monitoring before it becomes management
Digital work creates abundant traces. A factory can measure commits, prompts, suggestions, reviews, latency, tool use, queue time, corrections, access patterns, and communication. That does not make every trace legitimate performance evidence.
Algorithmic management can allocate, direct, monitor, and evaluate work. ILO and JRC cases across logistics and healthcare found different effects by country, sector, institutions, and regulation, including surveillance, work intensity, and job-quality concerns.C22-R07 OECD evidence likewise identifies autonomy, privacy, monitoring, intensity, health, and unequal effects as material job-quality dimensions.C22-R01
Before collecting worker-linked data, state:
- the decision purpose;
- the minimum data required;
- who can access it;
- retention and deletion;
- prohibited uses and inferences;
- how a person can inspect and correct it;
- whether it influences allocation, evaluation, pay, discipline, or employment;
- how unequal effects and psychosocial risk are reviewed; and
- who can stop the collection.
ISO 45003 supplies guidance for managing psychosocial risks arising from work design and organizational factors.C22-S02 Conformity does not prove psychological safety. It reinforces the obligation to treat surveillance, intensity, ambiguity, role conflict, and loss of control as production risks rather than private worker resilience problems.
Global distribution belongs inside the evidence
Exposure is not adoption. Adoption is not automation. Automation is not displacement. Displacement is not an employment forecast.
ILO task-exposure analysis distinguishes augmentation and automation potential and identifies unequal patterns across occupations and gender.C22-R02 A 2026 ILO–World Bank analysis across 135 countries finds that digital infrastructure and task composition can distribute disruption risks and augmentation opportunities differently.C22-R08
These are potential-exposure analyses, not realized software-work outcomes. They prevent a factory designed in one high-resource setting from speaking as if its conditions were universal.
Where lawful and appropriate, disaggregate evidence by relevant role, experience, disability, gender, age, location, contract, language, working arrangement, and infrastructure. Include contractors, outsourced contributors, maintainers, reviewers, support workers, and data labor when the work system depends on them.
An average improvement can coexist with concentrated harm. A local loss of access can disappear inside a global gain. Distribution is not a footnote to productivity; it is part of the decision.
The factory must preserve the right to revise the work
Human agency is not a soft value added after the architecture is complete. It is an operating property of the architecture.
The practical sequence is:
- bound the task and intended outcome;
- describe the work before and after;
- allocate human and machine authority;
- design credible verification;
- involve affected workers and representatives;
- protect learning, accessibility, and transition;
- constrain monitoring and provide recourse;
- measure output, quality, burden, well-being, access, skill, and distribution separately;
- decide to continue, adapt, constrain, or stop; and
- refresh the decision when the tool, work, people, evidence, or context changes.
F22.1 visualizes this as a loop, not a score. There is no fixed boundary at which work becomes “machine work.” The allocation remains provisional because capability, consequence, evidence, and human circumstances change.
A Software Factory worthy of the name should not merely make control more efficient. It should make the design of work visible, challengeable, measurable, and revisable by the people who carry its consequences.
Chapter 23 asks how far Digital Workers may act. Chapter 22 supplies the prior constraint: autonomy for software must not be purchased with unaccountable authority over people.