In development · Research status: evidence review and method development. The dimensions and indicators below are being tested, not settled. Several are proposals that have not been validated, and none has a universal threshold.

Responsible AI & Human Agency · Framework · Methods & measures

Measuring Human Agency in AI Systems

How do we know whether people still have meaningful agency when they use — or are affected by — AI?

By Daphne Iris van Vliet

“Fine. How would you know?”

The framework makes a normative claim: responsible AI must preserve human agency. The first question any sceptical organisation, auditor or regulator will ask is how you would know. This page is the answer in development. It sits underneath the human layer of the framework — retained capability, appropriate reliance, meaningful oversight, autonomy and choice, dignity and relational integrity, and human impact — and asks what evidence would establish each claim.

Do not ask only whether people feel able to exercise agency. Test whether they can.

A person may say they understand an AI system and still fail to identify its limits. They may say they can challenge it and still accept deliberately wrong recommendations. They may believe they are in control while workload, interface design, targets or the lack of an alternative make disagreement unlikely. Most responsible AI governance stops at declared human control. This method aims for demonstrated human agency.

From perception to outcome

PerceptionWhat do people believe?
KnowledgeWhat do they understand?
CapabilityWhat could they do?
BehaviourWhat do they actually do?
OutcomeWhat changes as a result?

All five are shaped by organisational conditions — time, authority, workload, incentives, alternatives — and by repeated AI use over time.

The chain exposes the gaps that ordinary assurance misses:

Perception–capability gap: people say they could challenge an output, but fail a challenge task.
Capability–behaviour gap: they can technically override, but rarely do, because of workload or incentives.
Behaviour–outcome gap: overrides happen, but usually make decisions worse.
Institutional gap: someone detects an error but has no authority to change the outcome.
Longitudinal gap: oversight works at first, but the underlying skill fades with sustained automation.

Different evidence establishes different claims

Claim	Weak evidence	Stronger evidence
People understand the AI	They say they do	They correctly identify its limits, failure conditions and when to escalate
Humans remain in control	Policy says they can override	Seeded wrong recommendations are actually rejected or corrected
AI supports expertise rather than replacing it	Professionals feel competent	Unaided performance stays within an agreed margin after sustained use
People can challenge decisions	An appeal link exists	People can find it, complete it, reach a competent human and get a wrong decision corrected
Users rely appropriately	Average trust score	Correct advice is accepted and wrong advice resisted, compared with human-only and AI-only baselines
Oversight is meaningful	A reviewer is assigned	The reviewer has time, evidence, competence and authority — and intervention tests show the workflow changes when it should

Surveys are not bad evidence; they are evidence of perception. This maps directly onto the framework's evidence levels: a policy is E0, a seeded-error test is E1, production override data is E2, an independent retest is E3, and evidence from the people doing or receiving the work is E4.

Six candidate dimensions

The evidence review suggests six dimensions. They are a working hypothesis: pilots will test whether they are empirically distinct, or whether some — such as challenge capacity and appropriate reliance — should be merged.

Understand & orientKnow the AI's role, scope, limits, uncertainty and when to escalate — shown, not self-rated.
Judge & rely appropriatelyForm an independent judgment, use good AI advice and resist bad advice.
Challenge & interveneDetect trouble and actually change what happens — with the time, information and authority to do so.
Contest & obtain remedyFor affected people: notice, a way to add context, a reachable human, correction and remedy.
Retain capability & autonomyStill able to do the work, and to choose, when AI is unavailable.
Sustain human conditionsWorkload, wellbeing, accessibility, voice and incentives that make formal control usable.

Two people, two kinds of agency

Most human-oversight models focus on the operator. But in hiring, credit, healthcare, welfare and customer service, the person most affected by an AI-supported decision may never touch the system. A tool can feel empowering to a recruiter while leaving the applicant with very little agency.

The operator or decision-makerUnderstand → judge → challenge → intervene → retain capability

The affected personNotice → understand → provide context → reach a human → contest → correct → obtain remedy

For affected people, the shift is from formal rights to exercisable rights. “Users can appeal” is not the same as showing that people know they can appeal, can find and use the route without disproportionate burden, reach an empowered human, correct wrong information and obtain timely remedy. This connects human agency to digital human rights, not only to workplace design. GDPR Article 22 is one legal example: for solely automated decisions with legal or similarly significant effects, it requires safeguards including human intervention and the right to contest.¹⁴

What can already be measured

Each measure carries one of four labels: Validated substantially validated for this construct and context; Adapted a validated instrument that needs revalidation for AI use; Experimental empirically supported, but without standard interpretation or thresholds; Proposed a governance indicator I have synthesised, not yet tested. Validated never means universally thresholded.

Construct	Instrument or method	Status	Limit
Perceived trust	Trust in Automation Scale and its three-item short form, validated for AI applications¹	Validated	Measures attitude, not whether reliance is correct. Never a success target.
Appropriate reliance	Relative AI reliance and relative self-reliance: accepting correct advice and resisting wrong advice, measured separately²	Experimental	Needs cases with known correct answers; no universal threshold.
Situation awareness	SAGAT — objective queries about task state, supported by a meta-analysis of 243 studies³	Adapted	Queries must be built and validated for each oversight task.
Review workload	NASA Task Load Index⁴	Adapted	No score at which oversight is known to fail.
Job autonomy	Work Design Questionnaire⁵	Adapted	Measures work design generally, not AI-specific authority.
Felt authority and impact	Psychological Empowerment Scale⁶	Adapted	Feeling empowered can coexist with having no real permissions.
Voice and procedural fairness	Organizational Justice Scale⁷	Adapted	Perceived voice needs behavioural evidence of influence.
Wellbeing	WHO-5, reviewed across 213 studies⁸	Adapted	Changes cannot be attributed to AI without a comparison.
Explanation quality	Explanation satisfaction and mental-model measures⁹	Adapted	Satisfaction with an explanation does not prove understanding.
AI literacy	Existing scales: a review found 16 scales across 22 validation studies, with none showing positive evidence on every quality criterion assessed¹⁰	Adapted	Mostly self-report; cannot replace a task-specific comprehension test.
Challenge capability	Seeded-error detection: realistic wrong recommendations mixed into the work	Experimental	Seeded cases must reflect real failure modes and harm.
Retained capability	Repeated unaided performance against a pre-AI baseline	Experimental	Learning, case mix and changing work complicate attribution.
Effective correction	Override effectiveness: the share of overrides that improve the decision — not override frequency	Proposed	Needs ground truth or expert review.
Effective contestability	Findability, completion, time and cost to human review, correction and remedy	Proposed	High reversal can mean a working appeal route — or a poor upstream system.

Why supported performance is not retained capability

Medicine. In a multicentre observational study, endoscopists' unaided adenoma detection fell from 28.4% to 22.4% after regular AI exposure.¹¹ A 2026 scoping review found evidence of clinical deskilling scarce but consistent across specialties.¹² This is a risk signal, not a proven causal law.
Education. In a field experiment with high-school mathematics students, AI assistance improved performance during practice, but unrestricted access harmed later performance without AI. A tutor designed to give hints rather than answers mitigated the loss.¹³
Clinical judgment. In a study of 223 dermatologists, higher trust was associated with greater reliance on AI — including lower self-reliance when the dermatologist was right and the AI was wrong.¹⁵
Hiring. In a resume-screening experiment with 528 participants, recommendations from simulated race-biased AI shifted people's candidate choices toward the AI's preference.¹⁶ It is a preprint, so treat it as emerging evidence — but it shows why “a human reviews it” cannot be assumed to neutralise bias.

None of this means AI harms competence, or should be rejected. It means AI-supported performance and unaided capability are different outcomes, and must be measured separately.

Fourteen indicators for piloting

A minimum set to test in pilots. They are deliberately not combined into a score.

Indicator	How	Status	Triggers review when…
Demonstrated limitation comprehension	Scenario test of scope, limits, uncertainty and escalation, alongside self-rating	Adapted	People say they understand but fail safety-critical items
Independent judgment	Record the human decision before showing the AI, on a sample	Experimental	Judgments converge on the AI, especially when it is wrong
Seeded-error detection	Mix realistic wrong outputs into the work	Experimental	Detection falls below a margin agreed in advance
Appropriate reliance pair	Accepting correct advice and resisting wrong advice, measured separately	Experimental	One improves while the other deteriorates
Harmful acceptance	Share of known wrong recommendations acted on	Experimental	Any high-severity case — whatever the average accuracy
Override effectiveness	Share of overrides that improve the outcome, and harmful overrides	Proposed	Effectiveness falls. Override counts are never a target.
Intervention and escalation	Drills: time to detect, stop, escalate and reach someone empowered	Proposed	Intervention is slower than harm can develop
Oversight capacity	Workload measure plus time per case, caseload and missed cases	Adapted	Workload rises while detection falls
Contestability effectiveness	Findability, completion, abandonment, time to human review, correction, remedy	Proposed	The human route is missing, inaccessible or cannot correct known errors
Unaided capability retention	Repeat representative tasks without AI against a baseline	Experimental	Decline beyond a margin agreed in advance
Autonomy and usable choice	Autonomy measures plus a real test of opting out and its consequences	Adapted	An option exists on paper but carries friction or penalty
Workload and wellbeing trend	Validated workload and wellbeing measures, by cohort	Adapted	Material deterioration plausibly linked to the deployment
Accessibility and challenge parity	Completion of core, opt-out and appeal tasks across relevant groups	Proposed	Any group cannot use a core human or appeal route
Voice with demonstrated influence	Record input, the organisation's response and what changed	Proposed	Participation repeatedly leaves no trace in decisions

How to read the evidence

No number means anything on its own. A very low override rate may mean the AI is excellent — or that reviewers have become passive. A high appeal-reversal rate may mean the appeal route works — or that the upstream system is poor. Zero complaints may mean zero harm — or that people do not know how to complain. High trust may reflect strong evidence — or persuasive design. Measures acquire meaning only through triangulation.

Thresholds follow five rules rather than generic cut-offs:

Baseline first — measure human-only or pre-AI performance before deployment.
Commit in advance — define acceptable deterioration before seeing the results.
Match the stakes — a drafting assistant and clinical triage do not share a threshold.
Weight by harm — a rare catastrophic failure is not averaged against many trivial successes.
Read in pairs — workload with detection, agreement with correctness, appeals with accessibility.

The result is an evidence profile, not a score. For example: Retained capability — demonstrated (E2 behavioural, E4 corroborated). Appropriate reliance — partly evidenced (E1 only). Meaningful oversight — red flag: override authority exists, but realistic workload testing shows reviewers cannot scrutinise outputs.

Measurement must not become surveillance

Never create more intrusive monitoring to prove that AI is human-centred than is needed to test the specific human-risk claim.

Prefer controlled samples over continuous individual telemetry.
Report at team or cohort level; never rank individual employees.
Keep assurance evidence separate from performance management, and never reuse “agency metrics” as adoption quotas.
Collect the minimum data, with retention limits, and involve workers or their representatives.
Watch for reactivity, gaming, Goodhart effects and selection bias — the people most burdened by AI are often least likely to answer a survey.

The GDPR's principles of purpose limitation and data minimisation set the legal baseline wherever personal data is involved.¹⁴

Stress tests in four settings

Setting	What conventional metrics miss	Key test
Clinical decision support	Reviewers appear to review but follow wrong advice; skills erode	Can a clinician resist a plausible wrong recommendation — and still perform safely after long AI use?
Recruitment	Reviewers inherit AI bias; applicants cannot see or challenge the decision	Does a recruiter resist a biased recommendation — and can an applicant get meaningful reconsideration?
Education	Better answers today, weaker learning tomorrow	Does the learner still have the capability when AI is unavailable?
Customer service	Fast automated resolution hides an inaccessible route to a human	Can a frustrated or vulnerable customer leave the automated path early enough?

In customer service, relational design matters too: human-like cues in chatbots can increase compliance with their requests,¹⁷ while for customers who are already angry, anthropomorphism lowered satisfaction.¹⁸

Where measurement is mature — and where it is not

Validated foundationsPerceived trust · workload · situation awareness · job autonomy and empowerment · procedural fairness · wellbeing

Adaptable but incompleteExplanation satisfaction · AI literacy · perceived understanding · voice · autonomy under AI

Emerging empirical measuresAppropriate reliance · automation-bias resistance · seeded-error detection · independent judgment · skill retention · dependence

Genuine gapsMeaningful oversight as one construct · practical contestability · affected-person agency · relational dignity · perceived versus actual control · incentive effects · long-term dependence

Where the research shows that part of human agency cannot yet be measured reliably, that finding belongs here too. It is not a failure of the project; it is what makes the method honest.

Research agenda

Construct validation — are the six dimensions distinguishable, or do some overlap?
Criterion validity — do seeded-error and reliance tests predict real-world incident prevention?
Longitudinal capability — baselines, exposure measures and comparison groups.
Affected-person measurement — a practical contestability protocol.
Organisational causation — which conditions turn formal override authority into exercised authority?
Accessibility and measurement invariance across languages, disabilities and backgrounds.
Threshold validation — testing baseline-relative and harm-weighted rules, not inventing cut-offs.
Measurement ethics — a data-protection assessment for every indicator.

Human agency is demonstrated when people have the understanding, retained capability, practical authority and institutional support to form judgment, use AI selectively, detect and challenge failure, change consequential outcomes, contest decisions that affect them — and keep functioning when AI is unavailable. No survey, override button or workflow diagram can establish that on its own.

Method and limits

A structured evidence review, not a registered systematic review or meta-analysis, prepared in October 2026. Instrument statuses describe suitability for measuring human agency around AI, not whether a parent instrument has ever been validated. Several figures from my working research could not be matched to a verifiable public record and were left out. The six dimensions, the fourteen indicators, the threshold rules and the evidence profile are my proposals and have not been validated.

Sources

Validation study McGrath, M. J. et al. (2025). Measuring trust in artificial intelligence: validation of an established scale and its short form. Frontiers in Artificial Intelligence. doi:10.3389/frai.2025.1582880. ↩
Experimental Schemmer, M. et al. (2023). Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations. Proceedings of IUI 2023. doi:10.1145/3581641.3584066. ↩
Meta-analysis Endsley, M. R. (2021). A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness. Human Factors, 63(1). doi:10.1177/0018720819875376. ↩
Instrument Hart, S. G. & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index). Advances in Psychology, 52, 139–183. doi:10.1016/S0166-4115(08)62386-9. ↩
Validation study Morgeson, F. P. & Humphrey, S. E. (2006). The Work Design Questionnaire (WDQ). Journal of Applied Psychology, 91(6), 1321–1339. doi:10.1037/0021-9010.91.6.1321. ↩
Validation study Spreitzer, G. M. (1995). Psychological empowerment in the workplace: dimensions, measurement and validation. Academy of Management Journal, 38(5). doi:10.2307/256865. ↩
Validation study Colquitt, J. A. (2001). On the dimensionality of organizational justice: a construct validation of a measure. Journal of Applied Psychology, 86(3), 386–400. doi:10.1037/0021-9010.86.3.386. ↩
Systematic review Topp, C. W. et al. (2015). The WHO-5 Well-Being Index: A Systematic Review of the Literature. Psychotherapy and Psychosomatics, 84(3). doi:10.1159/000376585. ↩
Methods Hoffman, R. R. et al. (2023). Measures for explainable AI. Frontiers in Computer Science, 5. doi:10.3389/fcomp.2023.1096257. ↩
Systematic review Lintner, T. (2024). A systematic review of AI literacy scales. npj Science of Learning, 9. doi:10.1038/s41539-024-00264-4. ↩
Observational Budzyń, K. et al. (2025). Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy. Lancet Gastroenterology & Hepatology, 10(10), 896–903. doi:10.1016/S2468-1253(25)00133-5. ↩
Scoping review Heudel, P. E. et al. (2026). Artificial intelligence in medicine: a scoping review of the risk of deskilling and loss of expertise among physicians. ESMO Real World Data and Digital Oncology, 12. doi:10.1016/j.esmorw.2026.100693. ↩
Field experiment Bastani, H. et al. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. PNAS. doi:10.1073/pnas.2422633122. ↩
Legislation Regulation (EU) 2016/679 (GDPR), Articles 5 and 22. eur-lex.europa.eu. ↩
Experimental Küper, A. et al. (2025). Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists. Journal of Medical Internet Research, 27, e58660. doi:10.2196/58660. ↩
Preprint No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy (2025). arXiv:2509.04404. arxiv.org. ↩
Experimental Adam, M., Wessel, M. & Benlian, A. (2021). AI-based chatbots in customer service and their effects on user compliance. Electronic Markets, 31. doi:10.1007/s12525-020-00414-7. ↩
Experimental Crolic, C. et al. (2022). Blame the Bot: Anthropomorphism and Anger in Customer–Chatbot Interactions. Journal of Marketing, 86(1). doi:10.1177/00222429211045687. ↩