Under AI Conditions 1.2.7-4

Why AI Intensifies the Institutional Design Problem

Answerability addresses a problem that existed before AI: institutions degrade, drift, and get captured — and the mechanism is architectural, not malicious. AI does not create institutional capture. It accelerates every mechanism that produces it, introduces new mechanisms that traditional institutional design did not need to anticipate, and compounds the difficulty of every structural response.

The intensification operates through five structural channels, each mapping to a component of the innovation.

Automated Authority Without Legible Decision-Making

Institutions increasingly delegate consequential decisions to AI systems — hiring recommendations, credit assessments, resource allocations, risk evaluations, content moderation, eligibility determinations. Each delegation transfers a slice of institutional authority from a human decision-maker to a computational process. The structural problem is not the delegation itself — institutions have always delegated decision-making — but the legibility of the delegated authority.

Legible authority requires that decision-making authority be visible to those affected: who made the decision, under what criteria, within what scope of discretion, and subject to what oversight. When a human decision-maker operates under institutional rules, the legibility requirement can be satisfied through documentation, policy manuals, and accountability relationships. When an AI system operates under learned patterns, the legibility requirement faces a structural obstacle: the system may produce decisions that its operators cannot explain in terms of the criteria they specified. The decision is made. It affects persons. And the chain of authority — from institutional purpose through policy through implementation through outcome — has a gap in the middle where computation replaced legible reasoning.

This does not mean AI-mediated decisions are inherently illegitimate. It means they require additional institutional design. Three requirements extend legibility to AI-delegated authority. First, scope specification: the institution must define what decisions the AI system is authorized to make, under what conditions, and within what boundaries — and these specifications must be Tier One (public by default). Second, outcome auditability: the institution must be able to evaluate whether the AI system's decisions are consistent with the criteria it was authorized to apply — and this evaluation must be Tier Two (auditable on request). Third, override pathways: the institution must maintain the structural capacity to reverse, modify, or countermand AI-generated decisions when they fall outside authorized parameters or produce outcomes that violate the institution's commitments — and these pathways must be accessible to the accountability layers that the article specifies.

The design challenge is not to make AI decisions transparent in the technical sense of explainable AI. It is to maintain institutional legibility: the affected person must be able to identify who is responsible for the decision, what criteria governed it, and how to challenge it. If the answer to any of these questions is "the algorithm," legibility has failed — not because algorithms are inherently opaque, but because institutional accountability requires that authority trace to persons who can be held responsible.

Accountability Collapse Through Delegation Chains

AI creates a structural opportunity for accountability evasion that traditional institutional design did not need to address: the delegation chain that diffuses responsibility across so many actors — developers, deployers, operators, institutional administrators, vendor contractors — that no single actor can be held accountable for the outcome.

The mechanism operates through what might be called accountability diffusion. The developer built the model; the vendor customized it; the institutional IT department deployed it; the policy team set the parameters; the AI system produced the decision; the affected person experienced the consequence. When the affected person seeks accountability, each actor in the chain can plausibly claim that the failure originated elsewhere. The developer did not set the parameters. The vendor did not know the institutional context. The IT department did not design the policy. The policy team did not understand the model's behavior. The result is a consequential decision affecting persons that was produced by institutional processes and that no one will own.

Layered accountability must be extended to cover AI delegation chains. The design requirement: every AI system operating within an institution must have a designated accountability owner — a human decision-maker with the authority, the information, and the institutional standing to answer for the system's outputs. The accountability owner is not necessarily the person who built or deployed the system. They are the person whose institutional role makes them responsible for the outcomes the system produces — and who has the authority to modify, constrain, or withdraw the system when it fails.

The accountability owner must be positioned within the accountability ladder: subject to operational oversight, governance review, external audit, and participant recourse. The structural test is the same one the article applies to all institutional authority: if the AI system produces a harmful outcome, can the affected person identify who is responsible and pursue recourse? If the answer is no — because the delegation chain diffuses responsibility beyond the reach of any accountability layer — the institution has delegated authority without keeping responsibility locatable. Where the delegated authority is public or state-compelled force, that failure breaches a moral requirement. Where the institution is voluntary, it is a failure against the institution's own design recommendation — one it has every rational reason to correct, since an accountability owner it cannot name is a decision it cannot defend to a counterparty either.

Selective Transparency Under Opacity Pressure

AI systems create structural pressure toward institutional opacity. A machine learning model's decision process may be genuinely difficult to explain in human-legible terms. Proprietary models may be protected by trade secrets or contractual restrictions. The technical complexity of AI systems may exceed the capacity of non-expert oversight bodies to evaluate.

None of these pressures excuses institutional opacity. Selective transparency must be maintained regardless of the technical complexity of the institution's decision-making infrastructure. The three-tier framework applies with full force.

At Tier One, the institution must disclose that it uses AI systems, what decisions those systems influence, what criteria they are designed to apply, and what oversight mechanisms govern their operation. An institution that uses AI systems to make consequential decisions without disclosing that fact has violated the most basic transparency requirement — the affected persons do not know that the authority affecting them has been delegated to a computational process.

At Tier Two, the institution must be able to demonstrate — to auditors, oversight bodies, and affected persons pursuing recourse — that its AI systems are operating within their authorized parameters and producing outcomes consistent with the institution's stated criteria. This does not require full model interpretability. It requires outcome auditing: measuring what the system does against what it was authorized to do, with sufficient granularity to detect systematic bias, drift, or deviation.

At Tier Three, the personal data that AI systems process must remain protected by design. The fact that an institution uses AI does not change the privacy requirements — if anything, AI's capacity for inference and pattern detection makes the privacy requirements more demanding. An AI system that ingests Tier Three data to produce Tier Two decisions must be designed so that the decision outputs do not leak the personal data they were derived from. Sidebar (1.2.4-4), on AI conditions for privacy by design, addresses the technical dimensions of this requirement.

Adversarial Testing at Machine Speed

The article's fourth component — embedded adversarial testing — faces a structural challenge under AI conditions: the systems being tested operate at speeds and scales that traditional adversarial testing was not designed to match.

A red-team review of a human-operated institutional function can be conducted on a quarterly or annual cadence because human decision-making operates at human speed. An AI system that processes thousands of decisions per day can drift, degrade, or be captured between review cycles. The adversarial testing architecture must be adapted to the tempo of the systems it monitors.

Three adaptations extend adversarial testing to AI-operated institutions. First, automated monitoring: continuous, rule-based surveillance of AI system outputs against authorized parameters — not surveillance of persons, but surveillance of institutional processes. When outputs exceed authorized boundaries, the monitoring system alerts the accountability layers. Second, adversarial probing: structured tests in which known inputs with known expected outputs are periodically submitted to the AI system to verify that it continues to behave as designed. This is the AI equivalent of the red-team exercise — a deliberate test of whether the system has drifted from its specifications. Third, capability auditing: periodic evaluation of what the AI system can do — not merely what it has been authorized to do. The distinction matters because AI systems may develop capabilities through training that their operators did not intend and that exceed the institutional authorization. A system authorized to recommend candidates may develop the capability to infer protected characteristics and use them in its recommendations. Capability auditing detects this divergence between authorization and capability before it produces institutional harm.

Capture Through Model Dependence

A final AI-era pressure deserves explicit treatment: the structural conditions under which an institution becomes dependent on an AI vendor's model — and through that dependency, becomes subject to the vendor's priorities rather than its own.

The mechanism parallels the quiet enclosure diagnosed in (1.1.1) and the dependency power analysis constructed in bounded interdependence (1.2.6). An institution integrates a vendor's AI model into its operations. Over time, institutional processes, data formats, and staff competencies align with the model's requirements. Switching to an alternative model becomes increasingly costly — not because the vendor has imposed contractual lock-in (though it may have) but because the institution's operational architecture has co-evolved with the model. The vendor's pricing, terms, and design choices now operate as constraints on the institution — constraints that the institution did not choose and that its accountability architecture was not designed to govern.

Answerability must account for AI vendor dependence as a capture risk. The design response draws on multiple innovations: bounded interdependence's exit-rights and portability requirements, the restraint principle's replaceability criterion, and this unit's own accountability architecture. The specific institutional design requirement: AI vendor relationships must be structured so that the institution can replace the vendor without losing institutional capability — the same replaceability standard that the restraint test (1.2.3) applies to all institutional components. If the institution cannot function without a specific vendor's model, the vendor has become a chokepoint in the institutional architecture, and the institution's governance has been partially outsourced to an entity outside its accountability framework.