Preference for Explainable AI, NBER Working Paper 35240 by Alex Chan, studies participants acting as loan officers on real $10,000 loans from a US lender. The May 2026 paper finds that explanations revealing demographic penalties made participants more likely to override the AI recommendation. When bonuses depended on repayment, participants sought predictions but avoided explanations under some experimental conditions.
A secondary experiment examines difficulty recognising when explanations improve decisions. The research concerns lending, not insurance claims or benefits eligibility. Any application to benefits is an editorial governance question, rather than a claim that the experiment demonstrates the same behaviour in healthcare or HR.
Examine whether explanations are actually used This editor recommends that organisations distinguish making an explanation available from incorporating it into a decision process. The team should identify who reviews a tool’s output, what information that reviewer receives and which situations require additional examination. A supplier’s explanation feature is only one element of that operating arrangement.
The organisation can also review incentives. A performance target may shape which information someone chooses to inspect. The review should ask how quality, accuracy and appropriate escalation are recognised alongside speed or financial results. Those questions should be addressed in the organisation’s governance process rather than left entirely to the individual using the tool.
Define human review in practical terms For a benefits-related application, a proposed review process should explain who can challenge an output and how the challenge is handled. The organisation needs to understand what evidence is retained, what a reviewer can change and when a matter reaches a qualified specialist. Personal information used in that process should have a legitimate purpose and appropriate controls.
Evaluation can test whether the process is followed and whether reviewers understand the information supplied. This editor recommends keeping that evidence separate from a claim about the model’s predictive performance. A tool can be accurate on a selected measure while the surrounding decision process still requires improvement.
The paper makes an important distinction visible: people may value a prediction differently from an explanation, especially when their incentives differ. An organisation considering AI should therefore assess the decision process, incentives and review arrangements together, using evidence from its own application rather than assuming that technical explainability alone completes governance.
Sources: Source primaire ou référence du sujet [1]
Share this:
- More [3]
