Evaluators who received AI recommendations with written explanations were more likely to reject promising innovations than those who got no explanation at all, according to a new study from researchers at Harvard Business School, MIT, and the University of Washington. The findings challenge the widespread belief that AI-generated rationales improve human decision-making. Instead, the report shows that narrative explanations can suppress independent human judgment, causing reviewers to defer to incorrect AI decisions rather than verifying them on their own.
The study asked 228 seasoned evaluators to review nearly 50 submissions from an MIT innovation challenge under three conditions: no AI assistance, AI recommendations with written rationales, and black-box AI pass-fail decisions without explanation. Their choices were measured against decisions made by four human experts, treated as the correct baseline. Evaluators accepted LLM recommendations 67% of the time overall. They aligned with both black-box and narrative AI decisions roughly 75% of the time, but agreed with human expert decisions only 54% of the time. When AI suggested rejection with an accompanying rationale, evaluators disproportionately complied, cutting false positives but substantially raising false negatives—meaning they passed on ideas the experts judged promising. Black-box recommendations improved decision quality by aligning choices with expert judgment, but recommendations paired with narratives did not.
"Our findings reveal that LLM explanations do not necessarily improve decision-making," the researchers wrote. "Effective human-AI collaboration requires designs that preserve rather than supplant independent human judgment." The report explains that narrative explanations suppress productive overrides, where evaluators independently verify persuasive model outputs before deciding. LLMs provide convincing arguments that are easy to accept, effectively discouraging human verification. The researchers noted that people are cognitively wired to weigh negative information more heavily than positive—a tendency called negativity bias—and LLM explanations offer ready-made justifications for rejection decisions without independent checking.
The study attributes the effect to what it calls an "illusion of explanatory depth." LLMs are linguistically fluent and sound expert-like, so evaluators rely on surface cues such as coherence and seeming credibility, leading them to overestimate their understanding of a decision despite limited insight into its reasoning. Because rejecting a proposal feels more consequential and accountable than keeping options open—and maintains the status quo while avoiding risk—humans effectively offload their thinking to AI when given a plausible narrative. The researchers argue that in contexts like early-stage innovation screening, LLM narratives undermine performance by discouraging independent judgment, whereas simpler or more opaque recommendations preserve human discretion and verification.
The report recommends that enterprises treat AI explanations as behavioral interventions whose effects depend on how evaluators process information under uncertainty, not as universally helpful transparency tools. Organizations should test AI recommendations before deployment, and consider models that present contrasting narratives—reasons to reject alongside reasons to accept—or disclose uncertainty rather than binary decisions. In settings like quality control or fraud detection, LLM explanations could support conservative decision-making, but in early-stage screening they risk suppressing the productive disagreement needed to identify high-potential ideas. The bottom line: systems should be designed to invite human disagreement, not replace it. For executives weighing how much autonomy to cede to algorithmic advisors, the calculus isn't purely technical—it's about whether the architecture of collaboration leaves room for the friction that catches errors and surfaces outliers.