Source: Wang, Zhang & Lu (2025) — “The Power of Disagreement: A Field Experiment to Investigate Human-Algorithm Collaboration in Loan Evaluations,” Management Science
Human-algorithm collaboration sounds lovely until the human and the algorithm disagree.
Then suddenly everyone becomes very philosophical.
Do we trust the model? Do we trust the human? Do we ask for another dashboard? Do we form a committee and lose the will to live?
This paper studies disagreement in loan evaluations, which is exactly the kind of setting where the stakes are high and the phrase “just follow the algorithm” feels a little too casual.
The researchers ran a field experiment where human evaluators and algorithms worked together to assess loan applications. The key question was not whether humans should always follow or always override the model. It was whether humans could disagree at the right time.
That phrase matters.
Bad disagreement is just ego with a spreadsheet. The human ignores the algorithm because they “have a feeling.” Very cinematic, not always useful.
Good disagreement happens when the algorithm is wrong and the human catches it.
The study finds that human-algorithm collaboration can outperform either humans or algorithms alone, but the value depends on how disagreement forms and whether it is effective. One especially interesting finding is that algorithm self-contradiction can trigger useful disagreement. When the model’s rationale or signals do not fully line up, humans become more alert and are better able to challenge the recommendation at the right moment.
I love this because it reframes disagreement as a feature, not a failure.
The goal of human-in-the-loop systems is not blind agreement. If the human always agrees with the algorithm, why is the human there? Decoration?
My takeaway: collaboration value comes from structured friction. The best systems do not just tell humans what the algorithm thinks. They help humans notice when the algorithm may be wrong. That is where judgment earns its seat.