Source: Zhang & Narayandas (2026) — “Engaging Customers with AI in Online Chats: Evidence from a Randomized Field Experiment,” Management Science
Online chat is where customer patience goes to be tested.
You open the little bubble. The bot says hello. You explain your problem. The bot asks you to explain it again in a slightly different way. Somewhere around message four, your soul leaves your body.
So I am always interested in studies that test AI chat in the real world, not just in a demo where everyone behaves nicely.
This paper looks at engaging customers with AI in online chats through a randomized field experiment. The practical question is simple: can AI improve customer engagement in service conversations?
It probably can, but the hard part is not just response speed.
AI can answer quickly, scale support, and handle common questions. That is useful. But customer engagement depends on whether the interaction feels relevant, capable, and worth continuing. A fast bad answer is still bad. It is just bad with excellent latency.
The handoff also matters.
When should AI continue? When should a human step in? How should customers understand who they are talking to? What happens when the conversation moves from routine information to frustration, ambiguity, or money?
These are design questions, not just technology questions.
Companies often want AI chat because it reduces cost. Customers want AI chat only if it reduces effort. Those are not the same goal.
When the system is designed around cost alone, customers feel trapped. When it is designed around resolution, AI can become genuinely useful.
My takeaway: AI chat should be measured by whether it helps customers move forward, not whether it successfully keeps them away from humans. The best service automation knows when to answer, when to ask, and when to gracefully get out of the way.