Source: Chen & Chan (2024) — “Large Language Model in Creative Work: The Role of Collaboration Modality and User Expertise,” Management Science
There is a big difference between asking AI to think with you and asking AI to think instead of you.
This paper made that difference very hard to ignore.
The study looks at creative advertising work and separates two ways of using an LLM. In one mode, the AI acts like a ghostwriter: it generates the content and the human edits it. In the other, the AI acts like a sounding board: the human writes first, then the AI critiques.
Same tool. Same general task. Very different outcomes.
The finding I keep coming back to is that expertise changes the value of AI. For non-experts, AI critique can improve quality because it points out weaknesses they might not see. That feels intuitive. If you are learning, a good second opinion helps you notice what your eye has not yet learned to catch.
But for experts, letting AI write the first draft can be dangerous. Not because experts become less capable overnight. Because the first draft quietly anchors the work. Once AI sets the shape, the human may spend more energy polishing than thinking.
This is extremely relatable. Anyone who has edited a bad first draft knows the trap. At some point you are no longer asking, “Is this the right idea?” You are asking, “Can I make this sentence less embarrassing?” Different problem entirely 😭
The practical lesson is not “use AI” or “do not use AI.”
It is: choose the collaboration mode carefully.
If you need help seeing blind spots, use AI as a critic. If you need originality, judgment, or taste, be careful about handing over the first move.
My takeaway: AI can make creative work better, but only if it stays in the right seat. Sometimes the best role for AI is not writer. It is the annoying but useful person asking, “Are you sure this is actually good?”