New research from the MIT Media Lab suggests that while AI chatbots can help people spot misinformation in the moment, their regular use does not necessarily lead to learning, and may even weaken our ability to detect it on our own.
Work for a Member organization and need a Member Portal account? Register here with your official email address.
New research from the MIT Media Lab suggests that while AI chatbots can help people spot misinformation in the moment, their regular use does not necessarily lead to learning, and may even weaken our ability to detect it on our own.
AI chatbots are quickly becoming a major part of how people search for information, check facts, write, study, and make everyday decisions. But when people are using these systems are they actually learning how to make better judgments themselves, or are they offloading the work in such a way that they weaken their own skills?
In a new study from the MIT Media Lab, researchers investigated whether conversations with an AI chatbot could help people become better at detecting misinformation over time. They found that while AI chatbots helped people spot misinformation in the moment — people did not necessarily get better at spotting such information on their own. In fact, some people got worse the more they used AI chatbots.
In a month-long study, researchers at the MIT Media Lab had 67 participants use an AI chatbot similar to ChatGPT, Gemini, and Claude. Across three sessions spread out over four weeks, participants repeatedly completed the same sequence of tasks:
This design allowed the researchers to separate two different effects: how much AI improves performance during assistance, and whether assistance from AI leads people to become better at the task themselves after assistance.
When participants interacted with the AI chatbot, their accuracy improved substantially. Across sessions, AI assistance increased accuracy by about 21 percentage points on average on a scale from 0 to 100% accurate. In other words, the chatbot was effective at helping people reach the correct answer in the moment. It often succeeded in correcting misjudgments and increasing confidence on the item under discussion.
That finding matters. It shows that conversational AI can be useful for immediate belief correction, especially in contexts where people are uncertain or initially mistaken.
But that was only part of the story.
The central question of the study was whether these improvements turned into lasting skills.
On average, they did not.
Participants’ unassisted performance on judging new news items did not improve over the course of the study. More strikingly, their performance immediately after using AI declined over time. By week four, people were doing worse on their own after using AI than they were at the beginning of the study.
This decline was especially driven by difficulty identifying fake content. Accuracy for real news remained relatively stable, but participants became less effective at spotting fake news items without AI help.
These results suggest an important distinction between AI use that increases performance in the moment versus AI use that leads to improvements in discerning abilities. AI support improved the former, but it did not reliably produce the latter.
To understand why some interactions helped more than others, the researchers analyzed thousands of human-AI conversations from the study, with a key pattern emerging.
When the AI immediately gave away the answer, or mainly presented facts for the user to accept, people tended to do worse later when left to make decisions on their own. These more directive interactions were associated with worse independent performance.
By contrast, interactions that were associated with improvements in skill were ones in which the AI slowed down and guided the user through the reasoning process. Strategies such as AI asking focused questions, probing the user’s thinking, and walking step by step through how to assess a news item were linked to stronger unassisted performance afterward.
This difference is important. It suggests that the problem is not simply “using AI,” but how AI is used. Systems that act like authoritative answer machines may help people complete a task, while systems that structure reflection and reasoning may be better suited for learning. When should AI systems be optimized for performance, learning or both?
Although this study focused on misinformation detection, the implications extend far beyond it. AI tools are rapidly becoming part of how people write, research, study, and make everyday decisions. The key issue is not simply whether AI helps people get the right answer in the moment, but whether giving it that task strengthens or weakens people’s own ability to do it on their own.
This places responsibility on how these systems are designed and used. Tools that mainly provide quick answers may improve short-term performance but at the cost of people’s own (thinking) abilities. By contrast, systems that guide users through questions, explanations, and step-by-step evaluation are more likely to support learning and independent judgment but might come with an efficiency and performance trade-off.
As AI becomes more embedded in society, developers, institutions, and policymakers will need to consider the effects that AI might have on our abilities to think critically and complete certain tasks independently from AI. When is it okay for AI to offload a task without learning, and when is it necessary that people simultaneously learn or do not lose a skill. Evaluating systems only on efficiency or accuracy may miss an important question: whether widespread AI use ultimately strengthens human capability or slowly replaces it.
This study is part of a larger body of work on how AI shapes human reasoning and critical thinking, including research showing that deceptive AI reasoning can amplify belief in misinformation, how question-asking systems can better scaffold people’s reasoning than answer-giving systems, and how AI might be designed to strengthen rather than replace human judgment. The work is part of the MIT Media Lab’s Advancing Humans with AI (AHA) initiative, which focuses on building AI that supports human flourishing. Readers can explore related projects on deceptive AI, question-based AI reasoning support, and reasoning augmentation research.
For more information, read the full paper.