How AI Assistance Affects Human Skill Development: A Study of Learning with Logic Puzzles
2026-08-24 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors studied how using AI help affects learning in logic puzzles. They found that when AI help is easy to get, people use it more, but this can make their skills worse when the AI is no longer available. The researchers also showed that people who try harder on their own while using AI improve their skills more. Overall, the study suggests that relying too much on AI might stop people from getting better at solving problems by themselves.
AI assistanceskill developmentlogic puzzlestask performanceBayesian latent ability modelindependent reasoningAI request costlatent abilityexperimental study
Authors
Shang Wu, Catarina G Belem, Shuyuan Fu, Mark Steyvers, Padhraic Smyth
Abstract
While AI assistance can improve human task performance in the short term, it may also undermine the development of skills in the longer term. We examine this tension in a controlled logic-puzzle experiment involving on-demand AI assistance, where participants complete tasks before, during, and after AI is available. By experimentally varying AI request costs, we find that lower-cost assistance induces more frequent AI use. We also find that participants who request AI assistance during the AI-access phase perform worse at the task after assistance is removed, and their subsequent unassisted performance is overestimated when predicted from earlier AI-assisted performance. We use a Bayesian latent ability model to separate initial ability, post-AI ability, and participant-specific skill change, while estimating how independent reasoning during the AI-access phase relates to skill development. The results show that greater independent problem-solving effort is associated with larger gains in latent ability, consistent with the interpretation that skill development is weaker when AI assistance substitutes for independent reasoning.