Are generative artificial intelligence systems such as ChatGPT truly creative? A research team led by Professor Karim Jerbi from the Department of Psychology at the Université de Montréal, and including AI pioneer Yoshua Bengio, also a professor at Université de Montréal, has just published the largest comparative study ever conducted on the creativity of large language models versus humans.
Published in Scientific Reports (Nature Portfolio), the findings reveal that generative AI has reached a major milestone: it can now surpass average human creativity. However, the most creative individuals still clearly outperform even the best AI systems.
AI reaches the threshold of average human creativity
The study tested the creativity of several large language models (including ChatGPT, Claude, Gemini, and others) and compared their performance with that of 100,000 human participants. The results mark a turning point: some AI models, such as GPT-4, now exceed the average creative performance observed in humans on tasks of divergent linguistic creativity.
“Our study shows that some AI systems based on large language models can now outperform average human creativity on well-defined tasks,” explains Professor Karim Jerbi. “This result may be surprising — even unsettling — but our study also highlights an equally important observation: even the best AI systems still fall short of the levels reached by the most creative humans.”
Analyses conducted by the study’s two co-first authors — postdoctoral researcher Antoine Bellemare-Pépin (Université de Montréal) and PhD candidate François Lespinasse (Université Concordia) — reveal a new and intriguing reality. While some generative AI systems now surpass average human creativity, the highest levels of creativity remain distinctly human.
In fact, the average performance of the most creative half of participants exceeds that of all AI models tested, and the top 10% of the most creative individuals open an even wider gap.
“We developed a rigorous framework that allows us to compare human and AI creativity using the same tools, based on data from more than 100,000 participants, in collaboration with Jay Olson from the University of Toronto,” says Professor Karim Jerbi, who is also an associate professor at Mila.