AI Could Democratize Science, But Is Not Without Risks
AI is influencing pretty much every discipline, so it makes sense that science would be no exception. In the endless debate over whether artificial intelligence will replace or merely assist human workers,. . .
AI is influencing pretty much every discipline, so it makes sense that science would be no exception. In the endless debate over whether artificial intelligence will replace or merely assist human workers, scientists have tended to occupy a comfortable middle ground. Their work, they reason, demands creativity, rigor, and deep domain knowledge—qualities that ought to keep them safely employed even as AI transforms other professions. Recentresearchsuggests this confidence may be misplaced, at least in certain corners of academia. An aide in the labThe researchers, from Northwestern University, developed SciSciGPT, an AI system designed to function as a research collaborator in the field known as the “science of science”—a meta-discipline that studies how scientific research itself works. The system represents an intriguing test case for AI’s potential to reshape how research gets done, whilst simultaneously illustrating the technology’s current limitations. SciSciGPT operates through a multi-agent architecture, whereby specialised AI modules handle different aspects of the research workflow. One agent searches academic literature, another wrangles data from sprawling scholarly databases, a third performs statistical analyses and generates visualisations, whilst a fourth evaluates the quality of the work produced. A central “ResearchManager” coordinates these specialists, breaking down user queries into discrete tasks and assembling the results into coherent answers. The system has achieved some interesting results. For instance, when it was asked to map the collaboration networks in elite universities, the technology was able to identify the right datasets, generate queries, and produce results in a polished visualization model, all within minutes. It’s a result that would have taken humans many hours, even if they used AI coding assistants. In another demonstration, the system successfully replicated findings from a published paper after being shown merely a screenshot of a figure, interpreting the visualization, extracting appropriate data, and recreating the analysis with remarkable fidelity. The adoption gapThat scientists are experimenting with such tools comes as little surprise. Asurveyof over 6,000 researchers at Germany’s Max Planck Society and Fraunhofer Society found that 42% currently use AI in their work, with over a quarter employing it daily. Most of this usage covers tasks, such as writing code, testing ideas, and creating first drafts of research papers. Typically, scientists are deploying it on the kind of tasks that take up a lot of time. The research also revealed, however, that female scientists are using AI much less than male scientists. This isn’t so much due to any capability issues but rather a general lack of familiarity. When women started to use AI, they found it just as useful as men did. This pattern suggests that systems like SciSciGPT could either democratise research or entrench existing inequalities, depending on institutional support and training. The survey also exposed how poorly prepared many researchers are for effective AI use. When asked to construct a prompt to identify a visual phenomenon, only 21% succeeded, which is a sobering indication that AI literacy remains nascent even among active users. Researchers cited legal uncertainty, lack of access to the best tools, and absence of institutional guidelines as major barriers. Fast food researchSuch automation offers obvious benefits. It accelerates early-stage research, the exploratory work that often consumes disproportionate time relative to its intellectual challenge. It lowers barriers to entry for researchers lacking advanced programming skills. And by documenting every step of its workflow, it potentially enhances reproducibility—a perennial challenge in computational social science. The productivity gains can be dramatic. Researchfrom UC Berkeley and Cornell University found that when scientists begin using AI, their output increases between 36% and 60%, with non-native English speakers seeing gains of up to 89%. Yet this surge comes with a troubling caveat: AI-generated complexity in academic writing now correlates negatively with publication success, the opposite pattern seen in human-written work. In other words, AI obscures weak scholarship behind elaborate language rather than enhancing genuinely rigorous research. The result is what researchers now derisively term “scientific AI slop. ”Similar limitations were acknowledged by the SciSciGPT team. They found that while expert reviewers lauded its speed and technical soundness, the large amount of documentation can easily overwhelm users. What’s more, it also exhibits the same behaviors as other LLMs, so it can go astray, as well as produce different outputs even when identical prompts are used, which is far from ideal in a world in which reproducibility is key. The discovery ceilingThese shortcomings reflect broader patterns in scientific AI. A comprehensivestudyinvolving researchers across biology, chemistry, materials science, and physics found that state-of-the-art models achieve 60-75% accuracy on genuine scientific discovery tasks—respectable but markedly worse than their 84-86% performance on conventional science exams. The gap reveals an uncomfortable truth: excelling at standardised tests does not guarantee readiness for discovery. More troubling still, the strategies driving recent AI breakthroughs, such as scaling model size and increasing reasoning effort, show diminishing returns on scientific tasks. Between medium and high reasoning efforts, accuracy barely budges. Leading models from different companies exhibit remarkably correlated error patterns, suggesting they share systematic weaknesses inherited from similar training data. Models excel at projects involving well-structured data and codified knowledge but struggle when projects require rigorous long-term planning, such as multi-step chemical synthesis. Indeed, when models did well on individual tasks, this wasn’t a reliable indicator that they would thrive on more complex projects. The democratization dilemmaThis raises uncomfortable questions about scientific democratisation. SciSciGPT’s advocates argue it will broaden participation by enabling researchers with limited technical skills to explore data effectively. Yet as AI becomes embedded in everyday research tools, which the German survey shows is already happening, those same researchers risk never developing the analytical capabilities needed to verify, challenge, or refine AI-generated insights. The Northwestern researchers suggest a way around some of these challenges is via a “capability maturity model,” which creates a pathway from basic functional models towards more sophisticated AI tools. At the moment, they believe that SciSciGPT sits on the lower rungs of this ladder. Perhaps more importantly, there are the questions AI raises in terms of what actually constitutes research assistance in an AI age. For instance, a SciSciGPT analysis could ordinarily take a doctoral student a few days. Who would deserve the credit in such a scenario? Our traditional concepts of intellectual ownership are likely to strain under this new approach. After all, the German survey found that researchers already lack clear guidelines on AI usage, particularly in areas with legal uncertainty. For now, SciSciGPT remains firmly in the “augmentation” camp, empowering human researchers rather than replacing them. Its creators emphasise that it supports exploration and prototyping rather than final publication. But the trajectory seems clear. As language models improve, such systems will claim ever-larger portions of the research workflow. Optimists see this movement as making research both more effective and also open to more people. Pessimists, on the other hand, worry that we’re deskilling researchers and pushing a research agenda that is driven not by scientific need but by what AI is capable of. As is so often the case, the truth sits in between. SciSciGPT shows what AI is currently capable of, but also a reminder that the technology isn’t capable of “doing” science, at least not yet. So the key is largely one of governance, and how the scientific community responds to AI. This will determine whether AI supports or undermines research. The science of science must now grapple with the science of AI-assisted science. It is a recursion that would make any meta-researcher’s head spin.