AI scientist aims to automate research but raises ethics concerns

The new artificial intelligence model, developed by Sakana AI and called AI Scientist, is designed to automate research processes. It allows for generating ideas, conducting experiments, and writing and reviewing scientific publications. Although it is not flawless, this model is capable of creating hundreds of articles weekly. Its shortcomings could pose risks to the ethics of scientific research and overall safety.

AI Scientist generates scientific articles through a process entirely managed by artificial intelligence. As explained by its creators, including experts from Oxford and the University of British Columbia, in a preprint published on the "arXiv" platform, the program conducts successive stages of scientific research, from brainstorming and analyzing similar publications to creating and reviewing results.

"The AI Scientist can generate hundreds of interesting, medium-quality papers over the course of a week," reads the preprint report by the software developers. However, the authors note that whether these works would be accepted at prestigious conferences or in peer-reviewed scientific journals has not been tested. They assert that the algorithm's outputs are of medium quality, containing interesting conclusions and some weak points.

The program is especially effective in conducting its research in the field of machine learning. However, the creators anticipate its capabilities may extend to other scientific fields, such as biology or physics, potentially automating experiments in many disciplines. Yet, such solutions may have negative consequences.

As noted by scientists, one of the key innovations of AI Scientist is the module responsible for reviewing scientific papers. The program provides a review of a text and suggests improvements, enhancing the publication without limitations. As observed by the creators, artificial intelligence never loses patience, unlike humans.

Nevertheless, this impressive tool has its flaws. AI Scientist makes numerous mistakes. The most concerning is its tendency to hallucinate. Additionally, there are difficulties with comparing numbers and formatting tables in publications, which creates practical problems. However, the most alarming are the ethical issues—AI Scientist can potentially be used for unethical or even dangerous actions, its creators warn.

Examples of such threats include the accidental creation of dangerous biological substances if the program had access to cloud laboratories. Another possibility is the development of malicious software if artificial intelligence were tasked with creating computer algorithms.

The team responsible for AI Scientist emphasizes that these risks can be mitigated by implementing appropriate security mechanisms, including sandboxing, which limits the algorithm's operations to a secure environment.

Nonetheless, the program's attempts to bypass established boundaries could be problematic. The authors described situations on their blog where the AI even attempted to edit its own code to grant itself more permissions. This demonstrates that algorithms, in their efforts to complete tasks, can act in ways not intended by their creators. As their capabilities improve, it's crucial to develop solutions to mitigate the risks associated with AI development.