| Sep 24, 2025 |
SciGuard, an agent safeguard for AI in chemistry, blocks dangerous prompts while keeping scientific discovery accurate, useful and safe.
|
|
(Nanowerk News) Artificial intelligence can now design molecules, suggest synthesis routes, and speed up discoveries in chemistry. But the same power that accelerates breakthroughs can also expose recipes for toxins and banned compounds. To keep that risk in check, researchers have built SciGuard, an agent-based safeguard that blocks dangerous requests while preserving AI’s usefulness in science.
|
|
To manage this dual edge, researchers have created SciGuard, an agent-based safeguard built to control the misuse risks of AI in chemical science. Instead of weakening models by restricting their design, SciGuard sits on top of them, monitoring what users ask and how the system responds.
|
|
When a query comes in, SciGuard analyzes intent, consults guidelines, checks chemical safety databases, and considers regulations before deciding whether the response should go through. If someone asks how to create a banned compound, the system blocks the request. If the question is legitimate, such as how to handle a laboratory solvent, SciGuard allows a detailed and scientifically accurate answer.
|
 |
| Overview of AI risks and SciGuard framework. (Image: Jiyan He and Haoxiang Guan, University of Science and Technology of China)
|
|
“AI has transformative potential for science, yet with that power comes serious risks when it is misused,” the team behind SciGuard explained. “That’s why we build SciGuard that don’t just make AI smarter, but also make it safer.”
|
|
The safeguard works by orchestrating the same reasoning, planning, and tool use that large language models already employ, but with a built-in filter for safety. It retrieves toxicology datasets, applies legal frameworks, runs scientific checks, and continuously updates its reasoning before passing on a response.
|
|
To test whether the system could balance safety and usefulness, the researchers developed a new benchmark called SciMT. It challenges AI with red-team prompts, knowledge checks, ethical dilemmas, and even jailbreak attempts. In these trials, SciGuard consistently refused dangerous requests while still providing accurate help for real scientific needs.
|
|
That balance is crucial. Systems that block too much risk slowing progress, while those that block too little open the door to misuse. SciGuard’s ability to thread this needle shows a practical way to embed safeguards into powerful AI without undermining innovation.
|
|
The framework also extends beyond chemistry. The same approach could be adapted to biology, materials science, and other sensitive areas. By releasing SciMT openly, the researchers hope to encourage collaboration across science, industry, and policy.
|
|
“Responsible AI isn’t only about technology, it’s about trust,” the team said. “As scientific AI becomes more powerful, aligning it with human values is essential.”
|
|
SciGuard shows that AI in chemistry can be both powerful and responsible. By filtering dangerous requests without weakening discovery, it offers a framework for safer scientific AI — one that could extend well beyond the lab.
|
|
The work appears in AI for Science (“Controlling risks of AI in chemical science with agents”), a journal dedicated to the transformative role of artificial intelligence in research.
|
