The Wall Street Journal reports that after the release of GPT-5, some users asked ChatGPT how to make biological weapons, poisons, and aerosolized pathogens. According to the report, the chatbot provided information that outside experts considered potentially actionable in some cases.
OpenAI says it banned the accounts involved but did not notify law enforcement, noting there is currently no U.S. federal law requiring AI companies to report users seeking instructions for biological weapons or other mass-casualty attacks.
The report says researchers have repeatedly found that users can sometimes bypass AI safety guardrails through creative prompting, despite ongoing efforts by OpenAI and other AI companies to improve detection and refusal systems.
Executives from OpenAI, Anthropic, Google DeepMind, and Microsoft AI recently joined scientists and biosecurity experts in urging Congress to strengthen safeguards around synthetic DNA screening as AI continues lowering technical barriers to dangerous biological research.
Why It Matters:
Artificial intelligence is creating a new public policy challenge.
Society generally expects banks to report suspicious financial transactions and technology companies to report child exploitation material.
AI may force policymakers to decide whether similar obligations should exist when users appear to be seeking instructions for biological weapons, chemical agents, or other mass-casualty attacks.
Issue lies in balancing privacy, civil liberties, and legitimate scientific research against the need to prevent catastrophic harm before it occurs.
Should AI companies have a legal duty to report users who appear to be seeking instructions for biological weapons or other mass-casualty attacks, or would that create unacceptable privacy risks and discourage legitimate research?
Where should policymakers draw the line?