What AI was actually asked to build
In 2023, a research group led by Kevin Esvelt at MIT used commercial language models to generate genetic sequences of known pathogens, then walked the output past biosecurity officials to gauge how worried they should be. The exercise was a stress test of the drafting stage, not a demonstration of a finished capability. Nobody built anything.
Fact. The sequences were assembled from material already present in public virology literature and in the training data of models like GPT-4. Moving from a written sequence to an agent that spreads between hosts requires chemical synthesis, a high-containment laboratory, animal work, and years of tacit technique that is not distributed in papers. None of those steps are API endpoints.
Interpretation. What got cheaper in 2023 was the first draft of a research plan, which was already the least protected and least decisive part of the process. A graduate student with a well-stocked literature review could produce a similar draft in a weekend, and could not synthesize it any faster than before.
The phage result is the real test case
Fact. In August 2026, a Stanford and Arc Institute team led by Brian Hie published in Science the first working viral genomes designed end to end by generative models. The group used the Evo 1 and Evo 2 genome language models to write bacteriophage genomes modeled on PhiX174, a virus that infects E. coli rather than people, and sixteen of the synthesized designs came back viable in the lab. As War on the Rocks noted in its October analysis, the successful designs kept more than 92 percent identity to known viruses.
The result is real, and it is narrower than the headlines suggest. These are bacteria-infecting phages built on a well-characterized template, with a fast, cheap readout for whether a design works. Nothing in the paper shows a model inventing a mammalian pathogen from scratch.
Interpretation. This is the pattern to expect when a model is evaluated on a problem with a physical referee. Phage viability has ground truth: either the DNA works in cells or it does not, and the failure modes are chemical rather than rhetorical. A model that scores well on language benchmarks can still waste a bench scientist’s month, which is a real cost, just not a security event.
That distinction matters because it separates two very different claims now circulating in policy circles: that models can propose harmful biology, which is well documented, and that models can substitute for the experimental loop, which the published record does not yet support. Our earlier look at how AI started doing science rather than assisting it made the same point from the other direction: the systems are strongest where results are checkable quickly and weakest where the feedback loop runs through a wet lab.
Synthesis, not sequence design, is the choke point
Fact. Gene synthesis in the United States is screened. Providers including Twist Bioscience and Integrated DNA Technologies state publicly that they check customer orders against pathogen and toxin lists, following guidance from the National Science Advisory Board for Biosecurity that was updated in 2024. In January 2025, the Commerce Department proposed a rule under its information and communications technology authorities covering biotechnology equipment, including DNA synthesizers, which would push identity verification and sequence screening from industry practice into federal requirement.
The weakness in that layer is coverage rather than design. Screening binds the suppliers that sell into US and European supply chains, while orders placed with unregulated overseas synthesis providers are checked far less consistently, a gap the Commerce proposal is explicitly aimed at closing.
Interpretation. Defense here works for the same reason many digital controls work: it operates on defaults rather than on user virtue. That is the argument we traced in why a default setting behaves like policy, and it applies cleanly to nucleic acid synthesis. A rule that makes screening the unremarkable standard path outperforms any amount of instruction aimed at individual researchers.
Counter-sequencing also matters. Platforms that check synthesis requests for overlap with known pathogens inherit their intelligence from reference databases, so a sufficiently novel design inherits a degree of blindness. Nobody has yet shown that this blindness is exploitable at scale, and the Stanford phage designs that worked stayed close to known viruses, which suggests the design step is not where an attacker would gain leverage.
What defenders measure instead of headlines
Fact. The live debate has shifted from model capability to defense posture, a framing made in War on the Rocks' recent piece "How to Defend Against AI-Designed Viruses." Meanwhile OpenAI's decision this week to share its progress in mathematics signals where frontier capability is actually being demonstrated: long chains of reasoning that specialists can verify line by line.
Biology has no equivalent scoreboard. Its verdicts arrive months later, on a bench, and the negative results are harder to publish than the alarming ones, which skews what the public sees in both directions.
Interpretation. The measurement discipline borrowed from misaligned-agent research is the most transferable tool available. Adversarial probing of a system before it ships, the approach examined in Nvidia's tool for catching misaligned agents, is the same basic move: red-team the model on the specific harm, record what it actually did, and publish the methodology so others can repeat it. Biology capability evaluations are following that playbook, unevenly, and without a shared benchmark yet.
Prediction: three things to watch
Prediction. First, the binding constraint through 2027 will stay on the synthesis side, not the model side. Expect screening rules to become the contested surface, with the Commerce proposal and its equivalents abroad drawing more litigation and lobbying than any model release.
Second, AI-generated biological designs will keep improving on paper while the gap between a plausible sequence and a replicable agent narrows slowly, because the missing ingredient is bench iteration under containment. Headlines will continue to compress that gap into a sentence.
Third, the first genuinely alarming incident will probably look boring: a screening miss on a routine order rather than a model demo. The response to that event will be written by whoever has already documented their evaluation methods, which is an argument for publishing boring negative results now while the topic still attracts attention.
Comments
No comments yet. Be the first to share your thoughts.
Leave a comment