The Dawn of AI Warfare: From Hypotheticals to Reality
The Convention on Certain Conventional Weapons, an international forum focused on lethal autonomous systems, meets biannually at the United Nations in Geneva. When Branka Marijan attended in November 2017, she expected the five-day sessions to dwell on hypotheticals about killer robots. However, that year marked a turning point. The imagined future suddenly felt imminent and tangible.
On the first day, attendees watched Slaughterbots, a short film by the Future of Life Institute. It depicted a fictional defense contractor pitching an AI-powered drone capable of unassisted precision killings. "They used to say guns don’t kill people, people do," the CEO states. "But people don’t. They get emotional, disobey orders, aim high. Let’s watch the weapons make the decisions." The room grew tense, as Marijan recalls, because the Pentagon was already developing similar technology.
Project Maven and the Shift to Autonomous Systems
That 2017 meeting was the first after the launch of Project Maven, a US Department of Defense initiative using AI to analyze drone surveillance footage. By late 2017, Google had joined as a major tech partner. "The systems we were talking about were not futuristic," said Marijan, senior researcher at Project Ploughshares. "They were existing platforms with degrees of autonomy, capable of selecting and engaging targets based on sensor data."
The Human Role in Drone Warfare
The world had already witnessed drone warfare—deadly machines directed by humans. Now, it faced a future where humans might be removed entirely. "These were not Terminator-like figures," Marijan explained. "It was about the enablement of autonomy."
Anthropic's Battle with the Pentagon: Setting Red Lines
Nearly a decade later, fully autonomous lethal weapons remain undeveloped, but they sit at the center of a high-stakes battle between the US government and AI startup Anthropic. Anthropic aims to preserve two "red lines": bans on domestic mass surveillance and weapons that identify, track, and kill with zero human involvement. Since early 2026, it has emerged as the only military AI contractor imposing meaningful limits on what experts call the final frontier of AI warfare.
The Pentagon's Demands and Legal Clash
In January 2026, Defense Secretary Pete Hegseth demanded renegotiation of DOD AI contracts, removing gray areas to allow "any lawful use." Anthropic objected. The DOD later designated Anthropic a military supply chain risk, and President Donald Trump banned government agencies from using its Claude system. Though relations have warmed with the release of Anthropic's cybersecurity model Mythos, a court battle continues.
Industry Disagreements on AI Limits
"It’s not government-created technology like the Manhattan Project," said Andrew Reddie, associate research professor at UC Berkeley. "This is a pain point when startups engage directly with the Pentagon." Even within Silicon Valley, there's "a lot of disagreement" over when limits should be set.
The Evolution of Military AI: From Surveillance to Kill Chains
Beyond the Anthropic debate, AI has been deeply embedded in military operations for decades. In the 2000s, AI enabled parsing unprecedented data volumes, creating a surveillance revolution. The late 2010s saw advanced facial recognition and machine vision systems. "Even without full autonomy, AI compresses kill chains to mere seconds," said Maddy Batt, legal fellow at Tech Justice Law. "Humans are not making assessments required by international humanitarian law to prevent civilian harm."
DOD Directive 3000.09 and Autonomous Weapons Policy
At the center of debates is DOD Directive 3000.09, one of the few policies governing lethal autonomous weapons. Written in 2012, it defines such systems as those that "once activated, can select and engage targets without further intervention by an operator." It mandates that humans exercise "appropriate levels" of judgment over force use.
Historical Precedents: Phalanx CIWS
Some missile defense programs, like the Phalanx CIWS, may have crossed this line decades ago. This automated weapon system defends naval vessels from missiles in milliseconds, without human intervention. Experts argue such systems operate in defense-only, fixed environments—engaging threats but not deciding. "The 'and' is doing a lot of work in that statute," Reddie said. "We have systems that can decide and systems that can engage, but you can’t have both."
The Future of Autonomous Warfare: Crossing the Rubicon
As the debate unfolds, AI's influence in military operations continues to grow. "We’ve kind of crossed the rubicon while we pretend that we haven’t," Reddie noted. Even Anthropic seems to think its red lines won't hold for long, given historical patterns.
Key Takeaways on AI and Military Ethics
- Autonomous weapons remain undefined but are increasingly integrated into military systems.
- Corporate red lines face pressure from government demands for broader AI use.
- Human oversight is shrinking as AI compresses decision-making to milliseconds.
- International law struggles to keep pace with AI-driven warfare capabilities.
Comment