An Engineer's Elegant Solution to the Pentagon - Anthropic Debate about AI in the Kill Chain
Hint: Contractual compromise is the wrong approach
I came across this assessment in a Facebook post by my friend and go-to expert on everything related to Industrial Control System risk, Tim Roxey. As a non-engineer, I found it to be clear, concise, and convincing regarding the Pentagon - Anthropic controversy over how Claude could be used by the Pentagon. If you like this, and you want to read the full version with footnotes, visit Tim’s Substack and subscribe.
The Pentagon and Anthropic are in a very public fight over whether AI companies should set the rules for how their technology is used in military operations. Undersecretary Emil Michael says it’s “not democratic” for Anthropic to restrict military use of Claude. Anthropic says its models shouldn’t be used for autonomous weapons or mass surveillance. Secretary Hegseth is reportedly close to designating Anthropic a “supply chain risk” — a classification normally reserved for foreign adversaries.
Both sides are talking past each other, and here’s why: neither one has an engineering framework for the argument they’re actually having.
The Pentagon’s position rests on DoDD 3000.09, the directive on autonomy in weapon systems. That directive requires “appropriate levels of human judgment over the use of force.” But as multiple analysts have documented — including the people who wrote the update — 3000.09 doesn’t ban autonomous weapons, doesn’t require a human in the loop (that phrase never appears in the policy), and leaves “appropriate” deliberately undefined. That was defensible when the directive governed deterministic software that could be formally verified. It is dangerously inadequate for generative AI systems whose cognitive outputs are non-deterministic and cannot be mathematically proven correct.
Anthropic’s position rests on its terms of service. That’s a contractual restriction a company can be pressured to revise — which is exactly what’s happening.
What’s missing is the engineering argument. Through the ISA Global Cybersecurity Alliance, and my own research over the past decade, my colleagues and I have developed a hybrid quality assurance framework that resolves this impasse by grounding it in the actual technical characteristics of generative AI.
The core concept is the Bright Line — an architecturally enforceable boundary above which generative AI may not exercise autonomous control authority. This isn’t a policy preference or a corporate restriction. It’s an engineering conclusion: the same non-determinism that gives large language models their analytical power makes it impossible to verify they’ll never produce a catastrophically wrong recommendation in an untested context. For applications where consequences are irreversible — which describes every lethal engagement — a human must serve as the final safeguard.
The implementation is the Command Broker architecture. Above the Bright Line, the AI and the weapons platform are physically separated by a human decision-maker. The AI analyzes, recommends, and explains. The human decides and acts. The defense is architectural, not procedural.
Here’s the thing: this maps exactly to how military command authority already works. Commanders don’t delegate lethal authority to staff analysts. They receive analysis and decide. The Command Broker codifies that existing command relationship in the system architecture.
This reframes the entire dispute. It’s not that Claude shouldn’t be used in the kill chain. It’s that above the Bright Line, the system architecture must enforce the separation between AI recommendation and lethal action that military command authority already demands. The Pentagon gets generative AI across kill-chain planning, analysis, and assessment — where they’re already seeing what their own CDAO calls “significant advantage.” Anthropic’s safety concerns are satisfied by engineering, not by a usage policy that can be renegotiated under political pressure. And Congress gets a governance architecture with clear categories, boundaries, and evaluation criteria for the annual LAWS reporting required by the FY2025 NDAA through 2029.
The middle ground isn’t a negotiated compromise between contractual positions. It’s an engineered solution derived from the technology itself.