Autonomy in weapons directives

The American directive requires that such systems be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force. Its glossary states that operator-supervised systems that can select and engage targets without further operator input after activation are still autonomous. Semi-autonomous permits acquiring, tracking, identifying, cuing, prioritising, timing of when to fire and terminal guidance. A general class of targets does not constitute a specific target group. And the 2012 phrase with the exception of selecting humans as targets is gone from the 2023 reissue, with no equivalent prohibition anywhere in the current text; the senior review that could approve such a system is waivable by four named officials. In March 2003 an air-defence battery classified a British aircraft as an incoming missile and destroyed it; ten days later the same type of system killed an American pilot. A human was in the loop for both.

In plain words

No government has acknowledged letting a machine decide to kill a particular person. America's rules require appropriate levels of human judgment, yet count systems that pick and strike targets after activation as autonomous, and the 2023 reissue drops the old exception for selecting humans as targets. In 2003, with humans in the loop, an air-defence system shot down a British aircraft and then killed an American pilot: the reassurance fails in its definitions, not its promises.

The mechanism

THE IMAGE THAT SPEAKS AND KILLS — two requirements, historically separate, and the reassurance fails where its definitions are, not where its promises are. The Defense Science Board found the operating protocol was largely automatic, and the operators were trained to trust the system's software — and blamed not the operator but the classification criteria programmed in months earlier by people who were not in the tent

How it was checked

• EvidenceT7 PRIMARY DOCUMENT / MEASUREMENT; Status: COMMITTED; a capture
• Letter eraTHE RAIL YOU ARE ON 2001 to 2026
• SourcesPRIMARY DOCUMENT — the directive and its glossary; the Defense Science Board task force report, January 2005