Canonical instabilities of autonomous weapons under fog-of-war constraints: extending the Data Rate Theorem to cognitive systems

Rodrick Wallace (1)

1 New York State Psychiatric Institute

Under fog-of-war conditions, skilled human operators of complex man-machine cockpit weapon systems have repeatedly been unable to avoid fratricide incidents and violations of the laws of land warfare. In spite of these continuing catastrophic failures, autonomous weapons have been proposed that entirely remove the man from the command loop. Formal analysis, via an extension of the Data Rate Theorem, shows that fully autonomous weapons under constraints on incoming environmental information will be subject to sudden, highly punctuated collapse to a dysfunctional ground state in which 'all possible targets are enemies'.

Mots clés: control theory; fratricide; information theory; laws of land warfare; Rate Distortion Theorem; spontaneous symmetry breaking

Domaines: Intelligence artificielle [cs.AI]

Rodrick Wallace. Canonical instabilities of autonomous weapons under fog-of-war constraints: extending the Data Rate Theorem to cognitive systems. 2016. ⟨hal-01304193⟩