Ethical Control of Unmanned Systems

Autonomous systems can be ethically supervised by humans without constant communications. Continuing progress builds on many work-years of effort.

News. Presentation "Ethical Control of Autonomous Unmanned Systems: A Practical Approach" this Wednesday 1430-1450, 12 April 2017 at NPS in Ingersoll 122 auditorium as part of the annual CRUSER Technical Conference (TechCon) 2017.

The backdrop for this work:

  • Roboticists tend to build software systems in unique and dissimilar ways, but nevertheless share a common repertoire of directable capabilities.
  • A number of philosophers view unmanned systems as inherently uncontrollable, and therefore propose international protocols banning their existence.
  • Potential opponents are likely to use such systems as weapons of war regardless, and do not particularly care about ethical command and control.

The gist of our work:

  • Releasing uncontrolled robots at sea with potential for lethal force is not permissible under Law of Armed Conflict (LOAC).
  • Naval officers must act ethically and maintain supervisory control of such devices even if direct communications might be lost.
  • Formal mission orders can describe both tasking and constraints on action that are logically validatable and executable by a wide variety of robots.
  • Identification Friend Foe Neutral (IFFN) is much simpler on the ocean than it is on land.
  • A feasible path forward exists that allows Navy commanders to similarly task and trust unmanned systems to act - and appropriately avoid acting - just as they might with other trusted human partners.

Short-form and long-form papers, presentations:

short
form 		Davis, Duane T., Brutzman, Donald P., Blais, Curtis L. and McGhee, Robert B., "Ethical Mission Definition and Execution for Maritime Robotic Vehicles: A Practical Approach," MTS/IEEE OCEANS 2016, Monterey California USA, 19-23 September 2016, 10 pages. slides (.pdf)
full
length 		Brutzman, Donald P., Davis, Duane T., Blais, Curtis L. and McGhee, Robert B., "Ethical Mission Definition and Execution for Maritime Unmanned Systems: A Practical Approach," draft paper for IEEE Journal of Oceanic Engineering, submitted 28 January 2017, 29 pages. slides (.pdf)

Abstract. Many types of robotic vehicles are increasingly utilized in both civilian and military maritime missions. Some amount of human supervision is typically present in such operations, thereby ensuring appropriate accountability in case of mission accidents or errors. However, there is growing interest in augmenting the degree of independence of such vehicles, up to and including full autonomy. A primary challenge in the face of reduced operator oversight is to maintain full human responsibility for ethical robot behavior.

Informed by decades of direct involvement in both naval operations and unmanned systems research, this work proposes a new mathematical formalism that maintains human accountability at every level of robot mission planning and execution. This formalism is based on extending a fully general model for digital computation, known as a Turing machine. This extension, called a Mission Execution Automaton (MEA), allows communication with one or more "external agents" that interact with the physical world and respond to queries/commands from the MEA while observing human-defined ethical constraints.

An important MEA feature is that it is language independent and results in mission definitions equally well suited to human or robot execution (or any arbitrary combination). Formal description logics are used to enforce mission structure and semantics, provide operator assurance of correct mission definition, and ensure suitability of a mission definition for execution by a specific vehicle, all prior to mission parsing and execution. Computer simulation examples show the value of such a Mission Execution Ontology (MEO).

The flexibility of the MEA formalism is illustrated by application to a prototypical multiphase area search and sample mission. This paper presents an entirely new approach to achieving a practical and fully testable means for ethical mission definition and execution. This work demonstrates that ensuring ethical behavior during mission execution is achievable with current technologies and without requiring artificial intelligence abstractions for high-level mission definition or control.

Announcement and naval context: "Ethical Control of Unmanned Systems," Network Optional Warfare (NOW) Blog, 8 March 2017.

Related developmental work: