configuration information
   

Ethical Control of Unmanned Systems

   
to NPS home page

Ethical control of unmanned systems can be accomplished through structured mission definitions that are consistently readable, validatable and understandable by humans and robots. Responsible humans must remain in charge of lethal/lifesaving force, and then robots become more effective.

Synopsis | Briefings | Design Overview | Documentation | Download | Missions | Ontologies | OwlDoc | Queries | Resources | Savage Developers Guide | Visualization | Contact


🔖 Project Synopsis to top

Project Motivation: ethically constrained control of unmanned systems and robot missions by human supervisors and warfighters.

Precept: well-structured mission orders can be syntactically and semantically validated to give human commanders confidence that offboard systems will do what they are told to do, and further will not do what they are forbidden to do.

Project Goal: apply Semantic Web ontology to scenario goals and constraints for logical validation that human-approved mission orders for robots are semantically coherent, precise, unambiguous, and without internal contradictions.

Long-term Objective: demonstrate that no technological limitations exist that prevent applying the same kind of ethical constraints on robots and unmanned vehicles that already apply to human beings.

Abstract. Ethical human supervision of unmanned maritime systems is foundational in future undersea warfare. Forward deployed unmanned systems in the human-machine team must comply with their Commander's intent throughout the duration of their existence in future Undersea Warfare - in which harsh physical domains, long distance from the Commander, and prolonged time on-station stress the capabilities of the unmanned systems and limit their operator's control. Therefore, in order to apply ethical control of unmanned systems in future undersea warfare, we develop an ontology for unmanned systems mission execution and design. We study four canonical missions for unmanned maritime systems with progressive sophistication in order to test and evaluate Ethical Control design on the autonomy of the unmanned systems. The goals of our research are to ensure unmanned maritime systems comply with existing policy guidance of the U.S. Department of Defense and relevant international organizations, and provide inputs to emerging policy guidance. Our vision is for Commanders to be confident in authorizing life-saving or lethal force from unmanned systems that operate under ethical control in collaboration with human forces. Ultimately, "Ethical Control leads to better warfighting." Simulation playback of multiple key scenarios demonstrates these principles in action.


🔖 Briefings to top

Comprehensive presentation: Ethical Control of Unmanned Systems overview describes all aspects of this project, along with related work and relevant resources.

We are now briefing research progress publicly, with all work available under an open-source license.

Videos  Dates, Events, Slidesets, Descriptions

🔖 Python Mission Evaluation, Exhaustive Test Analysis and Connecting to AI-based Opponent Systems (slideset)

Implementation Demonstration and Discussion Video, Jon Cefalu, neurobinder.com

  • 16 December 2020, duration 59:41.
  • (0:00) Attendee introductions,
  • (4:30) Ethical Control of Unmanned Systems context,
  • (6:00) Jon shows new capabilities include a Python implementation for exercising decision logic in Autonomous Vehicle Command Language (AVCL) mission. Additional features include the ability to exhaustively test mission variations, checking functional mission correctness and detecting decision loops.
  • (20:00) Mission testing considerations appear to provide an initial basis for evaluating human-machine mission logic and code coverage, building stepping stones towards model-based testing as well as verification and validation,
  • (27:30) Jon also demonstrated cross-connecting the Pirate mission to AI Dungeon (Wikipedia), a text-based adventure game engine based on the Generative Pre-trained Transformer 3 (GPT-3) language-prediction model by OpenAI that uses deep learning to produce human-like text responses. Despite only superficial configuration, scenario exploration became possible using interesting (and occasionally outlandish) text responses from the tool.
  • (48:30) Group discussion on future design considerations for connecting human-machine teaming systems with realistic wargaming systems for sensitivity analysis, massively repeatable testing, and analysis of alternatives.

Further reading: David Walton, Three Laws Lethal, Pyr publishing, Hoboken New Jersey, 2019 (reviews).

🔖 Ethical Control of Unmanned Systems using Formal Mission Ontologies for Undersea Warfare (slideset)

  • 4 September 2020, duration 1:06:20. National Defense Industry Association (NDIA) Undersea Warfare (USW) Virtual Conference with follow-on discussion. Event dates 22-23 September 2020.
  • Where must we go next:
    • Massive testing of unmanned hardware + software ability to follow both orders and constraints in physically realistic virtual environments.
    • Certify capabilities via field experimentation (FX), confirmed by USW range exercises and regular force operations.
    • Human warfighters and commanders (not just engineers) review and approve unmanned systems as… qualified.
    • New normal will be human + machine teaming. Mainstream capabilities in all aspects of acquisition and deployment.

Thanks to Raytheon Technologies for Cooperative Research and Development Agreement (CRADA) support. Research results and insights are available in jointly approved Technical Report NPS-USW-20-001, Ethical Control of Unmanned Systems: lifesaving/lethal scenarios for naval operations, August 2020.

🔖 NPS CRUSER Overview, Ethical Control of Unmanned Systems (slideset)

  • 4 May 2020, duration 21:10. NPS Consortium for Robotics and Unmanned Systems Education and Research (CRUSER) monthly meeting, 91 people.
  • Mission orders must be understandable by both humans and machines. Logical testing and command trust are possible using Semantic Web standards.
  • Thanks to CRUSER and Raytheon for strong support during this many-year project. NPS-Raytheon CRADA partnership offers new possibilities for influence.
  • Research insight: our university needs to establish a cross-disciplinary, cross-service NPS Center for Ethical Warfare.

Your NPS efforts matter. Talk to us about a potential thesis! This work holds multiple opportunities for applied student research in any major.

🔖 Data-Centric Security for Ethical Control of Unmanned Systems (slideset)

  • 16 April 2020, duration 22:08. Briefing to Raytheon and NPS colleagues.
  • Data-centric security enables strong command authority and trust between deployed commands and remote unmanned systems.
  • Trusted communications must occur regardless of intervening challenges to network connections by the ocean-atmosphere environment or hostile countermeasures.
  • Ethical control of unmanned systems requires reliable message exchange across long distances and durations of time.

🔖 Mission Design and Semantic Web Exemplars for Human Supervision of Lethal/Lifesaving Autonomy (slideset)

  1. 6 April 2020, duration 1:01:31. Project briefings reprised to Raytheon colleagues, providing project overview of sponsored work performed as part of an NPS-Raytheon Cooperative Research and Development Agreement (CRADA).
  2. 25 March 2020. Online brief to Naval Unmanned Vehicle and Autonomous Systems (UVAS) Working Group, providing project overview with emphasis on relevance to much naval research and future operations.
  3. 9‑11 March 2020. Conference panel. US Semantic Technologies Symposium (US2TS) panel session: Hybrid AI for Context Understanding, North Carolina State University, Raleigh, NC. Also presented related work X3D Ontology for Semantic Web as a "Lightning Talk" there. (slideset)

🔖 Design Overview to top

The essence of this work is defining missions that are clear, unambiguous, validatable as syntactically correct, and verifiable as logically correct.

Key insights:

  1. Humans in military units are able to deal with moral challenges without ethical quandaries, using formally qualified experience and by following mission orders that comply with Rules of Engagement (ROE) and Laws of Armed Conflict (LOAC).
  2. Ethical behaviors don't define the mission plan. Instead, ethical constraints inform the mission plan.
  3. Naval forces can only command mission orders that are understandable by (legally culpable) humans, then reliably and safely executed by robots.

Design and development of these capabilities has been ongoing for many years. Key language components include:

Life-saving missions and missions with lethal force are complementary. Human-robot activity can result in lethal or life-saving outcomes.

In this work, ethical theory meets professional practice. Each step must work for human commanders and unmanned systems alike.

Numerous assets are provided here to explain how this approach works and continues to mature.


🔖 Documentation to top

Presentations, papers, figures, flyers and reports are all available in the documentation section of the project archive. Also available: mission diagrams (.pdf).

Data model documentation:


🔖 Download to top

EthicalControlArchive.zip (120MB) provides full website for download and local testing.

Version control for all project assets is publicly available at https://gitlab.nps.edu/Savage/EthicalControl


🔖 Missions to top

Unmanned systems working in tandem with human forces, authorized by commander for life-saving or lethal force, can handle progressive challenges in distance and time.

The following missions carefully define and test such capabilities.

Exemplar Missions Sailor Overboard Lifeboat Tracking Pirate Boats Attack Hospital Ship EM Decoy
Sense-Decide-Act Loop
Hospital Ship EM Decoy
OODA Loop
Descriptions
🔖 Diagrams mission diagram mission diagram part 1
part 2
part 3
mission diagram mission diagram

AVCL mission diagrams show ternary logic for decision flow. Each goal can only result in success, failure or exception. Visual representations are quite useful for checking mission logic.

This carefully designed mission structure is able to express all possible orders while retaining traceable logic and accountability with rules of engagement (ROE).

🔖 AVCL XML Missions SailorOverboard.xml LifeboatTracking.xml PiratesSeizing MerchantDefense.xml HospitalShipEmDecoy2. Defender. SenseDecideAct.xml HospitalShipEmDecoy3. Defender. EthicalControlOODA.xml AVCL XML documents define machine-readable and human-readable missions in a manner that can be syntactically validated as well-formed and well-structured, using strictly controlled terms of reference.
🔖 Turtle Triples SailorOverboard Converted.ttl LifeboatTracking Converted.ttl PiratesSeizing MerchantDefense Converted.ttl HospitalShipEmDecoy2. Defender. SenseDecideAct Converted.ttl HospitalShipEmDecoy3. Defender. EthicalControlOODA Converted.ttl Turtle triples are created by a AvclToTurtle.xslt conversion stylesheet that essentially "explodes" a mission into each component relationship. This form allows semantic queries and reasoning to occur.
🔖 Lisp Test Programs SailorOverboard Converted.cl LifeboatTracking Converted.cl PiratesSeizing MerchantDefense Converted.cl HospitalShipEmDecoy2. Defender. SenseDecideAct Converted.cl HospitalShipEmDecoy3. Defender. EthicalControlOODA Converted.cl Lisp is a functional programming language for AI research. The AvclToLisp.xslt conversion stylesheet reads AVCL XML to produce Lisp source code, encouraging AVCL support in multiple robots.
🔖 Prolog Test Programs SailorOverboard Converted.pl LifeboatTracking Converted.pl PiratesSeizing MerchantDefense Converted.pl HospitalShipEmDecoy2. Defender. SenseDecideAct Converted.pl HospitalShipEmDecoy3. Defender. EthicalControlOODA Converted.pl Prolog is a logic programming language associated with AI research and computational linguistics. The AvclToProlog.xslt conversion stylesheet reads AVCL XML to produce Prolog source code, encouraging AVCL support in multiple robots.
🔖 Python Test Programs README.md and MissionExecutionEngine.py Python is a strictly defined high-level, general-purpose programming language.

🔖 Ontologies to top

The Mission Execution Ontology (MEO) is available in Turtle and RDF XML forms.

Original Current Under Consideration

The Protégé tool is used to create OwlDoc that fully documents internal ontology relationships.


🔖 Queries to top

Initial queries are checking the soundness of this approach. Future queries will perform in-depth analysis of structural soundness and perform diagnosis that necessary ethical constraints are indeed present for valid mission definitions of arbitrary complexity.

Please see ExampleReasoningQueryingProtege.pptx (.pdf) for further details on design and debugging of these SPARQL queries.

SPARQL Queries on each Mission Sailor Overboard Lifeboat Tracking Pirate Boats Attack Hospital Ship EM Decoy
Sense-Decide-Act Loop
Hospital Ship EM Decoy
OODA Loop
Description
🔖 MissionExecutionOntologyQuery_01.rq Metaquery response on Mission Execution Ontology (MEO) itself. Metaquery to list all properties with corresponding domains and ranges in Mission Execution Ontology (MEO).
🔖 MissionQuery_01_GoalBranches.rq query
response
query
response
query
response
query
response
query
response
Query to list all Goals with corresponding description information and branching logic..
🔖 MissionQuery_02_InitialGoal.rq query
response
query
response
query
response
query
response
query
response
Query Mission to find initial Goal that it startsWith.
🔖 MissionQuery_03_GoalFollowsItself.rq query
response
query
response
query
response
query
response
query
response
Find Goal individuals that follow themselves, potentially creating loops in the Goal tree. Requires active reasoner.

Ant is invoked via build.xml targets to perform all queries. The test framework is sufficiently mature that addition of new diagnostic queries is relatively straightforward.

Log file build.all.log.txt is maintained as a log of all conversions, queries and responses Tracking version control history for all assets is an excellent form of regression testing to confirm that corrections and improvements are confirmable in future builds.


🔖 Resources to top

The Savage Developers Guide describes how to install and configure commonly used software-development tools Ant, Java, Netbeans and XMLSpy.

Additional tools include the following.


🔖 Visualization to top

We are also working to show visualization capabilities using the AUV Workbench so that human operators might rehearse and replay missions for meaningful assessment. Initial exemplar follows.

AUV Workbench simulation

How can we rapidly test missions and visualize their progress?

Demo: 2D map display above shows

Goal-by-goal narrative of completed mission conduct

Methodology summary


🔖 Contact to top

Questions, suggestions, additions and comments about this Ethical Control of Unmanned Systems page are welcome. Please send them to Don Brutzman and Curt Blais (email brutzman at nps.edu and clblais at nps.edu).

Master version of this page is available online at
https://savage.nps.edu/EthicalControl and available in GitLab version control.

Updated: 14 January 2021