Ethical Control of Autonomous Systems

🔖 Ethical Computing: Metrics for Measuring AI's Proficiency and Competency for Ethical Reasoning

Symposium workshop overview. The prolific deployment of Artificial Intelligence (AI) across different applications have introduced novel challenges for AI developers and researchers. AI is permeating decision making for the masses: from self-driving automobiles, to financial loan approval, to military applications. Ethical decisions have largely been made by humans with vested interest in, and close temporal and geographical proximity to the decision points. With AI making decisions, those ethical responsibilities are now being pushed to AI designers who may be far-removed from how, where, and when the ethical dilemma occurs. Such systems may deploy global "ethical" rules with unanticipated or unintended local effects or vice versa. While explainability is desirable, it is likely not sufficient for creating "ethical AI", i.e. machines that can make ethical decisions. These systems will require the invention of new evaluation techniques around the AI's proficiency and competency in its own ethical reasoning. Using traditional software and system testing methods on ethical AI algorithms may not be feasible because what is considered "ethical" often consists of judgements made within situational contexts. The question of what is ethical has been studied for centuries. This symposium invites interdisciplinary methods for characterizing and measuring ethical decisions as applied to ethical AI.

This event is one of nine workshops for Association for Advancement of Artificial Intelligence (AAAI) Spring Symposium, 21-23 March 2022, and is organized by our collaborating colleagues at Raytheon. Work building on these capabilities is presented in three sessions. (schedule)

• A Tiered Approach for Ethical AI Evaluation Metrics,
  Peggy Wu, Brett Israelsen, Kunal Srivastava, Hsin-Fu "Sinker" Wu, and Robert Grabowski.

  Abstract. Advances in machine learning are enabling autonomy to operate in environments of increasing complexity, including scenarios with ethical concerns. For many Artificial Intelligence (AI) systems, decisions are driven by the goal to maximize reward. Policies may contain unintended consequences known as reward hacking. The AI is optimizing within the constraints defined by the domain and goals and does not have the capability to distinguish between benign and negative consequences beyond specifications. This paper describes an ongoing effort to develop an application-agnostic framework for AI systems to simulate actions, characterize potential outcomes, and perform introspection to articulate the motivations for action. Such a framework provides the foundational work for higher-level ethical reasoning using consequential and deontological ethics than other approaches in AI ethics. This enables metrics from consequential ethics to be used to assign ethical value of actions based on outcomes. Simultaneously, metrics from deontological ethics can be applied to evaluate the universality of its motivations. A Trolley Problem -inspired maritime search-and-rescue scenario is used to operationalize and demonstrate this framework.

• Doctrine and Ethics Compliant Autonomy Using An Ontological Framework,
  Don Brutzman, Curt Blais, Hsin-Fu "Sinker" Wu, Richard Markeloff, and Carl Andersen.

  Abstract. Ensuring ethical robot behavior requires complex representations and methodologies designed to guarantee it. Our approach extends frameworks already used by the U.S. military to ensure human ethical and doctrinal behavior by human beings. These have built in advantages of being able to express complex plans and constraints, yet remaining intelligible to humans, a requirement for ethical responsibility and liability. To extend the framework to machines, mission constructs are expressed using an Autonomous Vehicle Command Language (AVCL) expressing mission actions and outcomes that can readily be translated to runnable source code in several programming languages. Missions written in AVCL can be validated via translation to an RDF/OWL Mission Execution Ontology (MEO) supporting queried proofs of ethical correctness. MEO ensures that missions are both semantically valid and compliant with ethical constraints. These technologies implement a simulation, testing, and certification regime that can serve as a foundation for human authority over and trust in robots capable of lethal force.

  This paper is dedicated to the memory of Rich Markeloff who made substantial contributions towards our understanding, adaptation and usage of advanced Semantic Web capabilities supporting ethical control of autonomous systems.

• Meaningful Metrics for Demonstrating Ethical Supervision of Unmanned Systems,
  Don Brutzman and Curt Blais.

  Abstract. Metrics for AI are important, as illustrated by the workshop topics of interest. We note that commonplace gaps in applied AI derive from "Here are the measurements we know how to take" which are too easily over-extrapolated into conclusions of interest. In other words, such precise metrics are necessary and appealing but may not broadly apply to general situations. We assert that necessary subsequent questions are "How do we define meaningful objectives and outcomes for a current unmanned system," "How do we measure those characteristics that indicate expected success/failure," and "Once we can measure meaningful results, how do we assemble exemplars into test suites that confirm successful completion across ongoing system life cycles?"

  This discussion session seeks to find common threads among all workshop contributions that may help advance progress on these fundamental challenges.

🔖 Ethical Control of Unmanned Systems: Repeatable Mission Evaluation through Unmanned Systems Data Strategy

Humans must be able to effectively control unmanned systems holding the capacity for lethal and life-saving force. Trust can only be achieved if robots can provably follow human orders, both for what to do and what not to do, over long distances in space and long durations of time. Semantic Web techniques provide a scalable framework for comprehensive progress that can demonstrate compatibility with policy, law, and treaty obligations. Building best-practice workflows for data and metadata from unmanned systems can leverage both field experimentation (FX) and simulation to support archival data re-use and repeatable analysis.

• CRUSER 2021 project briefing: session video, slideset pptx, and slideset pdf.

  • 10 December 2021, duration 1:12:33.
  • (0:05) Welcome, speaker introductions, agenda
  • (2:55) Curtis Blais: Data modeling and formal ontology development, Mission Execution Ontology (MEO), and C2SIM standard with NATO.
  • (21:30) Kristen Fletcher: Legal and ethical policies, ramifications for unmanned systems in a rapidly evolving world.
  • (31:55) Terry Norbraten: Software design development and testing, data conversion using Data Format Description Language (DFDL).
  • (44:50) Don Brutzman: DFDL significance, unmanned systems data strategy, readiness to scale up, establishing NPS experimentation as a DoD Test Range.
  • (53:15) Speaker summaries and student reactions.
• KLV DIS X3D demo video

Although many component capabilities can be found today, the overall picture for common practice remains incomplete. We have achieved useful progress on multiple missing capabilities: regularization of data-collection workflows by operators of unmanned systems, ability to parse and formally define collected information for re-use, plus ability to replay streams for unit-test assessment and TestDevOps. We are ready to move to the next level: scaling up to archivally support all NPS field experimentation (FX), establishing an unclassified defense test range useful to all services, industry partners, and allied nations.

🔖 Python Mission Evaluation, Exhaustive Test Analysis and Connecting to AI-based Opponent Systems (slideset)

Implementation Demonstration and Discussion Video, Jon Cefalu, neurobinder.com

• 16 December 2020, duration 59:41.
• (0:00) Attendee introductions,
• (4:30) Ethical Control of Autonomous Systems context,
• (6:00) Jon shows new capabilities include a Python implementation for exercising decision logic in Autonomous Vehicle Command Language (AVCL) mission. Additional features include the ability to exhaustively test mission variations, checking functional mission correctness and detecting decision loops.
• (20:00) Mission testing considerations appear to provide an initial basis for evaluating human-machine mission logic and code coverage, building stepping stones towards model-based testing as well as verification and validation,
• (27:30) Jon also demonstrated cross-connecting the Pirate mission to AI Dungeon (Wikipedia), a text-based adventure game engine based on the Generative Pre-trained Transformer 3 (GPT-3) language-prediction model by OpenAI that uses deep learning to produce human-like text responses. Despite only superficial configuration, scenario exploration became possible using interesting (and occasionally outlandish) text responses from the tool.
• (48:30) Group discussion on future design considerations for connecting human-machine teaming systems with realistic wargaming systems for sensitivity analysis, massively repeatable testing, and analysis of alternatives.

Further reading: David Walton, Three Laws Lethal, Pyr publishing, Hoboken New Jersey, 2019 (reviews).

🔖 Ethical Control of Unmanned Systems using Formal Mission Ontologies for Undersea Warfare (slideset)

• 4 September 2020, duration 1:06:20. National Defense Industry Association (NDIA) Undersea Warfare (USW) Virtual Conference with follow-on discussion. Event dates 22-23 September 2020.
• Where must we go next:
  • Massive testing of unmanned hardware + software ability to follow both orders and constraints in physically realistic virtual environments.
  • Certify capabilities via field experimentation (FX), confirmed by USW range exercises and regular force operations.
  • Human warfighters and commanders (not just engineers) review and approve unmanned systems as… qualified.
  • New normal will be human + machine teaming. Mainstream capabilities in all aspects of acquisition and deployment.

Thanks to Raytheon Technologies for Cooperative Research and Development Agreement (CRADA) support. Research results and insights are available in jointly approved Technical Report NPS-USW-20-001, Ethical Control of Unmanned Systems: lifesaving/lethal scenarios for naval operations, August 2020.

🔖 NPS CRUSER Overview, Ethical Control of Unmanned Systems (slideset)

• 4 May 2020, duration 21:10. NPS Consortium for Robotics and Unmanned Systems Education and Research (CRUSER) monthly meeting, 91 people.
• Mission orders must be understandable by both humans and machines. Logical testing and command trust are possible using Semantic Web standards.
• Thanks to CRUSER and Raytheon for strong support during this many-year project. NPS-Raytheon CRADA partnership offers new possibilities for influence.
• Research insight: our university needs to establish a cross-disciplinary, cross-service NPS Center for Ethical Warfare.

Your NPS efforts matter. Talk to us about a potential thesis! This work holds multiple opportunities for applied student research in any major.

🔖 Data-Centric Security for Ethical Control of Unmanned Systems (slideset)

• 16 April 2020, duration 22:08. Briefing to Raytheon and NPS colleagues.
• Data-centric security enables strong command authority and trust between deployed commands and remote unmanned systems.
• Trusted communications must occur regardless of intervening challenges to network connections by the ocean-atmosphere environment or hostile countermeasures.
• Ethical control of unmanned systems requires reliable message exchange across long distances and durations of time.

🔖 Mission Design and Semantic Web Exemplars for Human Supervision of Lethal/Lifesaving Autonomy (slideset)

1. 6 April 2020, duration 1:01:31. Project briefings reprised to Raytheon colleagues, providing project overview of sponsored work performed as part of an NPS-Raytheon Cooperative Research and Development Agreement (CRADA).
2. 25 March 2020. Online brief to Naval Unmanned Vehicle and Autonomous Systems (UVAS) Working Group, providing project overview with emphasis on relevance to much naval research and future operations.
3. 9‑11 March 2020. Conference panel. US Semantic Technologies Symposium (US2TS) panel session: Hybrid AI for Context Understanding, North Carolina State University, Raleigh, NC. Also presented related work X3D Ontology for Semantic Web as a "Lightning Talk" there. (slideset)

AVCL mission diagrams show ternary logic for decision flow. Each goal can only result in success, failure or exception. Visual representations are quite useful for checking mission logic.

• test.all.missions.python.log.txt exhaustively tests all XML mission definitions
• Video: Python mission evaluation demonstration shows how a mission execution engine can read and simulate AVCL XML missions for individual runs or exhaustive testing.
• SailorOverboardMission8State.py example single mission
• SailorOverboardMission8StateInitialRun.txt
• SailorOverboardMission8StatePythonSyntaxHighlighted.html