| Kind | Value | Annotation |
| enumeration | RE01.1_IdentifiedTermsDevelopmentUse | | appinfo | | Have all parties identified all the important terms being used in the development and use of the IAS that require definition? |
|
| enumeration | RE01.10_SameDefinitionRiskFramework | | appinfo | | Are all parties using the same risk management framework? |
|
| enumeration | RE01.11_SameDefinitionsLifecycleTechnicalStandards | | appinfo | | Are all parties using the same technical standards throughout the entire lifecycle of the IAS? |
|
| enumeration | RE01.12_SameDefinitionsQuantitativeAnalysis | | appinfo | | Are all parties using the same metrics for quantitative analysis (e.g., analyzing confidence levels, comparing similarities, measuring differences)? |
|
| enumeration | RE01.2_ConsistentDoctrinalTerminology | | appinfo | | Are all parties (when they come from different organizations with different doctrine with respect to IAS use) using consistent and non-conflicting doctrinal terminology? |
|
| enumeration | RE01.3_RapidlyEmergingTerminology | | appinfo | | Does IAS use require the use of rapidly emerging terminology that must be defined and agreed upon before use? |
|
| enumeration | RE01.4_SameDefinitionsAI | | appinfo | | Are all parties using the same definitions for "artificial intelligence," "intelligent autonomous systems," "autonomy," "automatic," and "autonomous functionality?" |
|
| enumeration | RE01.5_SameDefinitionsPeaceWar | | appinfo | | Are all parties using the same definitions for "peacetime status" and wartime status?" |
|
| enumeration | RE01.6_SameDefinitionDegreeAutonomy | | appinfo | | Are all parties using the same definitions for IAS "degree of autonomy?" |
|
| enumeration | RE01.7_SameDefinitionRealisticOperationalEnvironment | | appinfo | | Are all parties using the same definition for "realistic operational environment" for IAS developmental and operational test and evaluation purposes? |
|
| enumeration | RE01.8_SameDefinitionsModelTestTrainValidate | | appinfo | | Are all parties using the same definitions for "training data, input data and feedback data?" |
|
| enumeration | RE01.9_SameDefinitionsOperationalPhases | | appinfo | | Are all parties using the same definitions for the several and distinct operational phases? |
|
| enumeration | RE02.1.1_ConsiderationsOperationalPhases | | appinfo | | Has the concept of operation been analyzed to verify that no temporal or spatial accountability gaps exist regarding the use of IAS? |
|
| enumeration | RE02.1.2_TransferCommandControl | | appinfo | | Has the concept of operation been analyzed to verify that no transfer of command and control over the IAS can occur without specific authorization by the person(s) designated to be accountable for the use of the IAS? |
|
| enumeration | RE02.1.3_SingleDecisionAuthority | | appinfo | | Is/are the person(s) designated to be accountable for the use of an IAS the only person(s) with the physical ability to transfer decision-making capability to the IAS? |
|
| enumeration | RE02.1.4_DesignatedChangeAuthority | | appinfo | | Is/are the person(s) designated to be accountable for the use of an IAS the only person(s) able to authorize in situ changes to the IAS’s configuration created by exposure to incoming data streams? |
|
| enumeration | RE02.1.5_ArmedConflictEthicsPolicyTraining | | appinfo | | Have all persons who may be designated to be accountable for the use of an IAS received training on the Law of Armed Conflict and ethics policies? |
|
| enumeration | RE02.1.6_UnderstandRulesEngagement | | appinfo | | Have all persons who may be designated to be accountable for the use of an IAS been briefed on the current and prevailing rules of engagement? |
|
| enumeration | RE02.1.7_UnderstandingDecisionAccountability | | appinfo | | Have all persons who may be designated to be accountable for the use of an IAS understand that transfer of decision-making capabilities to an IAS does not transfer accountability for the results of any decisions made by that IAS? |
|
| enumeration | RE02.1.8_SecureWartimeModes | | appinfo | | Can IAS lethal capabilities (or other capabilities authorized only for use during wartime) be disabled during peacetime and only be activated after a verifiable transmission is received from an accountable (military or civilian) authority? |
|
| enumeration | RE02.1.9_PreconfigurationFullAuto | | appinfo | | Can the IAS engagement parameters be pre-set to either allow or prohibit it from developing its own target selection, discrimination, or engagement criteria? |
|
| enumeration | RE02.2.1_HumanOveridesROEPerson | | appinfo | | Does the IAS allow for human judgment to be exercised over the IAS during this phase while complying with the relevant rules of engagement? |
|
| enumeration | RE02.2.2_DesignatedHumanOversightPerson | | appinfo | | If one is required, has the commander designated someone to be accountable for exercising human judgement over any IAS in use during this phase? |
|
| enumeration | RE02.2.3_EndOpUseAccountablePerson | | appinfo | | Is/are the accountable person(s) able to disable, redirect or recall the IAS if they obtain evidence that it may be operating in a manner contrary to law, policy, rules of engagement, or outside of expected technical parameters? |
|
| enumeration | RE02.2.4_MultipleSystemsComplexity | | appinfo | | Are there multiple connected IAS involved in the mission? |
|
| enumeration | RE02.2.5_CertifiedUnknownAutonomy | | appinfo | | Have all systems assessed as having no autonomous functionality been certified to that affect? |
|
| enumeration | RE02.2.6_IASSystemFullScopePerson | | appinfo | | Is/are the person(s) accountable for executing the tasks in this phase aware of all IAS available for use? |
|
| enumeration | RE02.2.7_IASSystemCertifiedPerson | | appinfo | | Is/are the person(s) accountable for executing the tasks in this phase trained and certified in the use of all IAS available for use? |
|
| enumeration | RE02.2.8_SingleControlPersonAccountable | | appinfo | | Will the IAS respond to instructions only from the person(s) designated to be accountable for the use of the IAS? |
|
| enumeration | RE02.2.9_ControlWithinMissionObjectives | | appinfo | | Will the IAS respond only to instructions that follow the commander’s intent or equivalent authoritative statement of the overarching mission objectives? |
|
| enumeration | RE02.3.1_TimeGapsControlChanges | | appinfo | | Can the IAS transfer command and control between entities authorized to exercise this command and control without creating any temporal gaps in accountability? |
|
| enumeration | RE02.3.2_InheritedControlAccountabilityDebt | | appinfo | | Is there a process by which the person(s) designated to be accountable for the use of an IAS in any particular phase are made aware of any decisions made by an IAS in a previous phase that they have "inherited" and are now accountable for? |
|
| enumeration | RE02.3.3_SystemIndicatesPositiveChangeControls | | appinfo | | Does the IAS clearly communicate the transfer of command and control to and/or from the human operator? |
|
| enumeration | RE02.3.4_OperationalBriefingControlAuthorities | | appinfo | | Is accountability for the use of the IAS, to include knowledge of what actions may transfer accountability to others or to themselves, understood by all participants in the operation? |
|
| enumeration | RE02.3.5_PersonControlForceFunctions | | appinfo | | Can the human operator take command and control of just the use of force functions and allow the IAS to continue to execute other tasks (i.e., navigation, sensor data ingest) autonomously? |
|
| enumeration | RE02.3.6_PartialAutonomyRequiresHuman | | appinfo | | Are there subsystems in the IAS that can continue to operate with full autonomy, while other subsystems need human judgment? |
|
| enumeration | RE02.3.7_JointOperationalBiasSystem | | appinfo | | Will the transfer of command and control between (culturally different) coalition and allied forces be subject to any cultural biases in training data sets that might affect IAS functions? |
|
| enumeration | RE02.3.8_SharedAccountabilityJointOperations | | appinfo | | Do the commander and human operators understand the implications of sharing accountability when transferring command and control in coalition and allied operations? |
|
| enumeration | RE02.3.9_HumanJudgementTakeControls | | appinfo | | Is the human operator sufficiently trained to appropriately judge when to take command and control even when the IAS does not recommend it? |
|
| enumeration | RE02.4.1_MoralBufferHuman | | appinfo | | Are protections in place to prevent the problem of the "moral buffer," where the user mentally transfers accountability for negative consequences to the machine? |
|
| enumeration | RE02.4.2_PersonAccountableFriendlyFire | | appinfo | | Has/have an accountable person(s) been identified for the inadvertent use of force (to include friendly-fire incidents) caused by the IAS? |
|
| enumeration | RE02.4.3_AwarenessAdversaryC2Loss | | appinfo | | Are enemy actions that result in loss of IAS command and control and subsequent negative outcomes sensed, communicated, and recorded? |
|
| enumeration | RE02.4.4_ReliableCommunicationsAccountability | | appinfo | | Are communications between the IAS and the person(s) designated to be accountable for its use reliable enough (e.g., consistent, frequent) to support this accountability? |
|
| enumeration | RE02.4.5_MicromanagedFirePerson | | appinfo | | Is command and control over the IAS limited to a human (mindlessly) pressing a "fire" button in response to indications from an IAS? |
|
| enumeration | RE02.4.6_OpsTeamCommunicationConstructive | | appinfo | | Is there communication between the human operators who are each accountable for understanding their individual functions of the IAS that they operate? |
|
| enumeration | RE02.4.7_AllOpsTeamAccountable | | appinfo | | Are all human operators accountable for the outcome of IAS use, even if they were only involved in a subcomponent of the overall mission? |
|
| enumeration | RE03.1_DegreeAutonomyAdjustable | | appinfo | | Can the degree of IAS autonomy be adjusted? |
|
| enumeration | RE03.10_IASDetermineAdversaryLoss | | appinfo | | Can the degree of IAS autonomy be chosen to allow the IAS to calculate probable enemy losses for a candidate action, compare these losses to how the candidate action contributes to the success of the mission, and make a "return on investment" assessment before taking the action? |
|
| enumeration | RE03.11_DegreeAutonomyModifiableViaMagazine | | appinfo | | Can the degree of IAS autonomy be adjusted downward for platforms with larger magazines in consideration of the increased level of risk that the larger magazine presumably entails? |
|
| enumeration | RE03.12_DegreeAutonomyModifiableViaEnvironmentRisks | | appinfo | | Can the degree of IAS autonomy be adjusted downward to reduce risk when operational conditions are such that there is a higher perceived risk to the IAS from either enemy action, environmental factors or malfunction? |
|
| enumeration | RE03.13_DegreeAutonomyMissionDependent | | appinfo | | Is the human operator fully equipped and prepared to take over if the IAS malfunctions or breaks down mid-operation? |
|
| enumeration | RE03.14_DegreeAutonomyAdustableIndividually | | appinfo | | Does the human operator fully understand the differences between an automatic system and an autonomous system? |
|
| enumeration | RE03.15_DegreeAutonomyLowestSelectable | | appinfo | | Can the degree of autonomy be adjusted based on various levels of abstraction (e.g., task, function, or mission)? |
|
| enumeration | RE03.2_DegreeAutonomySharable | | appinfo | | Can the degree of IAS autonomy be made known (through markings or public communications such as a notice to mariners) to supported friendly forces, to noncombatants, and/or to enemy combatants at the discretion of the operator or commander? |
|
| enumeration | RE03.3_ReportsRightSelfDefense | | appinfo | | Can the IAS make it known to anyone who might be in a position to disable, board, or seize it that it has - and will exercise - its universal right to self-defense? |
|
| enumeration | RE03.4_DegreeAutonomyAdjustableDynamically | | appinfo | | Can the commander or operator throttle the IAS to dynamically increase or decrease the degree of autonomy to adjust to the dynamics of the operational situation? |
|
| enumeration | RE03.5_AutonomyThresholdLethalForceWeaponSystem | | appinfo | | Is there an established and accepted threshold degree of autonomy, that when exceeded, the IAS becomes a LAWS, where a human no longer selects the target? |
|
| enumeration | RE03.6_DegreeAutonomyModifiableViaRuleSets | | appinfo | | Can the degree of IAS autonomy be conditionally changed, either through predetermined rule sets, or resulting from emergent information not covered by the predetermined rules, such as the elevation of the defense condition (DEFCON)? |
|
| enumeration | RE03.7_DegreeAutonomyMissionDependent | | appinfo | | Can the degree of IAS autonomy for a multi-mission capable platform be chosen and/or adjusted independently for each mission? |
|
| enumeration | RE03.8_DegreeAutonomyAdustableIndividually | | appinfo | | Can the degree of autonomy for each autonomous function of an IAS be adjusted individually? |
|
| enumeration | RE03.9_DegreeAutonomyLowestSelectable | | appinfo | | Can the degree of IAS autonomy be selected by the IAS such that it is the lowest degree of autonomy required to accomplish the mission? |
|
| enumeration | RE04.1.1_AlternativesAutonomousSystems | | appinfo | | Can the contemplated operation be executed with a non-autonomous alternative? |
|
| enumeration | RE04.1.10_OrganizationManningNecessitity | | appinfo | | Does the existing force structure and manning necessitate the use of IAS? |
|
| enumeration | RE04.1.11_PlatformRecommendsAutonomyLevel | | appinfo | | Can the IAS help determine whether to use autonomous, semi-autonomous, or non- autonomous functional modes based on current circumstances and conditions? |
|
| enumeration | RE04.1.12_OrganizationImprovedOutcomes | | appinfo | | Can the use of IAS improve military capabilities? |
|
| enumeration | RE04.1.13_UseReducesLossHumanLife | | appinfo | | Can the use of IAS reduce loss of human life? |
|
| enumeration | RE04.1.2_RulesAlternativeAutonomousSystems | | appinfo | | Do rules of engagement or policy require the commander to consider all available non-autonomous alternatives before employing an IAS? |
|
| enumeration | RE04.1.3_AdvantageAutonomousSystem | | appinfo | | Will failure to transfer decision-making capabilities to the IAS result in a military disadvantage? |
|
| enumeration | RE04.1.4_HumanFactorBoringTasking | | appinfo | | Is the contemplated task monotonous or fatiguing (i.e., "dull") to the point that human performance will degrade but IAS performance will not? |
|
| enumeration | RE04.1.5_HumanFactorDirtyHazardousMaterial | | appinfo | | Does the contemplated task put humans in harm’s way by exposing them to (dirty) hazardous materials? |
|
| enumeration | RE04.1.6_HumanExposureAdversary | | appinfo | | Does the contemplated task put humans in harm’s way by exposing them to (dangerous) enemy action or violent weather conditions? |
|
| enumeration | RE04.1.7_HumanFactorLossOperationalEffectiveness | | appinfo | | Does the use of the IAS deprive the commander of human ingenuity, creativity, flexibility, or (operational) art capabilities that makes the force less capable? |
|
| enumeration | RE04.1.8_AutonomousSystemSafer | | appinfo | | Can the IAS conduct an operation more safely than a human can? |
|
| enumeration | RE04.1.9_FunctionalNonAutomousSystemsAvailable | | appinfo | | Does the commander have any systems that provide the same capability as the IAS, but do not depend on AI or autonomy? |
|
| enumeration | RE04.2.1_SensorPerceptualSpace | | appinfo | | Is the perceptual space of the IAS clearly superior to that of the nonautonomous alternative (i.e., infrared or radio frequency emission detection)? |
|
| enumeration | RE04.2.2_SceneUnderstandingPerceptualDetail | | appinfo | | Can the IAS consistently catch important clues that human operators would miss or misconstrue? |
|
| enumeration | RE04.2.3_GreaterOperationalPersistance | | appinfo | | Does the IAS consistently have greater operational range or longer time on station than the nonautonomous alternative? |
|
| enumeration | RE04.2.4_GreaterManeuverability | | appinfo | | Can the IAS physically outmaneuver any enemy platforms it might face, for which the nonautonomous alternative cannot? |
|
| enumeration | RE04.2.5_GreaterTargetAccess | | appinfo | | Can the IAS consistently service targets that are inaccessible to the nonautonomous alternative? |
|
| enumeration | RE04.2.6_MultitargetTasking | | appinfo | | Can the IAS consistently conduct multiple tasks simultaneously, freeing up humans for other tasks that the human is better suited to than the IAS? |
|
| enumeration | RE04.3.1_OperationalPerceptualModelingEquipmentAdvantage | | appinfo | | Can the IAS create a model of a given object and differentiate the object (military versus civilian) as well or better than the nonautonomous alternative can? |
|
| enumeration | RE04.3.2_OperationalPerceptualModelingPeopleAdvantage | | appinfo | | Can the IAS create a model of a person and differentiate enemy combatants, friendly combatants, and noncombatants as well as, or better than, than the nonautonomous alternative can? |
|
| enumeration | RE04.3.3_DifferentConfidenceLevels | | appinfo | | Is a level of confidence required to be established for the IAS to distinguish a target in different operational situations? |
|
| enumeration | RE04.3.4_OutOfDistributionDetection | | appinfo | | Does the IAS know when a given object is outside of its training set (i.e., out-of-distribution detection)? |
|
| enumeration | RE04.4.1_RelativeTargetingMinimizeCollateralDamage | | appinfo | | Would the use of IAS allow the commander to use more precise munitions to minimize collateral damage? |
|
| enumeration | RE04.4.2_StopAttacking | | appinfo | | Can the IAS determine the appropriate level of proportionality in attack during the conduct of a mission (e.g., when to continue, when to cease action)? |
|
| enumeration | RE04.4.3_RelativeTargetIdentificationMinimizeBlast | | appinfo | | Can the IAS provide better identification and more accurately strike a specific location on a target with smaller munition and reduced blast radius? |
|
| enumeration | RE04.4.4_HighValueTargetIdentification | | appinfo | | Can the IAS discern whether or not the target is sufficiently valuable to risk collateral damage? |
|
| enumeration | RE04.5.1_DominatesEnvironmentOperationalPerception | | appinfo | | Is the speed of operation of the IAS in synch with the speed of the situational assessment? |
|
| enumeration | RE04.5.2_ActionSpeedErrorRates | | appinfo | | Does the superior decision speed of the IAS over the human justify the risk of an incorrect decision by the IAS that a human would not likely make incorrectly? |
|
| enumeration | RE04.5.3_FunctionSpeedNecessatesUse | | appinfo | | Will the IAS be used for situations where human reactions are too slow for an effective response? |
|
| enumeration | RE04.5.4_HumanInLoopSlowsSystem | | appinfo | | Will IAS decision-making speed decrease if it must allow human operators to oversee and approve its actions? |
|
| enumeration | RE04.5.5_OperationalWindowJustifiesUse | | appinfo | | Does the mission require action in windows of opportunity too short for effective human intervention/action? |
|
| enumeration | RE04.5.6_AdversaryAdvantageWithoutDecisionAutonomy | | appinfo | | Will the failure to transfer decision-making capabilities to the IAS result in the enemy having a faster OODA loop? |
|
| enumeration | RE04.6.1_EnvironmentChangeQuick | | appinfo | | Are battlefield information flows so fast as to justify a reliance on an IAS instead of a human decision maker? |
|
| enumeration | RE04.6.2_EnvironmentChangeLarge | | appinfo | | Are battlefield information flows so large as to justify a reliance on an IAS instead of a human decision maker? |
|
| enumeration | RE04.6.3_EnvironmentChangeNumerous | | appinfo | | Are battlefield information flows so varied as to justify a reliance on an IAS instead of a human decision maker? |
|
| enumeration | RE04.6.4_AdversaryIASSystemAccelerateEnvironmentDominance | | appinfo | | Will the use of the IAS cause the pace of events on the battlefield to accelerate beyond the point that the commander can comprehend the events and take decisive action? |
|
| enumeration | RE04.6.5_AdversaryIASRequiresIAS | | appinfo | | Will the enemy use of the IAS cause the pace of events on the battlefield to accelerate beyond the point that the commander can comprehend and control these events? |
|
| enumeration | RE04.7.1_OverwhelmAdversaryDecisionMaking | | appinfo | | Can the decisions of the IAS overwhelm the adversary’s decision-making process? |
|
| enumeration | RE04.7.2_ComplicateAdversaryLessPredictable | | appinfo | | Can the decisions of the IAS complicate the adversary’s decision-making process by being less predictable than the decisions of a human operator? |
|
| enumeration | RE04.7.3_IASFunctionalMultipleAdversaryDilemmas | | appinfo | | Can the IAS provide both speed and scale of action to impose multiple dilemmas on the enemy? |
|
| enumeration | RE04.7.4_IASFunctionalMultipleFluidKillChains | | appinfo | | Will the IAS enable superposition of multiple, fluidly composed and independent kill chains (or webs) that eliminate, or at least severely curtail, response options available to the enemy? |
|
| enumeration | RE04.7.5_IASFunctionalOverwhelmAdversary | | appinfo | | Can the IAS create an operational tempo that does not permit the adversary to regroup or concentrate? |
|
| enumeration | RE05.1.1_IASDetermineHumanControl | | appinfo | | Can the IAS determine if its (possibly changing) operational environment requires "human-in-the-loop," "human-on-the-loop", or "human-out-of-the-loop" control and make that requirement known to a human operator? |
|
| enumeration | RE05.1.10_NumberOperatorsSufficient | | appinfo | | Are the number of human operators available sufficient to the number of IAS requiring command and control? |
|
| enumeration | RE05.1.11_IASDetectEnemyControl | | appinfo | | Can the IAS detect enemy attempts to wrest command and control away and notify the human operator? |
|
| enumeration | RE05.1.12_IASDetectMalfunctions | | appinfo | | Can the IAS detect malfunctions or out of tolerance performance conditions that could result in a loss of command and control and notify the human operator? |
|
| enumeration | RE05.1.13_IASMonitorControlLink | | appinfo | | Can the IAS constantly monitor the availability of a control link, even when not under the direct and immediate direction of the human operator? |
|
| enumeration | RE05.1.14_IASProhibitedActiveMode | | appinfo | | Can an IAS operating in a passive mode be prohibited from changing to an active mode of operation absent a direct instruction to do so by a human operator? |
|
| enumeration | RE05.1.15_IASEngageEmergentTarget | | appinfo | | Can the IAS engage an emergent target, not on any preplanned list, without validation from a human operator? |
|
| enumeration | RE05.1.16_HumanRejectAutonomousDecision | | appinfo | | Can the human operator choose to reject automated decision-making capabilities at their own discretion and at any time? |
|
| enumeration | RE05.1.17_HumanSelectMinConfidence | | appinfo | | Can the human operator select a minimum confidence level that must be attained before the IAS can act without approval? |
|
| enumeration | RE05.1.18_IASForcedCheckIn | | appinfo | | Can the IAS be made to have to check in with a human operator and be in receipt of an acknowledgement before proceeding with further actions? |
|
| enumeration | RE05.1.19_IASIdentifyLostLink | | appinfo | | Can the IAS identify conditions that might cause it to lose access to the control link with the human operator? |
|
| enumeration | RE05.1.2_IASNewBehaviorsProhibited | | appinfo | | Is the IAS prohibited from learning and executing new behaviors based on sensor inputs or data feeds received once decision-making capabilities have been transferred to it? |
|
| enumeration | RE05.1.3_IASConsultHuman | | appinfo | | If the IAS is allowed to modify its behavior, does it consult the human operator beforehand? |
|
| enumeration | RE05.1.4_IASOperationInitiationProhibited | | appinfo | | Is the IAS prohibited from initiating operation in the absence of a control link to a human operator? |
|
| enumeration | RE05.1.5_IASMovingProhibited | | appinfo | | Is the IAS prohibited from moving to a location where the control link to a human operator can be degraded or lost? |
|
| enumeration | RE05.1.6_HumanControlSwarm | | appinfo | | Is the human operator still able to provide command and control over the actions of the IAS when it is employed as part of, and can act based on the conditions within, a swarm of other IAS? |
|
| enumeration | RE05.1.7_HumanChooseInLoop | | appinfo | | Is the human operator able to choose between a "human in, on, or out of the loop?" |
|
| enumeration | RE05.1.8_CommanderManageComplexity | | appinfo | | Does the IAS enable the commander to determine and/or manage the symmetries/asymmetries in the level of complexity between own forces and enemy forces? |
|
| enumeration | RE05.1.9_HumanTerminateEngagement | | appinfo | | If a situation becomes too complex or unfolds too rapidly for the human operator to comprehend, can the human operator terminate an engagement? |
|
| enumeration | RE05.2.1_IASActionsLimited | | appinfo | | Can the commander selectively limit the actions that an IAS can take before relinquishing command and control over it? |
|
| enumeration | RE05.2.2_CommanderUnderstandRisk | | appinfo | | Does the commander have a checklist of conditions that must be met, risk mitigation steps that have been taken, and an understanding of the risks being incurred, before relinquishing command and control over an IAS which possesses no lethal force capabilities, but whose autonomous operation could result in a lethal outcome via some secondary affect? |
|
| enumeration | RE05.2.3_CommanderAcknowledgeAccountabilityNonLethal | | appinfo | | Has the commander acknowledged to his or her supporting Staff Judge Advocate, before relinquishing command and control of the IAS, that by doing so he or she is not relinquishing accountability for the outcomes of its actions? |
|
| enumeration | RE05.2.4_CommanderAcknowledgeAccountabilityLethal | | appinfo | | Does the commander have a checklist of conditions that must be met, risk mitigation steps that have been taken, and an understanding of the risks being incurred, before relinquishing command and control over an IAS capable of selecting targets and applying lethal force to them? |
|
| enumeration | RE05.3.1_IASVerifyRequest | | appinfo | | Can the IAS verify that a request for transfer of command and control is from a human operator who is authorized to do so? |
|
| enumeration | RE05.3.2_IASDetermineControlTransferred | | appinfo | | Can the IAS determine when command and control has been transferred? |
|
| enumeration | RE05.3.3_IASNotifiedControlDelegated | | appinfo | | Is the IAS notified when the commander delegates control, to include target engagement authority, to a subordinate commander? |
|
| enumeration | RE05.3.4_IASTransferControlSafely | | appinfo | | Can the IAS transfer control back to a commander, or between commanders safely and in a manner that does not result in any lapses? |
|
| enumeration | RE05.3.5_TransferCausedLossC2 | | appinfo | | Can transfer of IAS control result in a loss of command and control, and can the transfer be delayed until a time when that possibility is at an acceptable minimum? |
|
| enumeration | RE05.3.6_IASDefaultMinimumRisk | | appinfo | | Can the IAS default to a minimal risk operating condition in the event that transfer of command and control was not successful? |
|
| enumeration | RE05.3.7_IASDetectAbsentOperator | | appinfo | | Can the IAS detect the absence or unavailability of a human operator before transferring command and control and default to a minimal risk operating condition until the human operator becomes available? |
|
| enumeration | RE05.3.8_IASControlledByAllies | | appinfo | | Can transfer of command and control of the IAS occur between coalition and allied forces? |
|
| enumeration | RE05.4.1_IASRecognizeEmergence | | appinfo | | Can the IAS recognize and cease the conduct of emergent behaviors? |
|
| enumeration | RE05.4.10_IASMisreadEnvironment | | appinfo | | Is the IAS susceptible to misread the environment and mismatch the appropriate action (e.g., come across a different environment but take the same action it has done before)? |
|
| enumeration | RE05.4.2_IASLearningProhibited | | appinfo | | Is the IAS prohibited from learning during operations where there is no human in the loop? |
|
| enumeration | RE05.4.3_IASModifyBehavior | | appinfo | | Can the IAS still learn but refrain from modifying behavior until evaluated by a human operator? |
|
| enumeration | RE05.4.4_CommandersAwareIASEmergence | | appinfo | | Are commanders made aware of IAS under their command that have the ability to learn during operations and execute emergent behaviors based on incoming sensor data? |
|
| enumeration | RE05.4.5_CommandersUnderstandEmergence | | appinfo | | Do commanders understand the potential occurrence of emergent behaviors with the IAS, based on its type and implementation of autonomous decisions? |
|
| enumeration | RE05.4.6_EmergenceThresholdsSet | | appinfo | | Are thresholds identified and set to alert the commander to the existence and nature of emergent behaviors? |
|
| enumeration | RE05.4.7_IASEmergenceLimitable | | appinfo | | Can the commander limit the ability of the IAS to learn during operations and exhibit emergent behavior? |
|
| enumeration | RE05.4.8_IASAlertsCodeChanges | | appinfo | | Can the IAS notify the commander when it alters its operating code? |
|
| enumeration | RE05.4.9_IASAlertActionsDeviate | | appinfo | | Can the IAS retain a record of every action it has ever taken and notify the commander when it is about to take an action that deviates from previous actions by a predefined threshold or standard? |
|
| enumeration | RE05.5.1_IASContinueSafely | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), continue autonomously to execute the most recent mission or task instruction if that is the safest option? |
|
| enumeration | RE05.5.10_IASReduntantCCSystems | | appinfo | | Does the IAS have multiple redundancies in critical systems that enable graceful degradation of performance instead of a catastrophic loss of command and control? |
|
| enumeration | RE05.5.11_IASSwitchBackupSystems | | appinfo | | Does the IAS have internal sensors capable of detecting current or impending system failures or battle damage such that it can autonomously switch to an appropriate redundant backup system? |
|
| enumeration | RE05.5.2_IASMaintainMotion | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), hold speed and course or loiter to await re-establishment of command and control? |
|
| enumeration | RE05.5.3_IASAbortMission | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), abort mission and return to base? |
|
| enumeration | RE05.5.4_IASInitiateEmergencyComms | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), initiate emergency (and possibly non-secure) communications to receive further instructions? |
|
| enumeration | RE05.5.5_IASSelfDestruct | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), destroy itself? |
|
| enumeration | RE05.5.6_IASProhibitedSelectingTargets | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), be prohibited from autonomous selection and engagement of individual targets that have not been previously selected by an authorized human operator? |
|
| enumeration | RE05.5.7_IASProhibitedSelectingTargetGroups | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), be prohibited from autonomous selection and engagement of specific target groups that have not been previously selected by an authorized human operator? |
|
| enumeration | RE05.5.8_IASProhibitedTargetingDecisions | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), be prohibited from autonomously making any hostile and/or targeting decision whatsoever? |
|
| enumeration | RE05.5.9_IASFollowHumanIntent | | appinfo | | Can the IAS, in the event that command and control is lost for any reason (and based on pre-determined criteria), follow the original human intent regarding target selection? |
|
| enumeration | RE05.6.1_OperatorVerifyC2 | | appinfo | | Can the human operator verify that he/she has re-established command and control to prevent an IAS from continuing to execute failsafe protocols? |
|
| enumeration | RE05.6.2_IASAlertRecoveryCrew | | appinfo | | Does the IAS immediately alert emergency service providers and recovery crews in the event of a collision or malfunction who can verify safe operating condition before returning the IAS to service? |
|
| enumeration | RE06.10.1_CanDetectSurrenderViolation | | appinfo | | Can the IAS detect when a surrendering enemy combatant has violated the terms of surrender by attempting to escape? |
|
| enumeration | RE06.10.2_CanDetectHostileActViolation | | appinfo | | Can the IAS detect when a surrendering enemy combatant has violated the terms of surrender by committing a hostile act? |
|
| enumeration | RE06.10.3_CanInferPerfidiousActsPeople | | appinfo | | Can the IAS detect and communicate perfidious acts to human operators? |
|
| enumeration | RE06.1.1_CanCalculateConfidenceFactor | | appinfo | | Can the IAS calculate a confidence factor for positive identification assessments of persons and objects protected from the use of force and prohibit the use of force if that confidence factor is below a minimum threshold set by the commander? |
|
| enumeration | RE06.11.1.1_CanEnsureProtectionMilitaryMedicalPeople | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) military medical personnel? |
|
| enumeration | RE06.11.1.2_CanEnsureProtectionMilitaryReligiousLeader | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) military religious personnel? |
|
| enumeration | RE06.11.1.3_CanEnsureProtectionHumanShields | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) human shields (persons placed around or within valid military target to hinder attack)? |
|
| enumeration | RE06.11.2.1_CanRecognizeAgreedSurrenderSignals | | appinfo | | Can the IAS detect and perceive any signal of surrender that all combatant sides have previously agreed to use? |
|
| enumeration | RE06.11.2.2_CanRecognizeSurrenderTrainedPeople | | appinfo | | Can the IAS detect and perceive indications of surrender from enemy combatants that have been trained on how to indicate surrender to the IAS? |
|
| enumeration | RE06.11.2.3_CanRecognizeSurrenderUntrainedPeople | | appinfo | | Can the IAS detect and perceive indications of surrender from enemy combatants that have not been trained on how to indicate surrender to the IAS? |
|
| enumeration | RE06.11.2.4_CanIdentifyReadySurrender | | appinfo | | Can the IAS detect and perceive the changes in orientation and force posture of combatants that are commonly associated with the act of surrender? |
|
| enumeration | RE06.11.2.5_CanIdentifyRetreatingSurrenderingIncapacitatedPerson | | appinfo | | Can the IAS distinguish between retreating and surrendering enemy forces? Incapacitated persons |
|
| enumeration | RE06.11.2.6_CanIdentifyWoundedSickPerson | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) wounded or sick persons? |
|
| enumeration | RE06.11.2.7_CanIdentifyShipwreckedPerson | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) shipwrecked persons? |
|
| enumeration | RE06.11.3.1_CanProtectUSCitizen | | appinfo | | Can the IAS detect and not intentionally harm US citizens in a manner that would violate Constitutional protections? |
|
| enumeration | RE06.11.3.2_CanIdentifyUSCitizenEnemyCombatant | | appinfo | | Can the IAS detect and distinguish the presence of US citizens who are acting as enemy combatants? |
|
| enumeration | RE06.11.4.1_CanIngestBlueForceTracker | | appinfo | | Can the IAS ingest data from all blue force tracker systems? |
|
| enumeration | RE06.11.4.2_CanCalculateConfidenceFactorFriendlyForces | | appinfo | | Can the IAS obtain, analyze and use information to assess risk to friendly forces and calculate a confidence factor for each assessment? |
|
| enumeration | RE06.11.4.3_CanCalculateConfidenceFactorIFFPlatforms | | appinfo | | Can the IAS detect, distinguish and designate (crewed) vessels and aircraft as friendly, neutral, or adversarial and calculate a confidence factor for each designation? |
|
| enumeration | RE06.11.4.4_CanCalculateConfidenceFactorEnemyFriendlyNoncombatePeople | | appinfo | | Can the IAS distinguish between enemy combatants, friendly force personnel, and noncombatant individuals and calculate a confidence factor for each designation? |
|
| enumeration | RE06.1.2_CanTaskImprovePerceptionConfidence | | appinfo | | Can the IAS task own platform or other platform sensors to gather additional data to improve positive identification assessments when the confidence factors are below the minimum threshold set by the commander? |
|
| enumeration | RE06.12.1_CanLeverageProtectedObjectInterdependencies | | appinfo | | Can the IAS detect, or access information regarding, the interdependencies that objects protected from the use of force have with other objects (that may or may not be protected from the use of force)? |
|
| enumeration | RE06.12.2_CanLeverageRestrictedTargetsLocations | | appinfo | | Can the IAS access an up-to-date and complete list of restricted targets and areas? |
|
| enumeration | RE06.12.3_CanLeverageAdverseDestructionObjectsList | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) objects whose destruction might damage relations with local noncombatant populations? |
|
| enumeration | RE06.12.4_CanLeverageAdverseDestructionPropagandaList | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) objects whose destruction might provide propaganda value to enemy forces? |
|
| enumeration | RE06.12.5_CanDisseminateIdentifiedProtectedObjects | | appinfo | | Can the IAS provide information regarding objects protected from the use of force that will assist a human operator in making use of force decisions? |
|
| enumeration | RE06.1.3_CanCallbackImproveConfidence | | appinfo | | Can the IAS seek additional "reach back" intelligence information to improve positive identification assessments when the confidence factor is below the minimum threshold set by the commander? |
|
| enumeration | RE06.13.1_SafeListLocations | | appinfo | | Can the IAS determine when enemy combatants have entered into an area or structure that is protected from the use of force? |
|
| enumeration | RE06.13.2_SafelistObjects | | appinfo | | Can the IAS use signatures to distinguish military from civilian objects that are unique to either the military object only or the civilian object only? |
|
| enumeration | RE06.13.3_SafelistPeople | | appinfo | | Can the IAS account for unknown transient civilian or noncombatant personnel and/or equipment near a targetable military object? |
|
| enumeration | RE06.1.4_CanCommanderUpdateConfidenceThreshold | | appinfo | | Can the IAS decision not to act based on a confidence factor below the minimum threshold set by the commander be overridden by that commander? |
|
| enumeration | RE06.14.1_DetectObjectsFunctionalLoss | | appinfo | | Can the IAS detect a "mission kill," where an object has been damaged but not destroyed, to the point that it no longer has military utility and cease further use of force? |
|
| enumeration | RE06.1.5_CanOtherCommandersUpdateConfidenceThreshold | | appinfo | | Can the IAS decision not to act based on a confidence factor below the minimum threshold set by the commander be overridden by other commanders who have had that authority delegated to them by the commander? |
|
| enumeration | RE06.15.1_TargetingLongtermHealth | | appinfo | | Can the IAS prohibit engagements that would cause widespread, long-term, and/or severe impact to the health of general population (such as the destruction of a power plant that services a hospital or a refugee camp)? |
|
| enumeration | RE06.1.6_CanIASBeGeofenced | | appinfo | | Can the IAS deployment be limited to a well-defined area where only enemy combatants or military objects are present? |
|
| enumeration | RE06.16.1_ObjectIllIntent | | appinfo | | Can the IAS detect perfidious use of objects protected from the use of force (such as using an ambulance for military transport)? |
|
| enumeration | RE06.16.2_LocationIllIntent | | appinfo | | Can the IAS detect perfidious use of facilities protected from the use of force (such as using a house of worship as a sniper position? |
|
| enumeration | RE06.1.7_CanIASDetectUnknownPeopleInTarget | | appinfo | | Can the IAS detect the presence of (uncategorized) humans near or within a targeted object? |
|
| enumeration | RE06.17.1_SafeObjectsSovereignLocations | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) diplomatic offices, foreign missions, and the sovereign nonmilitary properties of other nations? |
|
| enumeration | RE06.17.10_SafeObjectsPOWDetentionLocations | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) prisoner of war camps and government detention facilities/prisons? |
|
| enumeration | RE06.17.11_SafeObjectsPreventsToxicRelease | | appinfo | | Can the IAS identify and not intentionally initiate (in a manner that would violate laws, policies, or the rules of engagement) engagements that may result in pollution or have potential to release toxic chemicals? |
|
| enumeration | RE06.17.12_SafeObjectsPreventsFloodingCivilians | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) dams or dikes whose engagement may result in flooding of civilian areas? |
|
| enumeration | RE06.17.13_SafeObjectsSpace | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) facilities whose engagement may threaten astronauts and/or manned space flight missions? |
|
| enumeration | RE06.17.14_SafeObjectsCivilianMeetingPlaces | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) civilian meeting places? |
|
| enumeration | RE06.17.15_SafeObjectsPublicUtilitiesFacilities | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) public utilities and facilities? |
|
| enumeration | RE06.17.16_SafeObjectsUnknown | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) facilities and/or structures with unknown functionality/purpose? |
|
| enumeration | RE06.17.17_SafeAgriculturalObjects | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) agricultural processing and storage facilities? |
|
| enumeration | RE06.17.18_SafObjectsDualUse | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) facilities that provide products or services for both civilian and military use? |
|
| enumeration | RE06.17.2_SafeObjectsSpecialLocations | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) religious, cultural, historical institutions, cemeteries, and structures? |
|
| enumeration | RE06.17.3_SafeObjectsMixedMedical | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) fixed medical facilities? |
|
| enumeration | RE06.17.4_SafeObjectsMobileMedical | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) mobile medical facilities? |
|
| enumeration | RE06.17.5_SafeObjectsMedicalGroundTransport | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) ambulances and clearly marked medical ground transport vehicles? |
|
| enumeration | RE06.17.6_SafeObjectsMedicalAircraft | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) air ambulances and clearly marked medical aircraft? |
|
| enumeration | RE06.17.7_SafeObjectsHospitalShips | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) hospital ships? |
|
| enumeration | RE06.17.8_SafeObjectsPublicEducationLocations | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) public education facilities? |
|
| enumeration | RE06.17.9_SafeObjectsRefugeeLocations | | appinfo | | Can the IAS identify and not intentionally harm (in a manner that would violate laws, policies, or the rules of engagement) civilian refugee camps? |
|
| enumeration | RE06.1.8_CanIASYieldCrashDamage | | appinfo | | Can an IAS, even if unarmed, result in collateral damage in the event of a malfunction (such as a crash)? |
|
| enumeration | RE06.2.1_CanRecognizeSafeSymbols | | appinfo | | Can the IAS recognize symbols that designate persons and objects protected from the use of force, such as a Red Cross or Red Crescent? |
|
| enumeration | RE06.3.1_CanPredictDamage | | appinfo | | Can the IAS estimate potential collateral damage and conduct collateral damage assessments before going forth with force decisions (from either the IAS or a human operator)? |
|
| enumeration | RE06.3.10_CanUpdateCollateralEffectModels | | appinfo | | Can the IAS ingest and utilize collateral effects analysis results prepared by the commander’s staff? |
|
| enumeration | RE06.3.11_CanDisseminateCollateralDamageAccuracyUpdates | | appinfo | | Can the IAS identify any temporal, spatial or technical limits on the accuracy of its collateral damage estimates and convey those to the operator/commander? |
|
| enumeration | RE06.3.12_CanIdentifyUnacceptableCollageralDamageRisk | | appinfo | | Can the IAS determine when its actions might pose a collateral damage hazard to persons or objects protected from the use of force, cease the potentially hazardous action, and communicate its inability to take this action to the commander? |
|
| enumeration | RE06.3.13_CanRequestAvailableMunition | | appinfo | | Can the IAS seek approval to use a larger or less precise munition if its supply of smaller and/or more precise munitions is exhausted? |
|
| enumeration | RE06.3.2_CanDifferencePredictedDamage | | appinfo | | Can the IAS conduct collateral damage assessments and compare them to the commander’s proportionality assessment for consistency? |
|
| enumeration | RE06.3.3_CanDisseminatePredictedDamage | | appinfo | | Can the IAS provide commanders and operators with the information needed to assess proportionality when the IAS cannot do so without human assistance? |
|
| enumeration | RE06.3.4_CanPredictMunitionDamagePattern | | appinfo | | Can the IAS obtain, analyze and use information on the effects radii and patterns of munitions to determine if persons or objects protected from the use of force (to include friendly combatants) are within those radii and patterns? |
|
| enumeration | RE06.3.5_CanAcceptCollateralEffectsLimitUpdates | | appinfo | | Can the IAS access the most up-to-date assessments of collateral effects radius assessments prepared by the commander’s staff? |
|
| enumeration | RE06.3.6_CanPredictSecondaryDamage | | appinfo | | Can the IAS obtain, analyze and use information on the potential for secondary explosions before delivering a munition? |
|
| enumeration | RE06.3.7_DamageModelRepairFidelity | | appinfo | | Can the IAS distinguish between actions that cause repairable/reversible harm and actions that cause irreparable/irreversible harm? |
|
| enumeration | RE06.3.8_CanUpdateDamageAssessments | | appinfo | | Can the IAS assist the proportionality assessment summary by tracking assessments and the confidence levels thereof? |
|
| enumeration | RE06.3.9_CanUpdateDamageModels | | appinfo | | Can the IAS incorporate outcomes of previous operations that resulted in unacceptable collateral damage to prevent a recurrence? |
|
| enumeration | RE06.4.1_CanSwitchMilitaryNecessityMode | | appinfo | | Can the IAS analyze the military necessity and appropriateness of an attack? |
|
| enumeration | RE06.4.2_CanMeasureTargetsDestructionContribution | | appinfo | | Can the IAS assess the contribution that destruction of a target would make toward achieving a legitimate military objective? |
|
| enumeration | RE06.4.3_CanSupportMilitaryObjectives | | appinfo | | Can the IAS deconflict requirements to not intentionally harm persons or objects (in a manner that would violate laws, policies, or the rules of engagement) with its requirement to support achieving the military objective? |
|
| enumeration | RE06.4.4_CanAbortNonmilitaryObjective | | appinfo | | Can the IAS cancel or suspend an attack if it becomes apparent that the objective is not a military one? |
|
| enumeration | RE06.4.5_CanAbortTargetsValueChange | | appinfo | | Can the IAS abort an attack if the target no longer has military value because some other target that it depends on for its military value has been destroyed? |
|
| enumeration | RE06.5.1_CanWeaponsSafeLocations | | appinfo | | Can the IAS reduce or disable any prompts that would activate the use of force when it is operating in, or transiting through, an uncontested area? |
|
| enumeration | RE06.5.2_CanIASAcceptWeaponsFree | | appinfo | | Can the IAS determine when the thresholds to authorize the use force by a human commander or operator have lowered over time, due to a growing indifference to taking human life or creating damage? |
|
| enumeration | RE06.5.3_WillMilitaryPersonBecomeNumb | | appinfo | | Will repeated employment of the IAS as a LAWS numb the human operator or commander to the taking of human life? |
|
| enumeration | RE06.6.1_CanDirectMedicalTasking | | appinfo | | Can the IAS determine who receives medical attention? |
|
| enumeration | RE06.6.2_CanWarnHumansInteractionHazards | | appinfo | | Can the IAS make humans present in the operational environment aware of any hazards caused by interacting with it? |
|
| enumeration | RE06.7.1_CanDistinguishCurrentHumanCombatantStatus | | appinfo | | Can the IAS distinguish between enemy combatants, friendly combatants, neutral combatants, and noncombatants? |
|
| enumeration | RE06.7.10_CanRecognizeHostileIntention | | appinfo | | Can the IAS distinguish between benign and hostile intentions? |
|
| enumeration | RE06.7.11_CanRecognizeHostileActions | | appinfo | | Can the IAS distinguish between benign and hostile actions? |
|
| enumeration | RE06.7.2_CanDistinguishHumanIntrinsicCharacteristics | | appinfo | | the IAS be prohibited from making determinations regarding who will be harmed - in a situation where harm to someone is unavoidable - based on any intrinsic characteristics of a person such as age, gender, or other physical or mental conditions? |
|
| enumeration | RE06.7.3_CanDamageModelUnknownHuman | | appinfo | | Can the IAS make determinations regarding who will be harmed - in a situation where harm to someone is unavoidable - based on an individual’s status as an enemy combatant, a friendly combatant, or a noncombatant person? |
|
| enumeration | RE06.7.4_CanIdentifyUnknownPerson | | appinfo | | Can the IAS account for unknown transient noncombatant personnel in or near a target area? |
|
| enumeration | RE06.7.5_CanIdentifyAdversarialHumansObjects | | appinfo | | Can the IAS distinguish between adversarial human combatants and adversarial equipment and adjust the effect accordingly? |
|
| enumeration | RE06.7.6_CanAvoidUnknownStatusPeople | | appinfo | | Can the IAS default to categorizing a person as an "unknown," and not engage in instances where positive identification is below the required confidence level? |
|
| enumeration | RE06.7.7_CanUsePoLStatusPeople | | appinfo | | Can the IAS obtain, analyze and use information on civilian patterns of life to distinguish noncombatants from combatants? |
|
| enumeration | RE06.7.8_CanBreakEngagementInterspersedNoncombatants | | appinfo | | Can the IAS determine when enemy combatants have interspersed into a group of noncombatants that cannot be engaged with force? |
|
| enumeration | RE06.7.9_CanUseCombatantOnlySignatures | | appinfo | | Can the IAS use signatures to distinguish combatants from noncombatants that are unique to either the combatant only or the noncombatant only? |
|
| enumeration | RE06.8.1_CanRecognizeNeutralizedPerson | | appinfo | | Can the IAS detect a "mission kill," where a person has been incapacitated and rendered hors de combat but not killed, and cease further use of force? |
|
| enumeration | RE06.8.2_CanRecognizeNeutralizedObject | | appinfo | | Can the IAS detect a "mission kill," where a crewed vehicle has been damaged but not destroyed, rendering the crew hors de combat, and cease further use of force? |
|
| enumeration | RE06.9.1_CanMinimizeCollateralDamage | | appinfo | | Can the IAS prohibit the use of any action, method or munition available to it that could conceivably cause collateral injuries or death or cause unnecessary suffering? |
|
| enumeration | RE06.9.2_CanModelNoncombatantCasualties | | appinfo | | Can the IAS prohibit the use of any action, method or munition available to it that could conceivably result in unacceptable amounts of noncombatant casualties? |
|
| enumeration | RE06.9.3_CanIdentifyPreventWidespreadLTSevereHealthPeople | | appinfo | | Can the IAS prohibit engagements that would cause widespread, long-term, and/or severe impact to the health of persons protected from the use of force? |
|
| enumeration | RE06.9.4_CanMinimizeInjuryPeople | | appinfo | | Can the IAS conduct an analysis to minimize the severity of injury, when causing some injury is unavoidable? |
|
| enumeration | RE07.1.1_PreoperationalDocumentationAvailable | | appinfo | | Is all pre-operational documentation made visible and accessible to the greatest extent possible, limited only by the potential creation of operational vulnerabilities? |
|
| enumeration | RE07.1.2_OpenChannelUnethicalIllegalApplicationsNoretaliation | | appinfo | | Is a formal procedure in place whereby individuals with concerns over unethical or illegal application of an IAS can (and are required to) document those concerns without fear of retaliation? |
|
| enumeration | RE07.1.3_RDTLifecycleDocumentationPersistance | | appinfo | | Is positive control of the individual IAS and all of its subcomponents documented as they move from one organization to the next in the research, development, and test and evaluation process? |
|
| enumeration | RE07.1.4_IASSystemClaimsMaterialsAvailableInspectionOversight | | appinfo | | Are all commercial vendor marketing materials and claims about acquired IAS available for inspection and oversight? |
|
| enumeration | RE07.2.1_TrainingDataProvananceCaptured | | appinfo | | Is the chain of custody for all IAS training data documented from creation to final use? |
|
| enumeration | RE07.2.10_TrainingDataLabelsDescriptionsDocumented | | appinfo | | Are the descriptions and labels that describe the IAS training data documented? |
|
| enumeration | RE07.2.11_TrainingDataSuspect | | appinfo | | Does the IAS use any open-source training data with absent or suspect provenance documentation? |
|
| enumeration | RE07.2.12_IASInjestSubsystemVerifications | | appinfo | | Can a higher-level IAS verify the reliability of inputs from its subsystem components that may or may not be autonomous? |
|
| enumeration | RE07.2.13_DataConsistentWithVerificationDesignUse | | appinfo | | Is the training data for the overall IAS the same training data that was used to design and verify the component autonomous subsystems? |
|
| enumeration | RE07.2.2_TrainingDataAccessDocumented | | appinfo | | Are all requests for sharing of IAS training data documented? |
|
| enumeration | RE07.2.3_TrainingDataAccessRestrictions | | appinfo | | Are there access controls and processes in place to ensure that only authorized persons may make modifications to IAS training data? |
|
| enumeration | RE07.2.4_TrainingDataIdentidyDocumented | | appinfo | | Is the identity of all persons who accessed, or who authorized access to the IAS training data documented? |
|
| enumeration | RE07.2.5_TrainingDataVersioningCaptured | | appinfo | | Are the current, and all previous versions of the IAS training data retained for inspection? |
|
| enumeration | RE07.2.6_TrainingDataChangesCaptured | | appinfo | | Are all modifications to the IAS training data documented? |
|
| enumeration | RE07.2.7_TrainingDataInspectionReady | | appinfo | | Is the IAS training data available for inspection immediately prior to use? |
|
| enumeration | RE07.2.8_TrainingDataRetentionPolicyDocumented | | appinfo | | Is the IAS training data retention policy documented? |
|
| enumeration | RE07.2.9_TrainingDataSourcesCaptured | | appinfo | | Is the nature and origin of all IAS training data documented? |
|
| enumeration | RE07.3.1_SourceCodeLifecycleChangeControlledCaptured | | appinfo | | Is the chain of custody for all IAS algorithm’s source code documented from creation to final use? |
|
| enumeration | RE07.3.10_DataLabelDescriptionsDocumented | | appinfo | | Are the descriptions and labels that describe the IAS algorithms documented? |
|
| enumeration | RE07.3.11_AlgorithmsDependesCodeSuspectProvenanceCaptured | | appinfo | | Do the IAS algorithms use any open-source code with absent or suspect provenance documentation? |
|
| enumeration | RE07.3.12_IASTechnicalArchitectureAvailable | | appinfo | | Is the design documentation describing the IAS algorithms technical architecture available for inspection? |
|
| enumeration | RE07.3.13_IdentifyPoorCodeSecurityVulnerabilities | | appinfo | | Are code scanning tools used that can identify poor coding practices and common security vulnerabilities? |
|
| enumeration | RE07.3.2_RequestsForSharingDocumented | | appinfo | | Are all requests for sharing of IAS algorithms documented? |
|
| enumeration | RE07.3.3_AlgorithmDevelopmentAccessControls | | appinfo | | Are there access controls and processes in place to ensure that only authorized persons may make modifications to IAS algorithms? |
|
| enumeration | RE07.3.4_IdentityAlgorithmAccessCaptured | | appinfo | | Is the identity of all persons who accessed, or who authorized access to IAS algorithms documented? |
|
| enumeration | RE07.3.5_RetentionAlgorithmsVersioning | | appinfo | | Are the current, and all previous versions of the IAS algorithms retained for inspection? |
|
| enumeration | RE07.3.6_AlgorithmModificationsCaptured | | appinfo | | Are all modifications to the IAS algorithms documented? |
|
| enumeration | RE07.3.7_IASAlgorithmsAvailableInspection | | appinfo | | Are the IAS algorithms available for inspection immediately prior to use? |
|
| enumeration | RE07.3.8_AlgorithmRetentionPolicyDocumented | | appinfo | | Is the IAS algorithm retention policy documented? |
|
| enumeration | RE07.3.9_AlgorithmNatureOriginDocumented | | appinfo | | Is the nature and origin of all IAS algorithms documented? |
|
| enumeration | RE07.4.1_ComputerHardwareProvenanceDocumented | | appinfo | | Is the chain of custody for all IAS computer hardware components documented from creation to final use? |
|
| enumeration | RE07.4.2_ComputerHardwarePersonAccessDocumented | | appinfo | | Is the identity of all persons who accessed, or who authorized access to IAS computer hardware components documented? |
|
| enumeration | RE07.4.3_AccessControlsComputerHardware | | appinfo | | Are there access controls and processes in place to ensure that only authorized persons may make modifications to hardware or software? |
|
| enumeration | RE07.4.4_ComputerHardwareModificationRepairsLogged | | appinfo | | Are all modification and repairs to the IAS computer hardware components documented? |
|
| enumeration | RE07.4.5_ComputerHardwareInspectionReady | | appinfo | | Are the IAS computer hardware components available for inspection immediately prior to use? |
|
| enumeration | RE07.4.6_NatureOriginComputerHardwareDocumented | | appinfo | | Is the nature and origin of all IAS computer hardware components documented? |
|
| enumeration | RE07.4.7_ComputerHardwareSuspectProvenanceDocumended | | appinfo | | Does the IAS use any commercial off the shelf computer hardware components with absent or suspect provenance documentation? |
|
| enumeration | RE07.4.8_ComputerHardwareTechnicalArchitectureInspectionReady | | appinfo | | Is the design documentation describing the IAS computer hardware components technical architecture available for inspection? |
|
| enumeration | RE07.5.1_CircumventionSafetyPoliciesRegulations | | appinfo | | Has the IAS been acquired using any waivers allowing circumvention of safety policies or regulations? |
|
| enumeration | RE07.5.10_DoDDirective300009LAWSApproval | | appinfo | | If the IAS has the potential to be considered a LAWS by DoD Directive 3000.09, has the IAS undergone the required review and approval process outlined in this policy? |
|
| enumeration | RE07.5.2_CircumventionEthicsPoliciesRegulations | | appinfo | | Has the IAS been acquired using any waivers allowing circumvention of ethics policies or regulations? |
|
| enumeration | RE07.5.3_CircumventionDevOpsVerificationValidation | | appinfo | | Has the IAS been acquired using any waivers allowing circumvention of any developmental or operational testing, verification, and validation? |
|
| enumeration | RE07.5.4_CircumventionRiskDebt | | appinfo | | Has the IAS been acquired using any accelerated/rapid/other acquisition authorities that dismiss or delay risk mitigating steps otherwise present in the deliberate acquisition system? |
|
| enumeration | RE07.5.5_WaiversInspectionReady | | appinfo | | Are all acquisition and development-related waivers documented and available for inspection? |
|
| enumeration | RE07.5.6_TestEvalutionPlanSecurelyAvailable | | appinfo | | Is the IAS test and evaluation plan made public to the greatest extent possible (without divulging operational vulnerabilities)? |
|
| enumeration | RE07.5.7_EthicalRiskMitigationDocumentation | | appinfo | | Have the IAS developers produced ethical risk mitigation documentation for the IAS similar to the technical risk mitigation documentation already required by acquisition regulations? |
|
| enumeration | RE07.5.8_CheckInvalidApprovalDocuments | | appinfo | | Do IAS approval documents record any conditions of use under which the approval would be invalid or rescinded? |
|
| enumeration | RE07.5.9_RetestingDocumentationAvailable | | appinfo | | Are all conditions that would require re-testing of a fielded IAS documented and made available to the end user? |
|
| enumeration | RE07.6.1_JustificationsMissingLegalReview | | appinfo | | Are any justifications for not conducting a legal review of a new or upgraded IAS documented and publicized? |
|
| enumeration | RE07.6.2_OversightBodyIndependenceChecks | | appinfo | | Is the independence of any and all required IAS oversight bodies or regulatory authorities documented and publicized? |
|
| enumeration | RE07.6.3_ConflictInterestScreeningRDTEPersons | | appinfo | | Have conflict of interest screenings been conducted for all persons involved in the research, development, test and evaluation processes? |
|
| enumeration | RE07.6.4_ConflictsInterestMitigationsWaived | | appinfo | | Have all identified conflicts of interest been either mitigated or waived? |
|
| enumeration | RE07.6.5_ConflictsJustificationsDocumentedPublished | | appinfo | | Have the justifications for conflict of interest mitigations and/or waivers been documented and publicized? |
|
| enumeration | RE08.1.1_OperationalDataFieldsAuditRetention | | appinfo | | Is the collection of IAS operational data fields and their retention in an IAS operational audit log mandatory? |
|
| enumeration | RE08.1.10_OperationalDataLogProcessUnthicalUseReporting | | appinfo | | Is a procedure in place whereby individuals who have examined IAS operational audit logs and have concerns regarding the potential unethical or illegal use, can make those concerns known without fear of retaliation? |
|
| enumeration | RE08.1.11_OperationalAuditDataC2TransferFields | | appinfo | | Are transfers of IAS command and control documented? Operational audit log data fields |
|
| enumeration | RE08.1.12_OperationalEnvironmentSensedObjectMetadata | | appinfo | | Does the IAS collect metadata associated with all sensed objects in its operation environment that includes object identification, confidence metrics associated with its state values (i.e., position , velocity, etc.), and historical state information? |
|
| enumeration | RE08.1.13_MetadataIncludesTimestamps | | appinfo | | Does the IAS collect time stamp metadata for all data fields and during the entire mission duration? |
|
| enumeration | RE08.1.14_SystemLogsCriticalInternalSystemConditions | | appinfo | | Does the IAS collect data on the operating conditions of all critical internal systems? |
|
| enumeration | RE08.1.15_IASRecordsC2 | | appinfo | | Does the IAS record all commands it receives? |
|
| enumeration | RE08.1.16_IASRecordIdentitesAccountableOperatorsCommanders | | appinfo | | Does the IAS record the identities of the accountable operator(s) and commander(s)? |
|
| enumeration | RE08.1.17_IASRecordsDecisions | | appinfo | | Does the IAS record actions taken, and decisions made by the IAS and by the human operators? |
|
| enumeration | RE08.1.18_IASRecordsSensedActionsPeople | | appinfo | | Does the IAS record the sensed actions of friendly force personnel, adversary force personnel and noncombatant persons? |
|
| enumeration | RE08.1.19_IASRecordsAlgorithmsUse | | appinfo | | Does the IAS record all algorithms in use (to include any version number) that are utilized to produce an action/decision? |
|
| enumeration | RE08.1.2_OperationalDataSupportingSensors | | appinfo | | Is the IAS instrumented with the internal and external environmental sensors needed to support the collection of all required operational audit log data fields? |
|
| enumeration | RE08.1.20_IASRecordsTrainingDataDependencies | | appinfo | | Does the IAS record all the training data dependencies of all algorithms in use (to include any version number) that are utilized to produce an action/decision? |
|
| enumeration | RE08.1.21_IASRecordsExternalsystemCommunicationsExchanges | | appinfo | | Does the IAS record its connections to, data exchanges with, and interdependencies with, other external systems to include other IAS? |
|
| enumeration | RE08.1.22_IASRecordsLossDataSignal | | appinfo | | Does the IAS record the loss of data signal connections? |
|
| enumeration | RE08.1.23_IASRecordsIntersystemCommunicationsExchanges | | appinfo | | Does the IAS record connections, data exchanges and interdependencies between its own internal subsystems? |
|
| enumeration | RE08.1.24_IASRecordsBDA | | appinfo | | Does the IAS record data from battle damage assessments? |
|
| enumeration | RE08.1.25_IASRecordsTechnicalMalfunctions | | appinfo | | Does the IAS record data regarding technical malfunctions and degradations of performance? |
|
| enumeration | RE08.1.26_IASRecordsPosition | | appinfo | | Does the IAS record its position relative to all battlespace management areas, including both exclusion and inclusion areas (e.g., Notices to Airman and Mariners)? |
|
| enumeration | RE08.1.27_IASRecordsRiskAssessments | | appinfo | | Does the IAS record the results of any risk assessments made on board during a mission and the resultant mitigation actions taken? |
|
| enumeration | RE08.1.3_AdequateStorageTransmissionOperationalAuditData | | appinfo | | Is the IAS equipped with sufficient data storage and/or transmission capabilities to capture and retain data at the frequency required for post-operational uses and for the entire duration of the mission? |
|
| enumeration | RE08.1.4_OperationalDataFieldsDocumentedAvailable | | appinfo | | Are all data descriptions understandable to non-technical persons who may use the data logs for post-operational reconstruction and analysis? |
|
| enumeration | RE08.1.5_OperationalAuditDataLegallySufficient | | appinfo | | Has the data in the operational audit log been verified to be of sufficient quality, quantity, and provenance to support any potential post-operational legal proceeding? |
|
| enumeration | RE08.1.6_OperationalAuditDataAccessControls | | appinfo | | Are access controls in place to limit access to the IAS operational audit logs to authorized personal only? |
|
| enumeration | RE08.1.7_OperationalAuditDataStewardIssuesRecorded | | appinfo | | Are challenges to the use of an IAS based on examination of the IAS operational audit log, and the results of those challenges recorded by the audit log data steward? |
|
| enumeration | RE08.1.8_OperationalAuditLogAccessesDocumented | | appinfo | | Are all requests for explanations or inspections of the IAS operational audit log, and the results of those inspections recorded by the audit log data steward? |
|
| enumeration | RE08.1.9_OperationalAuditDataLogAccessProceedures | | appinfo | | Is there a procedure in place to request, and approve requests, to examine the operational audit log? |
|
| enumeration | RE08.2.1_DecisionsSensorDataC2AlgorithmsTrainingData | | appinfo | | Are all decisions made by the IAS traceable back to the sensor data, operator commands, algorithms and training data used? |
|
| enumeration | RE08.2.2_DecisionsMapObjective | | appinfo | | Can all actions/decisions made by an IAS and the human operator who authorized the action/decision be traced back to the military objective or mission orders they were intended to support? |
|
| enumeration | RE08.2.3_DecisionTraceabilityEnvironment | | appinfo | | Can human operators retrace decision steps and interactions with the environment that resulted in an IAS action/decision of interest? |
|
| enumeration | RE08.2.4_DecisionSubsystemInformationHumanReview | | appinfo | | Are the inputs from component autonomous subsystems that were used by the IAS to make decisions available for review by a human operator? |
|
| enumeration | RE08.3.1_AuditLogPurgeDate | | appinfo | | Does the IAS operational audit log have a metadata field identifying the date after which retention is no longer required? |
|
| enumeration | RE08.3.2_AuditLogPurgeMethod | | appinfo | | Is IAS operational audit log data deleted automatically or manually with human oversight when its retention date is reached? |
|
| enumeration | RE08.3.3_AuditLogRetentionStatueLegalActions | | appinfo | | Are IAS operational audit logs retained beyond the statute of limitations date for any illegal acts that could possibly have been committed during an operation? |
|
| enumeration | RE08.3.4_AuditLogExtraRetentionAcquisition | | appinfo | | Do acquisition professionals who may wish to use IAS operational audit logs to improve the performance of this, or some other IAS, have a role in deciding when the audit logs are no longer retained? |
|
| enumeration | RE08.3.5_AuditLogExtraRetentionOthers | | appinfo | | Do military commanders, trainers, and doctrine writers who may wish to use IAS operational audit logs to improve future tactics, techniques and procedures have a role in deciding when the audit logs are no longer retained? |
|
| enumeration | RE08.3.6_DataLogStorageTransmissionSufficient | | appinfo | | Are data storage and transmission resources sufficient to retain all of the IAS operational audit log data created during the contemplated life cycle of the IAS? |
|
| enumeration | RE08.3.7_OperationalAuditLogsAdditionalHandlingCaveats | | appinfo | | Do the IAS operational audit logs require any additional handling caveats above and beyond existing document retention and destruction policies? |
|
| enumeration | RE08.3.8_OperationalAuditLogsQualityAssessed | | appinfo | | Is retained IAS operational audit log data that is subsequently reused, reassessed first for its quality and continued relevancy and applicability? |
|
| enumeration | RE08.3.9_OperationalDataLogAccessRequestsRecorded | | appinfo | | Are all accesses to retained IAS operational audit log data, either granted or denied, recorded by the data steward? |
|
| enumeration | RE08.4.1_OperationalAuditLogEnableFutureAlgorithmDevemelopment | | appinfo | | Does the IAS operational audit log enable training data generation for subsequent IAS algorithm development? |
|
| enumeration | RE08.4.2_OperationalAuditLogEnableAccountabilityAdverseActions | | appinfo | | Does the IAS operational audit log enable accountability for adverse actions? |
|
| enumeration | RE08.4.3_OperationalAuditLogEnableRefinementTesting | | appinfo | | Does the IAS operational audit log enable further IAS refinement and testing? |
|
| enumeration | RE08.4.4_OperationalAuditLogEnableTacticsProcedures | | appinfo | | Does the IAS operational audit log enable tactics techniques and procedures development? |
|
| enumeration | RE08.4.5_OperationalAuditLogEnableOperatorTraining | | appinfo | | Does the IAS operational audit log enable human operator training? |
|
| enumeration | RE08.4.6_OperationalAuditLogEnablePostMissionReview | | appinfo | | Does the IAS operational audit log enable post mission hot washes for corrective action? |
|
| enumeration | RE08.4.7_OperationalAuditLogEnableAcquisitionDifferentIAS | | appinfo | | Does the IAS operational audit log enable acquisition of IAS not related to the current IAS or its mission(s)? |
|
| enumeration | RE08.4.8_OperationalAuditLogAvailableInternalOversight | | appinfo | | Are the IAS operational audit logs made available for internal oversight? |
|
| enumeration | RE08.4.9_OperationalAuditLogAvailableExternalOversight | | appinfo | | Are the IAS operational audit logs made available for external third-party oversight? |
|
| enumeration | RE09.1.1_SelfIdentificationIAS | | appinfo | | Can the IAS alert the humans that they are interacting with an IAS and not another human? |
|
| enumeration | RE09.1.10_IASFalseNegativeResults | | appinfo | | Is the rate of IAS false negative results known and below a previously defined and acceptable value? |
|
| enumeration | RE09.1.11_IASOffOption | | appinfo | | Is there an option to turn off the IAS if there are too many false alarms? Provision of shared situational awareness |
|
| enumeration | RE09.1.12_CommunicationDuringFogOfWar | | appinfo | | Can the IAS communicate with all systems from which it must ingest data to achieve maximum situational awareness during the fog and friction of war? |
|
| enumeration | RE09.1.13_SelfMonitoringSystem | | appinfo | | Is the IAS equipped with system self‐monitoring sensors that can detect and communicate (internal) conditions that may lead to a negative outcome if not corrected? |
|
| enumeration | RE09.1.14_OperationalDesignVioloationPredictions | | appinfo | | Can the IAS predict when operational design domain conditions and constraints might be violated and communicate the future time of the violations, the probability of occurrence and the severity to a human operator? |
|
| enumeration | RE09.1.15_IASHumanTeamCommunication | | appinfo | | Can the IAS and the human teammates effectively communicate their state, intent, and current problems to the other teammate? |
|
| enumeration | RE09.1.16_IASFeulAmmoLevelPredictions | | appinfo | | Can the IAS communicate the current and predicted future status of fuel and ammunition levels to the human operator? |
|
| enumeration | RE09.1.17_IASMissionExecutionAvalilabillityPrediction | | appinfo | | Can the IAS communicate its current and predicted future availability for mission execution to the human operator? |
|
| enumeration | RE09.1.18_IASMalfunctionCommunication | | appinfo | | Can the IAS communicate (previously defined) system malfunctions to the human operator? |
|
| enumeration | RE09.1.19_IASCriticalMessageCommunication | | appinfo | | Can the IAS communicate critical messages to the human operator? |
|
| enumeration | RE09.1.2_DistinguishBlueForceHumansIAS | | appinfo | | Can the IAS distinguish between blue forces with human operators and other blue force IAS? |
|
| enumeration | RE09.1.20_IASEnviromentChangeAlerts | | appinfo | | Can the IAS detect both sharp and gradual changes in its operational environment and provide alerts to the human operator and other systems based on criticality? |
|
| enumeration | RE09.1.21_IASPerformanceChangeAlerts | | appinfo | | Can the IAS detect both sharp and gradual changes in its own performance and provide alerts to the human operator and other systems based on criticality? |
|
| enumeration | RE09.1.22_IASFuncitionalityFailureConcerns | | appinfo | | If an IAS’s autonomous functionality fails and it must transfer command and control of itself to a human operator, will that operator have been sufficiently engaged and be able to understand the IAS’s current situation in time to safely repurpose the IAS or avert danger/loss of the IAS? |
|
| enumeration | RE09.1.23_IASHumanAssistanceQueries | | appinfo | | Can the IAS determine when it is inappropriate to interrupt a human operator with a query for assistance? |
|
| enumeration | RE09.1.24_RuleVsValueDecisionsRecognition | | appinfo | | Can the IAS recognize the difference between rules-based decisions and value/judgement-based decisions and query the human operator for guidance in the latter case? |
|
| enumeration | RE09.1.25_DeductiveVsInductiveVsAbductiveDecisions | | appinfo | | Can the IAS recognize the difference between decisions based on deductive versus inductive vs abductive reasoning and query the human operator for guidance in the latter case? |
|
| enumeration | RE09.1.26_DoctrineWithHumanGuidanceImplementation | | appinfo | | Can the IAS upload and implement any doctrine or rules of engagement that identify predetermined situations where human guidance is a prerequisite to taking further action? |
|
| enumeration | RE09.1.27_IASIntructionAmbiguityIdenification | | appinfo | | Can the IAS identify ambiguous or incomplete instructions and request clarification from the human operator in sufficient time to be operationally effective or to prevent mistakes/mishaps? |
|
| enumeration | RE09.1.28_AnomalyDetectionAndHumanGuidance | | appinfo | | Can the IAS detect anomalies and seek human guidance when its programming says that it is under attack, but the political situation is completely benign? |
|
| enumeration | RE09.1.3_IASRelianceDiminishHumanSkills | | appinfo | | Will continued reliance on the IAS cause human operator skills to diminish such that they can no longer back up the IAS in the event that the IAS becomes unavailable? |
|
| enumeration | RE09.1.4_CanIASMakeDecisionsWhenHumanCannot | | appinfo | | Can the IAS make decisions when incapacitated human operators cannot? |
|
| enumeration | RE09.1.5_DecisionsIASCanMake | | appinfo | | What types of decisions is the IAS allowed to make or delegate? |
|
| enumeration | RE09.1.6_HumanStressDetection | | appinfo | | Can the IAS detect when the human operator is under too much stress to make a sound decision and unilaterally make decisions without human oversight that normally would require it? |
|
| enumeration | RE09.1.7_HumanUnderstandableIASExplanations | | appinfo | | Can the IAS provide explanations of its actions and/or predictions that are understandable by the human operator? |
|
| enumeration | RE09.1.8_HumanComplacencyDetection | | appinfo | | Can the IAS detect human user complacency? |
|
| enumeration | RE09.1.9_IASFalsePositiveResults | | appinfo | | Is the rate of IAS false positive results known and below a previously defined and acceptable value? |
|
| enumeration | RE09.2.1_IASHumanConformityResponse | | appinfo | | Can the IAS detect and respond appropriately to conforming and non-conforming human behaviors? |
|
| enumeration | RE09.2.2_HumanInstructionLawViolationDetection | | appinfo | | Can the IAS identify an instruction from a human operator that violates its preprogrammed definitions of the Law of Armed Conflict, policy, or rules of engagement, not carry out the instruction, and provide a reason to the human operator for not doing so? |
|
| enumeration | RE09.2.3_HumanCommanderIASDecisionOverriding | | appinfo | | Can an IAS decision to not carry out an instruction that violates its preprogrammed definition of the Law of Armed Conflict, policy, or rules of engagement be overridden by the human operator or some other commander? |
|
| enumeration | RE09.2.4_MissionCorrectionAbility | | appinfo | | Can the IAS request mission re–planning if planned mission execution violates mission success criteria, or other operational constraints (such as the crossing of a geographic border), and report detected violations to the human operator? |
|
| enumeration | RE09.2.5_IASArbitrationBetweenCompetingObjectives | | appinfo | | Can the IAS perform arbitration between competing mission and navigation objectives based on constraints configured by the human operator? |
|
| enumeration | RE09.2.6_NarrowDomainSituationDetection | | appinfo | | Can the IAS detect situations that fall beyond its narrow application domain that subsequently require it to contact a human operator for additional guidance? |
|
| enumeration | RE09.3.1_IASHumanQueryFrequency | | appinfo | | Can the IAS determine the optimal frequency of how often they must query humans for commands and ensure that they do not miss any commands? |
|
| enumeration | RE09.3.2_IASHumanLikeCommunications | | appinfo | | Can the IAS communicate with the human using natural language, gestures or haptics to ease and to accelerate information exchanges with human operators? |
|
| enumeration | RE09.3.3_IASHumanCognitiveLoadMonitoring | | appinfo | | Can the IAS monitor the attentional focus, cognitive load, and task status of the human operator, and only communicate information that does not overburden the human operator with unnecessary or irrelevant information or tasks? |
|
| enumeration | RE09.3.4_IASHumanEmotionPhysicalStateDetection | | appinfo | | Can the IAS detect the emotional and physical state of the human operator to maximize efficiency of communications and activities? |
|
| enumeration | RE09.3.5_HumanMachineJointTraining | | appinfo | | Can the IAS support human–machine joint training to allow an understanding to develop between the human operator and the IAS regarding team objectives, platform roles, co–dependency relationships, and mutual expectations for competence, dependability, predictability, and timeliness? |
|
| enumeration | RE09.3.6_HumanMachineActionDelegations | | appinfo | | Is the delegation of actions explicitly represented in the dialogue between the human and the machine? |
|
| enumeration | RE09.3.7_HumanDelegationModificationCriteria | | appinfo | | Can the human modify the conditions placed on the delegated actions and/or decisions of the machine? |
|
| enumeration | RE09.3.8_MachineActionMonitoringAndDocumentation | | appinfo | | Is there monitoring and documentation that the machine is acting within its prescribed boundaries? |
|
| enumeration | RE10.1.1_TrainedReplicatedOperationalEnvironments | | appinfo | | Did the IAS test and evaluation procedure receive an already-trained IAS, or is the IAS to be tested also to be trained in the replicated operational environment? |
|
| enumeration | RE10.1.10_TestEvaluationOngoingNewUnexpectedCircumstances | | appinfo | | Is there ongoing test and evaluation after fielding to evaluate the ability of the IAS to adapt to new, unexpected circumstances, or new input data? |
|
| enumeration | RE10.1.11_TestEvaluationMeasurableTestableAutonomousCapabilities | | appinfo | | Does the IAS possess measurable and testable autonomous capabilities? |
|
| enumeration | RE10.1.12_C2SafetyEngineersConsulted | | appinfo | | Have members of the AI and safety engineering research communities with expertise in the potential safety or failure risks from loss of command and control been consulted? |
|
| enumeration | RE10.1.13_IndependentThirdPartyTestingVerification | | appinfo | | Was the operational testing and verification witnessed by an independent third party (non-DOD) such as the Underwriter’s Laboratory or operational test and evaluation personnel from another federal agency? |
|
| enumeration | RE10.1.2_TestEvaluationAutonomous | | appinfo | | Did the IAS test and evaluation procedure test all components for the presence of autonomous capabilities? |
|
| enumeration | RE10.1.3_TestEvaluationResultsInspectionOversight | | appinfo | | Did the IAS test and evaluation procedure record all test results to make them available for inspection and oversight? |
|
| enumeration | RE10.1.4_TestEvaluationResultsUnsuitableModelTesting | | appinfo | | Did the IAS test and evaluation procedure use traditional model checking that may be unsuited for the IAS? |
|
| enumeration | RE10.1.5_TestEvaluationAuthorizedCertifiedConditions | | appinfo | | Did operational testing provide authorized and certified conditions for use of IAS? |
|
| enumeration | RE10.1.6_TestEvaluationSimulationEdgeCaseBoundaries | | appinfo | | Did the IAS test and evaluation procedure use simulations to push the system to its breaking point so that the boundaries between success and failure were tested and verified in a representative operational environment? |
|
| enumeration | RE10.1.7_TestEvaluationResultsLVCEnvironment | | appinfo | | Did the IAS test and evaluation procedure utilize an environment (e.g., Live Virtual Constructive environment) that best captures the behaviors that may emerge under real-world operational conditions? |
|
| enumeration | RE10.1.8_TestEvaluationSubcomponentResults | | appinfo | | Were components or sub-components of the IAS available for testing and fleet concept of employment and experimentation? |
|
| enumeration | RE10.1.9_TestEvaluationAutonomousResults | | appinfo | | Has the threshold been identified for adequate test and evaluation of autonomous capabilities with respect to their manned counterparts, and with respect to the risk of not having any autonomous systems? |
|
| enumeration | RE10.2.1_TestEvaluationComplexSimulation | | appinfo | | Did the IAS test and evaluation procedure use simulations adequately representative of the contemplated operational environment and document all operational parameters that would cause it to be used in an environment that the simulation did not faithfully replicate? |
|
| enumeration | RE10.2.10_TestEvaluationChangesNewTrainingData | | appinfo | | Did the IAS test and evaluation procedure consider how an IAS might overwrite all or part of its original software as a result of exposure to new training data, such that the procedure must be repeated on the new software baseline? |
|
| enumeration | RE10.2.11_TestEvaluationC2Hijacking | | appinfo | | Did the IAS test and evaluation procedure consider whether the data stream providing instructions to the IAS is vulnerable to hacking or "in flight" hijacking? |
|
| enumeration | RE10.2.12_TestEvaluationInfiltrationIndustrialSupplyChain | | appinfo | | Did the IAS test and evaluation procedure consider infiltration into the industrial supply chain? |
|
| enumeration | RE10.2.13_TestEvaluationDigitalTwinsExternalSystems | | appinfo | | Did the IAS test and evaluation procedure utilize virtual twin models to test the IAS in simulated environments to predict how it will interact with IAS and non-autonomous systems from neutral nations, allied nations and from other services? |
|
| enumeration | RE10.2.14_DevelopmentConductPlanning | | appinfo | | Did the IAS development include an AI-developed capability to conduct planning where there will be a need to design test plans in a timely manner to assure thorough testing while being able to predict certain behaviors with these systems (e.g., Autonomous Systems Test (AST) Planning)? |
|
| enumeration | RE10.2.15_DevelopmentGuarenteeSafety | | appinfo | | Did the IAS development include an AI-developed capability that will guarantee safety during testing through the use of immersion technologies to adapt tests to best suit IAS cognitive capabilities (e.g., conducting AST Execution & Control)? |
|
| enumeration | RE10.2.16_DevelopmentConductAssessments | | appinfo | | Did the IAS development include an AI-developed capability to conduct assessments (e.g., Autonomous System Performance Assessment) to capture how IAS interact with existing and newly fielded manned and unmanned systems in realistic operational environments? |
|
| enumeration | RE10.2.17_FunctionalComponentTestingReplicatedEnvironment | | appinfo | | Can functional components be tested in the replicated environment while assessing reliability and identification of vulnerabilities? |
|
| enumeration | RE10.2.18_HumanInteractionMeasuredreplicatedEnvironment | | appinfo | | Can human-IAS interaction be measured in the replicated environment while assessing reliability and identification of vulnerabilities? |
|
| enumeration | RE10.2.2_TestEvaluationSimulatedDataCoverage | | appinfo | | Did the IAS test and evaluation procedure use simulated or otherwise artificially generated data to replicate the contemplated operational environment? |
|
| enumeration | RE10.2.3_TrainingDataEnvironmentValidation | | appinfo | | Has the relevance of the IAS training data to the contemplated operational environment been validated? |
|
| enumeration | RE10.2.4_SimulatedDataValidatedRealWorldData | | appinfo | | Has any use of simulated data been validated against real-world data? Breadth of testing in the replicated environment |
|
| enumeration | RE10.2.5_TestEvaluationAdversarialMachineLearningAttacks | | appinfo | | Did the IAS test and evaluation procedure subject the system to adversarial machine learning attacks to identify potential attack vectors? |
|
| enumeration | RE10.2.6_TestEvaluationLimitedReplicatedEnvironment | | appinfo | | Did the IAS test and evaluation procedure limit the replicated environment to what the IAS will encounter in the contemplated operational environment and document all parameters that would cause it to be used in an environment test and evaluation did not faithfully replicate? |
|
| enumeration | RE10.2.7_TestEvaluationHumanInteractionRepresentative | | appinfo | | Did the IAS test and evaluation procedure consider interactions with human operators using a fully representative sample of all potential human operators? |
|
| enumeration | RE10.2.8_TestEvaluationEmergentBehaviorExploration | | appinfo | | Did the IAS test and evaluation procedure use computer models to generate thousands of data-based scenarios to anticipate and analyze any emergent behavior? |
|
| enumeration | RE10.2.9_TestEvaluationChangesNewSensorData | | appinfo | | Did the IAS test and evaluation procedure consider how an IAS might overwrite all or part of its original software as a result of exposure to new sensor data, such that the procedure must be repeated on the new software baseline? |
|
| enumeration | RE10.3.1_TestEvaluationBreakPointTesting | | appinfo | | Did the IAS test and evaluation procedure push the system to its breaking point so that this/these points are known to the operators? |
|
| enumeration | RE10.3.2_TestEvaluationRedTeamTesting | | appinfo | | Did the IAS test and evaluation procedure utilize red team attacks (kinetic and cyber) to capture the full range of behaviors that might emerge under real-world operational conditions? |
|
| enumeration | RE10.3.3_TestEvaluationInappropriateBehaviorProbingTesting | | appinfo | | Did the IAS test and evaluation procedure utilize red team attacks specifically designed to drive the system into inappropriate behaviors? |
|
| enumeration | RE11.1.1_BaselineLegalEthicalTraining | | appinfo | | Have all senior policy and decision makers, on scene commanders, scientists, engineers, acquisition officials, legal counsels, operators and supported troops received training in the Law of Armed Conflict, IAS ethics principles, and current rules of engagement? |
|
| enumeration | RE11.1.2_RecognitionLegallyEthicallQuestionableDecisionsActions | | appinfo | | Can human operators with the power to override IAS use during an operation recognize legally and ethically questionable decisions and actions? |
|
| enumeration | RE11.1.3_UnderstandingEthicalDilemmaScenarios | | appinfo | | Do all commanders and users that possess current proficiency certifications to operate the IAS, fully understand how the IAS processes ethical dilemma scenarios? |
|
| enumeration | RE11.2.1_CertificationProcessEducationTrainingRequirements | | appinfo | | Do training and education requirements exist for IAS as part of the certification process for the deploying forces that may employ them? |
|
| enumeration | RE11.2.2_CertificationsTrainingEducationRecorded | | appinfo | | Are IAS training, education, and qualification certifications required and recorded for all personnel? |
|
| enumeration | RE11.2.3_LegalMonitorLawsPoliciesRegulations | | appinfo | | Does the commander’s legal staff monitor the laws, ethics policies and regulations that govern the use of the IAS to ensure current training, education, and qualification certifications are compliant with them? |
|
| enumeration | RE11.2.4_LegalTechnicalUnderstanding | | appinfo | | Does the commander’s legal staff have a sufficient technical understanding of IAS? |
|
| enumeration | RE11.2.5_LegalReviewsTechnicalProcurementSoundness | | appinfo | | Are all legal reviews double checked by technical and procurement officials to ensure that they are based on a sound technical understanding of the IAS? |
|
| enumeration | RE11.3.1_UnderstandingGeneralNarrowModels | | appinfo | | Do all commanders and users understand the difference between general and narrow artificial intelligence? |
|
| enumeration | RE11.3.10_DegreeAutonomyTTPs | | appinfo | | Can all commanders and users determine the degree of autonomy needed in different operational environments and understand the risk mitigation tactics, techniques, procedures and technologies available to them? |
|
| enumeration | RE11.3.11_UnderstandingIASLimitationsDataContext | | appinfo | | Can the commanders discern the consequences of the IAS limitations? Understanding the role of data |
|
| enumeration | RE11.3.12_UnderstandingTrainingDataAttackVectors | | appinfo | | Do all commanders and users understand the dependence of IAS on training data and the new attack vectors this dependency opens up to the enemy? |
|
| enumeration | RE11.3.13_UnderstandingTrainTestValidateDatasets | | appinfo | | Do all commanders and users understand the difference between training data, input data and feedback data? |
|
| enumeration | RE11.3.2_UnderstandingModelBrittleness | | appinfo | | Do all commanders and users understand the concept of "brittleness" with respect to narrow artificial intelligence applications? |
|
| enumeration | RE11.3.3_UnderstandingIASOperationallyNecessary | | appinfo | | Do all commanders and users know how to evaluate when IAS are, and are not, necessary to achieving the military objective? |
|
| enumeration | RE11.3.4_UnderstandingIASTechnicalLimitations | | appinfo | | Do all commanders and users know the technical limitations of IAS? |
|
| enumeration | RE11.3.5_UnderstandingEnvironmentOperationalLimitationsIAS | | appinfo | | Do all commanders and users know the limitations of IAS imposed by the operational environment? |
|
| enumeration | RE11.3.6_UnderstandingDataEnvironmentLimitationsIAS | | appinfo | | Do all commanders and users know the limitations of IAS imposed by the data environment? |
|
| enumeration | RE11.3.7_UnderstandingComponentsAutonomousCapabilities | | appinfo | | Do all commanders and users know how to distinguish between components that do and do not possess autonomous capabilities? |
|
| enumeration | RE11.3.8_UnderstandingDegreeAutonomyIASCapabilitesLimitations | | appinfo | | Do all commanders and users understand the degrees of autonomy and their impact on IAS capabilities and limitations? |
|
| enumeration | RE11.3.9_UnderstandingConductRiskAssessmentIAS | | appinfo | | Do all commanders and users know how to conduct a risk assessment for the use of IAS? |
|
| enumeration | RE11.4.1_UnderstandingReasoningMethodsLimitations | | appinfo | | Do all commanders and users understand the difference between inductive, deductive, and abductive reasoning, and the limitations of algorithms in each? |
|
| enumeration | RE11.4.2_UnderstandingMLUsedIAS | | appinfo | | Do all commanders and users understand the different types of machine learning (supervised, unsupervised, reinforcement) used by the IAS and the operational environments amenable to each? |
|
| enumeration | RE11.5.1_RecognizeDataPoisoningAttack | | appinfo | | Can all commanders and users recognize a data poisoning attack? |
|
| enumeration | RE11.5.2_RecognizeRewardGamingMalfunction | | appinfo | | Can all commanders and users recognize an instance of "reward function gaming?" |
|
| enumeration | RE11.5.3_RecognizeAdversarialSpoofing | | appinfo | | Can all commanders and users recognize an instance of "adversarial spoofing?" |
|
| enumeration | RE11.5.4_RecognizeCatastrophicForgetting | | appinfo | | Can all commanders and users recognize an instance of "catastrophic forgetting?" |
|
| enumeration | RE11.5.5_RecognizeConceptDrift | | appinfo | | Can all commanders and users recognize an instance of "concept drift?" |
|
| enumeration | RE11.5.6_RecognizeModelInversion | | appinfo | | Can all commanders and users recognize an instance of "model inversion?" |
|
| enumeration | RE11.5.7_RecognizeDeepFake | | appinfo | | Can all commanders and users recognize a "deep fake?" |
|
| enumeration | RE11.5.8_RecognizeUnauthorizedReprogramming | | appinfo | | Can all commanders and users recognize an instance of reprogramming by unauthorized users? |
|
| enumeration | RE11.5.9_RecognizeTightlyCoupledSystem | | appinfo | | Can all commanders and users recognize a system that is "tightly coupled?" |
|
| enumeration | RE12.1.1_DurationLimitedUniformlyConsistent | | appinfo | | Can the commander reduce the mission duration such that the IAS risk mitigation conditions and planning factors will be, or can reasonably be expected to be, uniformly consistent? |
|
| enumeration | RE12.1.2_TargetTimeouts | | appinfo | | Does the target selection that the human operator provides the IAS have an expiration time within the duration of the mission, such that the human operator will be alerted to select and verify new targets after the expiration time has expired? |
|
| enumeration | RE12.1.3_AlterableDegreeAutonomy | | appinfo | | Is the commander able to alter the degree of autonomy of the IAS during long missions based on whether it is in a loitering mode, or an active task execution mode? |
|
| enumeration | RE12.1.4_CommsReliabilityDurationGeographicMissionExtent | | appinfo | | Is the reliability of communications links commensurate with both the duration and the geographic extent of the mission? |
|
| enumeration | RE12.1.5_ComparisonChangingMissionParameters | | appinfo | | Can the IAS compare risk mitigation and planning factors provided to it at the beginning of a long mission, and compare these to sensed conditions just prior to an action/decision and abort the action if they are different? |
|
| enumeration | RE12.2.1_ExpanseLimitedUniformlyConsistent | | appinfo | | Can the commander reduce the expanse of the operational environment to an area where risk mitigation conditions and planning factors are known to be, or are reasonably expected to be, uniformly consistent? |
|
| enumeration | RE12.2.2_MissionOperationalAreaDividedUniformlyConsistent | | appinfo | | Can the commander subdivide a large operational area into smaller ones where risk mitigation conditions and planning factors are known to be, or are reasonably expected to be, uniformly consistent, and assign one IAS to each area, with each IAS specifically programmed to address the conditions unique to each subsection? |
|
| enumeration | RE12.2.3_LocationBasedDegreeAutonomy | | appinfo | | Is the IAS equipped with a capability that tailors its approved degree of autonomy and authorized actions based on geographical location? |
|
| enumeration | RE12.2.4_IASRecognizedDisplacedWillCease | | appinfo | | If a target is verified within a geographic location but transits outside it later in time, can the IAS still engage, or does it immediately cease action? |
|
| enumeration | RE12.2.5_DistanceBasedDegreeAutonomy | | appinfo | | Is the IAS equipped with a capability that tailors its approved degree of autonomy and authorized actions based on its distance from a human operator? |
|
| enumeration | RE12.2.6_IASReportAORTooLarge | | appinfo | | Can the IAS recognize when it is assigned an area of regard too large for reliable autonomous operation? |
|
| enumeration | RE12.2.7_IASReportAsymetricRiskAreas | | appinfo | | Can the IAS recognize when it has been assigned to an area of regard where the fundamental risk mitigation conditions are not uniformly consistent? |
|
| enumeration | RE12.3.1_AutonomousFunctionTimeout | | appinfo | | Is the IAS equipped with a switch that will disable its autonomous functionalities after a predetermined set time has elapsed? |
|
| enumeration | RE12.3.2_AutonomousDisengageOutsideGeofence | | appinfo | | Is the IAS equipped with a geo-fence switch that will disable its autonomous functionalities if it strays from an authorized operational area? |
|
| enumeration | RE12.3.3_AutonomyModeChangedOverTime | | appinfo | | If risk is reduced with increased mission time or expanded geographic range, can the human operator increase autonomous capabilities of the IAS as needed? |
|
| enumeration | RE12.3.4_CommanderNotifiedHigherAuthorityOverride | | appinfo | | Is the commander notified if they have an IAS with a disablement switch that has been overridden by higher authority? |
|
| enumeration | RE12.3.5_IASDisablementSystemGranularity | | appinfo | | Is the IAS equipped with a disablement switch that can selectively disable only certain functions or subsystems? |
|
| enumeration | RE12.3.6_AutonomyConditionalReengagement | | appinfo | | Is the IAS equipped with a "soft" disablement switch that disables autonomous functionality but can be reversed if certain predetermined conditions are detected or an override authorization is received? |
|
| enumeration | RE12.3.7_IASAttemptRTB | | appinfo | | If the IAS is disabled, can it safely and securely return to a predetermined location? |
|
| enumeration | RE13.1.1_IASAffectBillOfRights | | appinfo | | Does the IAS potentially affect the rights enumerated in the first ten amendments to the US Constitution (the Bill of Rights)? |
|
| enumeration | RE13.1.2_IASAffectCivilEqualConstitutionRights | | appinfo | | Does the IAS potentially affect the rights enumerated in the Civil Rights Act of 1964, the Equal Protection Clause of the U.S. Constitution and all subsequent enactments? |
|
| enumeration | RE13.1.3_IASAffectEuropeanUSPersonalData | | appinfo | | Does the IAS potentially affect the rights enumerated in the European Union General Data Protection Regulation or the US Privacy Statutes regarding collection, transmission and safeguarding of personal data? |
|
| enumeration | RE13.1.4_IASAffectDueProcess | | appinfo | | Does the IAS potentially affect due process? |
|
| enumeration | RE13.1.5_IASAffectPIIAgreements | | appinfo | | Does the IAS potentially affect agreements to not retain personally identifiable information after termination of IAS use? |
|
| enumeration | RE13.1.6_IASAffectDictatesPublicConscience | | appinfo | | Does the IAS potentially affect the "dictates of public conscience?" |
|
| enumeration | RE13.1.7_IASAffectMassSurveillancePrivacyExpectation | | appinfo | | Does the IAS potentially affect the establishment of mass surveillance capabilities in an area where there is an expectation of privacy? |
|
| enumeration | RE13.2.1_IASUseRequiresNotification | | appinfo | | Are potentially affected persons notified that the IAS is in use? |
|
| enumeration | RE13.2.10_IASPotentiallyAffectedPersonsAwareSensitivePersonalInformation | | appinfo | | Are potentially affected persons made aware when sensitive personal information is collected and retained by the IAS? |
|
| enumeration | RE13.2.11_IASPotentiallyAffectedPersonsNotifiedPersonalDataDestroyed | | appinfo | | Are potentially affected persons notified when their personal data is destroyed after IAS use? |
|
| enumeration | RE13.2.12_IASPotentiallyAffectedPersonsAwareRetentionPoliciesPersonalData | | appinfo | | Are potentially affected persons made aware of retention policies and schedules for personal data? |
|
| enumeration | RE13.2.13_IASPotentiallyAffectedPersonsCanControlThirdPartyDisclosure | | appinfo | | Are potentially affected persons allowed to deny the disclosure of their personal data to third parties? |
|
| enumeration | RE13.2.14_IASPotentiallyAffectedPersonsPersonalDataProtectionPoliciesMeasures | | appinfo | | Are potentially affected persons made aware of personal data protection policies and measures? |
|
| enumeration | RE13.2.15_IASPotentiallyAffectedPersonsNotifiedTrainingPersonalData | | appinfo | | Are potentially affected persons notified of the purpose of the IAS that is trained using their personal data? |
|
| enumeration | RE13.2.16_IASPotentiallyAffectedPersonsInformedInteractingIAS | | appinfo | | Are potentially affected persons informed when they are interacting with an IAS and not a human being? |
|
| enumeration | RE13.2.17_IASPotentiallyAffectedPersonsMadeAwareDemographicProfessionalIntellectualDiversity | | appinfo | | Are potentially affected persons made aware of the demographic, professional and intellectual diversity of the staff that developed the IAS? |
|
| enumeration | RE13.2.18_IASPotentiallyAffectedPersonsChallengeNegativeDeterminations | | appinfo | | Are potentially affected persons made aware of negative determinations made by an IAS and given an opportunity to challenge those determinations to a human arbitrator? |
|
| enumeration | RE13.2.19_IASPotentiallyAffectedPersonsInformedPolicyLawRegsRules | | appinfo | | Are potentially affected persons informed as to what policies, laws, regulations, and/or rules governed the use of the IAS? |
|
| enumeration | RE13.2.2_IASPotentiallyAffectedPersonsAllowedMonitorUse | | appinfo | | Are potentially affected persons allowed to monitor the use of the IAS? |
|
| enumeration | RE13.2.20_IASPotentiallyAffectedPersonsTestValidationProcedures | | appinfo | | Are potentially affected persons informed as to the test and evaluation procedures used to validate the IAS? |
|
| enumeration | RE13.2.21_IASPotentiallyAffectedPersonsInformedRepurpose | | appinfo | | Are potentially affected persons informed when the IAS is repurposed for use from some other, unrelated original purpose? |
|
| enumeration | RE13.2.22_IASPotentiallyAffectedPersonsNotificationPerpetuationBias | | appinfo | | Are potentially affected persons made aware when the perpetuation of a bias by an IAS has been discovered? |
|
| enumeration | RE13.2.23_IASPotentiallyAffectedPersonsAdvisedUnintendedAdverseEffects | | appinfo | | Are potentially affected persons advised that IAS use can have adverse effects, even though they are unintended? |
|
| enumeration | RE13.2.3_IASPotentiallyAffectedPersonsAllowedOptIn | | appinfo | | Are potentially affected persons allowed to opt in or opt out of the use of the IAS? |
|
| enumeration | RE13.2.4_IASPotentiallyAffectedPersonsAllowedMonitorUse | | appinfo | | Are potentially affected persons made aware of the opportunities to, and the procedures for, requesting access to IAS source code for inspection purposes? |
|
| enumeration | RE13.2.5_IASPotentiallyAffectedPersonsAwareSOPConops | | appinfo | | Are potentially affected persons aware of the opportunities to, and the procedures for, requesting access to IAS concepts of operations and standard operating procedures for inspection purposes? |
|
| enumeration | RE13.2.6_IASPotentiallyAffectedPersonsAwareAccessTrainingDataInspection | | appinfo | | Are potentially affected persons aware of the opportunities to, and the procedures for, requesting access to IAS training data for inspection purposes? |
|
| enumeration | RE13.2.7_IASPotentiallyAffectedPersonsAdvisedAdverseImpacts | | appinfo | | Are potentially affected persons advised that IAS use can have adverse impacts based on age, race, color, ethnicity, sex, religion, national origin, gender, gender identity, sexual orientation, familial status, biometric information, or disability status or economic class? |
|
| enumeration | RE13.2.8_IASPotentiallyAffectedPersonsAdviseAdverseLegallyProtectedGroup | | appinfo | | Are potentially affected persons advised that IAS use can have adverse impacts based on membership in a group that enjoys legally protected status? |
|
| enumeration | RE13.2.9_IASPotentiallyAffectedPersonsAdviseAdverseImpactOpportunity | | appinfo | | Are potentially affected persons advised that IAS use can adversely impact their access to housing, education, employment, insurance, credit, or access to places of public accommodation? |
|
| enumeration | RE13.3.1.1_DemographicsMirroredDevelopmentIAS | | appinfo | | Does the demographic makeup of the IAS development team reflect that of the population(s) potentially affected by use of the IAS? |
|
| enumeration | RE13.3.1.10_DevelopmentTeamsIdentifyBiasSyntheticTrainingData | | appinfo | | Do development teams look for synthetic training data generated from models that are potentially subject to bias? |
|
| enumeration | RE13.3.1.11_DevelopmentTeamsCheckSynthicVSRealDataBias | | appinfo | | Do development teams validate potentially biased synthetic data against real data? |
|
| enumeration | RE13.3.1.12_DevelopmentTeamsIdentifyBiasTrainingDataCollection | | appinfo | | Do development teams look for bias seeping into the training data during the data collection process? |
|
| enumeration | RE13.3.1.13_DevelopmentTeamsIdentifySimpsonsParadoxDataFusion | | appinfo | | Do development teams look for "Simpson’s paradox," where fusing data introduces new bias to the IAS? |
|
| enumeration | RE13.3.1.14_DevelopmentTeamsIdentifyTechnologicalSolutionism | | appinfo | | Do development teams look for "technological solutionism," the perception that technology will lead to only positive solutions? |
|
| enumeration | RE13.3.1.15_DevelopmentTeamsIdentifyRepresentationBias | | appinfo | | Do development teams look for representation bias where the training data is not a representative reflection of the affected population? |
|
| enumeration | RE13.3.1.16_DevelopmentTeamsIdentifyOmittedVariableBias | | appinfo | | Do development teams look for omitted variable bias where the operational environment contains a variable not present in the models used to develop the IAS? |
|
| enumeration | RE13.3.1.17_DevelopmentTeamsIdentifySocialBehaviorBias | | appinfo | | Do development teams look for social or behavior bias where the affected person’s reactions to the IAS influences IAS actions/decisions? |
|
| enumeration | RE13.3.1.18_DevelopmentTeamsIdentifyRankingBias | | appinfo | | Do development teams look for ranking bias where the IAS actions or decisions are based on a small number of heavily weighted results and not on all available information? |
|
| enumeration | RE13.3.1.19_DevelopmentTeamsIdentifyAggregationBiasMultiModel | | appinfo | | Do development teams look for aggregation bias where the IAS requires multiple models to fairly represent all sub-groups within the parent group population? |
|
| enumeration | RE13.3.1.2_TeamAccessTrainingBalanceDemographics | | appinfo | | When it is not possible or practicable to assemble a development team with a demographic makeup reflective of the potentially affected population, do team members have access to training materials that educate them on the cultures and characteristics of those populations? |
|
| enumeration | RE13.3.1.20_DevelopmentTeamsIdentifyEvaluationBias | | appinfo | | Do development teams look for evaluation bias where IAS models may be erroneously validated? |
|
| enumeration | RE13.3.1.21_DevelopmentTeamsIdentifyTemoralBias | | appinfo | | Do development teams look for temporal bias where the operational context changes, perhaps imperceptibly, so that initial assumptions about the environment, and the training data chosen to represent it, are no longer valid? |
|
| enumeration | RE13.3.1.22_DevelopmentTeamsIdentifyAutomationBias | | appinfo | | Do development teams look for automation bias where human operators tend to reflexively accept decisions made by an automated system? |
|
| enumeration | RE13.3.1.23_DevelopmentTeamsIdentifyAssimilationBias | | appinfo | | Do development teams look for assimilation bias where human operators tend to modify information to fit into pre-existing analytical frameworks? |
|
| enumeration | RE13.3.1.24_DevelopmentTeamsIdentifyConfirmationBias | | appinfo | | Do development teams look for confirmation bias where developers gather training data that reaffirm conclusions they’ve already made? |
|
| enumeration | RE13.3.1.25_DevelopmentTeamsIdentifySimplificationBias | | appinfo | | Do development teams look for simplification bias where human operators tend to simplify phenomena they encounter? |
|
| enumeration | RE13.3.1.26_DevelopmentTeamsIdentifyActivityBias | | appinfo | | Do development teams look for activity bias where training data comes from a system’s most active users, rather than less active or inactive users? |
|
| enumeration | RE13.3.1.27_DevelopmentTeamsIdentifyAnnotatorBias | | appinfo | | Do development teams look for annotator bias where developers rely on automation as a heuristic replacement for their own information seeking and processing? |
|
| enumeration | RE13.3.1.28_DevelopmentTeamsIdentifyContentProductionBias | | appinfo | | Do development teams look for content production bias where nonexistent differences in the data result from structural, lexical, semantic or syntactic differences in use by developers? |
|
| enumeration | RE13.3.1.29_DevelopmentTeamsIdentifyExclusionBias | | appinfo | | Do development teams look for exclusion bias where specific groups of user populations are excluded from testing and subsequent analyses? |
|
| enumeration | RE13.3.1.3_DevelopmentTeamsUnintendedBiasesTrainingData | | appinfo | | Do development teams look for unintended biases in the training data? |
|
| enumeration | RE13.3.1.30_DevelopmentTeamsIdentifyFeedbackLoopBias | | appinfo | | Do development teams look for feedback loop bias where an algorithm learns from user behavior and feeds that behavior back into the model? |
|
| enumeration | RE13.3.1.31_DevelopmentTeamsIdentifyFundingBias | | appinfo | | Do development teams look for funding bias where biased results are reported in order to support or satisfy a funding agency or financial supporter? |
|
| enumeration | RE13.3.1.32_DevelopmentTeamsIdentifyHistoricalBias | | appinfo | | Do development teams look for historical bias where models are trained on past and potentially biased decisions? |
|
| enumeration | RE13.3.1.33_DevelopmentTeamsIdentifyMirrorImageBias | | appinfo | | Do development teams look for mirror imaging bias where an IAS is developed to counter an adversary force (such as in a military application) and the developers assume that members of the adversary force have the same cultural, ethical, and cognitive characteristics of members of their own force? |
|
| enumeration | RE13.3.1.34_DevelopmentTeamsIdentifyInheritedBias | | appinfo | | Do development teams look for inherited bias where tools that are built with machine learning are used to generate inputs for other machine learning algorithms? |
|
| enumeration | RE13.3.1.35_DevelopmentTeamsIdentifyInterpretationBias | | appinfo | | Do development teams look for interpretation bias where users interpret algorithmic outputs according to their internalized biases and views? |
|
| enumeration | RE13.3.1.36_DevelopmentTeamsIdentifyLinkingBias | | appinfo | | Do development teams look for linking bias where network attributes obtained from user connections, activities, or interactions differ and misrepresent the true behavior of the users? |
|
| enumeration | RE13.3.1.37_DevelopmentTeamsIdentifyLossSituationalAwarenessBias | | appinfo | | Do development teams look for loss of situational awareness bias where human- machine teaming leads to humans being unaware of their situation such that, when command and control of a system is given to them, they are unprepared to assume their duties? |
|
| enumeration | RE13.3.1.38_DevelopmentTeamsIdentifyModalConfusionBias | | appinfo | | Do development teams look for modal confusion bias where modal interfaces confuse human operators, who misunderstand which mode the system is using, taking actions which are correct for a different mode but incorrect for their current situation? |
|
| enumeration | RE13.3.1.39_DevelopmentTeamsIdentifyRankingBias | | appinfo | | Do development teams look for ranking bias where top-ranked information is perceived to be more important than lower ranking information? |
|
| enumeration | RE13.3.1.4_DevelopmentTeamsUnintendedIASAlgorithms | | appinfo | | Do development teams look for unintended biases in the IAS algorithms? |
|
| enumeration | RE13.3.1.40_DevelopmentTeamsIdentifySamplingBias | | appinfo | | Do development teams look for sampling bias where non-random sampling of subgroups, causing trends estimated for one population to not be generalizable to data collected from a new population? |
|
| enumeration | RE13.3.1.41_DevelopmentTeamsIdentifyTrainingDataBias | | appinfo | | Do development teams look for training data bias where algorithms are trained on one type of data and do not extrapolate beyond those data? |
|
| enumeration | RE13.3.1.42_DevelopmentTeamsIdentifyUncertaintyBias | | appinfo | | Do development teams look for uncertainty bias where predictive algorithms favor groups that are better represented in the training data, since there will be less uncertainty associated with those predictions? |
|
| enumeration | RE13.3.1.5_DevelopmentTeamsConsiderCulturalBiases | | appinfo | | Do development teams consider cultural biases in coalition and allied IAS and across joint services? |
|
| enumeration | RE13.3.1.6_DevelopmentTeamsConsiderSpuriousCorrelations | | appinfo | | Do development teams look for "data hubris" by attempting to identify the underlying causal mechanisms in their models so that spurious correlations identified by the IAS are not taken to be actual causal relationships? |
|
| enumeration | RE13.3.1.7_DevelopmentTeamsIdentifyBiasIASHumanOperators | | appinfo | | Do development teams look for bias introduced by IAS interaction with human operators? |
|
| enumeration | RE13.3.1.8_DevelopmentTeamsIdentifyBiasIASOpEnvSensor | | appinfo | | Do development teams look for bias introduced by IAS ingest of sensor data in the operational environment? |
|
| enumeration | RE13.3.1.9_DevelopmentTeamsIdentifyBlackBoxDecisionMakingBias | | appinfo | | Do development teams look for "black box" decision-making algorithms that make it difficult or impossible to detect bias? |
|
| enumeration | RE13.3.2.1_IASResultsRevisited | | appinfo | | Are IAS results revisited using human-on-the-loop oversight to look for biased results and rights violations that the IAS is not capable of identifying? |
|
| enumeration | RE13.3.2.2_IASOversightPersonsRepresentative | | appinfo | | Are human-on-the-loop oversight functions conducted by a group of persons representative of the population(s) potentially affected by the IAS? |
|
| enumeration | RE13.3.2.3_IASUseRestrictedDiscoveryBiasRightsViolation | | appinfo | | Is IAS use eliminated or curtailed when oversight groups discover bias or rights violations? |
|
| enumeration | RE13.3.2.4_IASBiasRightsDiscoveryDrivesEnterpriseModelUpdates | | appinfo | | When oversight groups discover bias or rights violations in one IAS, do they then look for the same bias in other IAS that have used all or part of the training data or algorithms used by the biased IAS? |
|
| enumeration | RE13.3.2.5_IASBiasRightsDiscoveryEnterpriseModelUpdatesConfirmation | | appinfo | | When oversight groups discover bias or rights violations in one IAS, do they ensure that a replacement or revision does not reproduce the same negative outcome, or introduce a new negative outcome? |
|
| enumeration | RE13.3.2.6_OversightReviewersBiasFamiliarity | | appinfo | | Are oversight groups familiar with "reviewer’s bias," where the reviewers may be susceptible to introducing their own sets of expertise, experiences, and biases into evaluations? |
|
