Integrated Data Uncertainty Management and Representation for Trustworthy and Trusted Autonomy in Space

Scope Title:

Integrated Data Uncertainty Management and Representation for Trustworthy and Trusted Autonomy in Space

Scope Description:

Multi-agent cyber-physical-human (CPH) teams in future space missions must include machine agents with a high degree of autonomy. In the context of this subtopic, by “autonomy” we mean the capacity and authority of an agent (human or machine) for independent decision making and execution in a specified context. We refer to machine agents with these attributes as autonomous systems (AS). In multi-agent CPH teams, humans may serve as remote mission supervisors or as immediate mission teammates, along with AS. AS may function as teammates with specified independence, but under the ultimate human direction. Alternatively, AS may exercise complete independence in decision making and operations in pursuit of given mission goals; for instance, for control of uncrewed missions for planetary infrastructure development in preparation for human presence, or maintenance and operation of crew habitats during the crew’s absence.

In all cases, trustworthiness and trust are essential in CPH teams. The term “trustworthiness” denotes the degree to which the system performs as intended and does not perform prohibited actions in a specified context. “Trust” denotes the degree of readiness by an agent (human or machine) to accept direction or advice from another agent (human or machine), also in a specified context. In common sense terms, trust is a confidence in a system’s trustworthiness, which in turn, is the ability to perform actions with desired outcomes.

Because behind every action lies a decision-making problem, the trustworthiness of a system can be viewed in terms of the soundness of decision making by the system participants. Accurate and relevant information forms the basis of sound decision making. In this subtopic, we focus on data that inform CPH team decision making, both in human-machine and machine-machine interactions, from two perspectives: the quality of the data and the representation of the data in support of trusted human-machine and machine-machine interactions. 

Consider data exchanges in multi-agent cyber-physical-human (CPH) teams that include AS, as described in the subtopic introduction. Data exchanges in multi-agent teams must be subject to the following conditions:

• Known data accuracy, noise characteristics, and resolution as a function of the physical sensors in relevant environments.
• Known data accuracy, noise characteristics, and resolution as a function of data interpretation if the contributing sensors have a perception component or if data are delivered to an agent via another perception engine (e.g., visual recognition based on deep learning).
• Known data provenance and integrity.
• Dynamic anomaly detection in data streams during operations.
• Comprehensive uncertainty quantification (UQ) of data from a single source.
• Data fusion and combined UQ if multiple sources of data are used for decision making.
• If data from either a single source or fused data from multiple sources are used for decision making by an agent (human or machine), the data and the attendant UQ must be transformed into a representation conducive to and productive for decision making. This may include data filtering, compression, or expansion, among other approaches.
• UQ must be accompanied by a sensitivity analysis of the mission/operation/action goals with respect to uncertainties in various data, to enable appropriate risk estimation and risk-based decision making by relevant agents, human or machine.
• Tools for real-time, a priori, and a posteriori data analysis, with explanations relevant to participating agents. For instance, if machine learning is used for visual data perception in decision making by humans, methods of interpretable or explainable AI (XAI) may be in order.

We note that deep learning and machine learning, in general, are not the chief focus of this subtopic. The techniques are mentioned as an example of tools that may participate in data processing. If such tools are used, the representation of the results to decision makers (human or machine) must be suitably interpretable and equipped with UQ.

Addressing the entire set of the conditions listed above would likely be impractical in a single proposal. Therefore, proposers may offer methods and tools for addressing a subset of conditions.

Proposers should offer both a general approach to achieving a chosen subset of the listed conditions and a specific application of the general approach to appropriate data types. The future orbiting or surface stations are potential example platforms because the environment would include a variety of AS used for habitat maintenance when the station is uninhabited, continual system health management, crew health, robotic assembly, and cyber security, among other functions. However, the proposers may choose any relevant design reference mission for demonstration of proposed approaches to integrated data uncertainty management and representation, subject to a convincing substantiation of the generalizability and scalability of the approach to relevant practical systems, missions, and environments.

Expected TRL or TRL Range at completion of the Project: 2 to 5

Primary Technology Taxonomy:

• Level 1 10 Autonomous Systems
• Level 2 10.1 Situational and Self Awareness

Desired Deliverables of Phase I and Phase II:

• Research
• Analysis
• Software

Desired Deliverables Description:

Since UQ and management in data is an overarching theme in this subtopic, an analysis of uncertainties in the processes and data must be present in all final deliverables, both in Phases I and II.

Phase I: For the areas selected in the proposal, the following deliverables would be in order:

1. Thorough but succinct analysis of the state of the art in the proposed area under investigation.
2. Detailed description of the problem used as the context for algorithm development, including substantiation for why this is a representative problem for a set of applications relevant to NASA missions.
3. Detailed description of the approach, including pseudocode, and the attendant design of experiments for testing and evaluation.
4. Hypotheses about the scalability and generalizability of the proposed approach to realistic problems relevant to NASA missions.
5. Preliminary software and process implementation.
6. Preliminary demonstration of the software.
7. Thorough analysis of performance and gaps.
8. Detailed plan for Phase II, including the design reference mission and the attendant technical problem.
9. Items 1 to 8 documented in a final report for Phase I.

Phase II:

1. Detailed description and analysis of the design reference mission and the technical problem selected in Phase I, in collaboration with NASA Contracting Officer Representative (COR)/Technical Monitor (TM).
2. Detailed description of the approach/algorithms developed further for application to the Phase II design reference mission and problem, including pseudocode and the design of experiments for testing and evaluation.
3. Demonstration of the algorithms, software, methods, and processes.
4. Thorough analysis of performance and gaps, including scalability and applicability to NASA missions.  
5. Resulting code.
6. Detailed plan for potential Phase III.
7. Items 1 to 5 documented in a final report for Phase II.

State of the Art and Critical Gaps:

Despite progress in real-time data analytics, serious gaps remain that will present an obstacle to the operation of systems in NASA missions that require heavy participation of AS, both in human-machine teams and in uncrewed environments, whether temporary or permanent. The gaps come under two main categories:

1. Quality of the information based on various data sources—Trustworthiness of the data is essential in making decisions with desired outcomes. This gap can be summarized as the lack of reliable and actionable UQ associated with data, as well as the difficulty of detecting anomalies in data and combining data from disparate sources, ensuring appropriate quality of the result.
2. Representation of the data to decision makers (human or machine) that is conducive to trustworthy decision making—We distinguish raw data from useful information of appropriate complexity and form. Transforming data, single-source or fused, into information productive for decision making, especially by humans, is a challenge.

Specific gaps are listed under the Scope Description as conditions the subsets of which must be addressed by proposers.

Relevance / Science Traceability:

The technologies developed as a result of this subtopic would be directly applicable to the Space Technology Mission Directorate (STMD), Science Mission Directorate (SMD), Exploration Systems Development Mission Directorate (ESDMD), Space Operations Mission Directorate (SOMD), and  Aeronautics Research Mission Directorate (ARMD), as all of these mission directorates are heavy users of data and growing users of AS. For instance, the Gateway mission will need a significant presence of AS, as well as human-machine team operations that rely on AS for habitat maintenance when the station is uninhabited, continual system health management, crew health, robotic assembly, among other functions. Human presence on the Moon surface will require similar functions, as well as future missions to Mars. All trustworthy decision making relies on trustworthy data. This topic addresses gaps in data trustworthiness, as well as productive data representation to human-machine teams for sound decision making.

The subtopic is also directly applicable to ARMD missions and goals because future airspace will heavily rely on AS. Thus, the subtopic is applicable to such projects as Airspace Operations and Safety Program (AOSP)/Advanced Air Mobility (AAM) and Air Traffic Management—eXploration (ATM-X). The technologies developed as a result of this subtopic would be applicable to the National Airspace System (NAS) in the near future as well, because of the need to process data related to vehicle and system performance.

References:

• Frontiers on Massive Data Analysis, NRC, 2013.
• NASA OCT Technology Roadmap, NASA, 2015.
• NASA AIST Big Data Study, NASA/JPL, 2016.
• IEEE Big Data Conference, Data and Computational Science Big Data Challenges for Earth and Planetary Science Research, IEEE, 2016.
• Planetary Science Informatics and Data Analytics Conference, April 2018.
• David L. Hall, Alan Steinberg: Dirty Secrets in Multisensor Data Fusion, The Pennsylvania State University Applied Research Laboratory. https://apps.dtic.mil/dtic/tr/fulltext/u2/a392879.pdf
• Martin Keenan: The Challenge and the Opportunity of Sensor Fusion, a Real Gamechanger, 5G Technology World, February 20, 2019. https://www.5gtechnologyworld.com/the-challenge-and-the-opportunity-of-sensor-fusion-a-real-gamechanger/