Validation and verification of the remote agent for spacecraft autonomy

Ben Smith, William Millar, Julia Dunphy, Yu Wen Tung, Pandu Nayak, Ed Gamble, Micah Clark

Research output: Contribution to conferencePaper

19 Citations (Scopus)

Abstract

The six-day Remote Agent Experiment (RAX) on the Deep Space 1 mission will be the first time that an artificially intelligent agent will control a NASA spacecraft. Successful completion of this experiment will open the way for AI-based autonomy technology on future missions. An important validation objective for RAX is implementation of a credible validation and verification strategy for RAX that also 'scales up' to missions that make full use of spacecraft autonomy. Autonomous flight software presents novel and difficult testing challenges that traditional flight software (FSW) does not face. Since autonomous software must respond robustly in an immense number of situations, the all-paths testing approaches used for traditional FSW is not feasible. Instead, we advocate a combination of scenario-based testing and model-based validation. This paper describes the testing challenges faced by autonomous spacecraft commanding software, discusses the testing strategies and model-validation methods that we found effective for RAX, and argues that these methods will 'scale up' to missions that make full use of spacecraft autonomy. Among the key challenges for validating autonomous systems such as the RAX are ensuring adequate coverage for scenario-based tests, developing methods for specifying the expected behavior, and developing automated tools for verifying the observed behavior against those specifications. Another challenge, also faced by traditional FSW, is the scarcity of high-fidelity test-beds. The test plan must be designed to take advantage of lower-fidelity test-beds without compromising test effectiveness.

Original languageEnglish (US)
Pages449-468
Number of pages20
StatePublished - Jan 1 1999
EventProceedings of the 1999 IEEE Aerospace Conference - Snowmass at Aspen, CO, USA
Duration: Mar 6 1999Mar 13 1999

Conference

ConferenceProceedings of the 1999 IEEE Aerospace Conference
CitySnowmass at Aspen, CO, USA
Period3/6/993/13/99

Fingerprint

Spacecraft
Testing
Experiments
Intelligent agents
NASA
Specifications

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Smith, B., Millar, W., Dunphy, J., Tung, Y. W., Nayak, P., Gamble, E., & Clark, M. (1999). Validation and verification of the remote agent for spacecraft autonomy. 449-468. Paper presented at Proceedings of the 1999 IEEE Aerospace Conference, Snowmass at Aspen, CO, USA, .
Smith, Ben ; Millar, William ; Dunphy, Julia ; Tung, Yu Wen ; Nayak, Pandu ; Gamble, Ed ; Clark, Micah. / Validation and verification of the remote agent for spacecraft autonomy. Paper presented at Proceedings of the 1999 IEEE Aerospace Conference, Snowmass at Aspen, CO, USA, .20 p.
@conference{c6476f5462ac43888a07eb6cd7e6cb40,
title = "Validation and verification of the remote agent for spacecraft autonomy",
abstract = "The six-day Remote Agent Experiment (RAX) on the Deep Space 1 mission will be the first time that an artificially intelligent agent will control a NASA spacecraft. Successful completion of this experiment will open the way for AI-based autonomy technology on future missions. An important validation objective for RAX is implementation of a credible validation and verification strategy for RAX that also 'scales up' to missions that make full use of spacecraft autonomy. Autonomous flight software presents novel and difficult testing challenges that traditional flight software (FSW) does not face. Since autonomous software must respond robustly in an immense number of situations, the all-paths testing approaches used for traditional FSW is not feasible. Instead, we advocate a combination of scenario-based testing and model-based validation. This paper describes the testing challenges faced by autonomous spacecraft commanding software, discusses the testing strategies and model-validation methods that we found effective for RAX, and argues that these methods will 'scale up' to missions that make full use of spacecraft autonomy. Among the key challenges for validating autonomous systems such as the RAX are ensuring adequate coverage for scenario-based tests, developing methods for specifying the expected behavior, and developing automated tools for verifying the observed behavior against those specifications. Another challenge, also faced by traditional FSW, is the scarcity of high-fidelity test-beds. The test plan must be designed to take advantage of lower-fidelity test-beds without compromising test effectiveness.",
author = "Ben Smith and William Millar and Julia Dunphy and Tung, {Yu Wen} and Pandu Nayak and Ed Gamble and Micah Clark",
year = "1999",
month = "1",
day = "1",
language = "English (US)",
pages = "449--468",
note = "Proceedings of the 1999 IEEE Aerospace Conference ; Conference date: 06-03-1999 Through 13-03-1999",

}

Smith, B, Millar, W, Dunphy, J, Tung, YW, Nayak, P, Gamble, E & Clark, M 1999, 'Validation and verification of the remote agent for spacecraft autonomy' Paper presented at Proceedings of the 1999 IEEE Aerospace Conference, Snowmass at Aspen, CO, USA, 3/6/99 - 3/13/99, pp. 449-468.

Validation and verification of the remote agent for spacecraft autonomy. / Smith, Ben; Millar, William; Dunphy, Julia; Tung, Yu Wen; Nayak, Pandu; Gamble, Ed; Clark, Micah.

1999. 449-468 Paper presented at Proceedings of the 1999 IEEE Aerospace Conference, Snowmass at Aspen, CO, USA, .

Research output: Contribution to conferencePaper

TY - CONF

T1 - Validation and verification of the remote agent for spacecraft autonomy

AU - Smith, Ben

AU - Millar, William

AU - Dunphy, Julia

AU - Tung, Yu Wen

AU - Nayak, Pandu

AU - Gamble, Ed

AU - Clark, Micah

PY - 1999/1/1

Y1 - 1999/1/1

N2 - The six-day Remote Agent Experiment (RAX) on the Deep Space 1 mission will be the first time that an artificially intelligent agent will control a NASA spacecraft. Successful completion of this experiment will open the way for AI-based autonomy technology on future missions. An important validation objective for RAX is implementation of a credible validation and verification strategy for RAX that also 'scales up' to missions that make full use of spacecraft autonomy. Autonomous flight software presents novel and difficult testing challenges that traditional flight software (FSW) does not face. Since autonomous software must respond robustly in an immense number of situations, the all-paths testing approaches used for traditional FSW is not feasible. Instead, we advocate a combination of scenario-based testing and model-based validation. This paper describes the testing challenges faced by autonomous spacecraft commanding software, discusses the testing strategies and model-validation methods that we found effective for RAX, and argues that these methods will 'scale up' to missions that make full use of spacecraft autonomy. Among the key challenges for validating autonomous systems such as the RAX are ensuring adequate coverage for scenario-based tests, developing methods for specifying the expected behavior, and developing automated tools for verifying the observed behavior against those specifications. Another challenge, also faced by traditional FSW, is the scarcity of high-fidelity test-beds. The test plan must be designed to take advantage of lower-fidelity test-beds without compromising test effectiveness.

AB - The six-day Remote Agent Experiment (RAX) on the Deep Space 1 mission will be the first time that an artificially intelligent agent will control a NASA spacecraft. Successful completion of this experiment will open the way for AI-based autonomy technology on future missions. An important validation objective for RAX is implementation of a credible validation and verification strategy for RAX that also 'scales up' to missions that make full use of spacecraft autonomy. Autonomous flight software presents novel and difficult testing challenges that traditional flight software (FSW) does not face. Since autonomous software must respond robustly in an immense number of situations, the all-paths testing approaches used for traditional FSW is not feasible. Instead, we advocate a combination of scenario-based testing and model-based validation. This paper describes the testing challenges faced by autonomous spacecraft commanding software, discusses the testing strategies and model-validation methods that we found effective for RAX, and argues that these methods will 'scale up' to missions that make full use of spacecraft autonomy. Among the key challenges for validating autonomous systems such as the RAX are ensuring adequate coverage for scenario-based tests, developing methods for specifying the expected behavior, and developing automated tools for verifying the observed behavior against those specifications. Another challenge, also faced by traditional FSW, is the scarcity of high-fidelity test-beds. The test plan must be designed to take advantage of lower-fidelity test-beds without compromising test effectiveness.

UR - http://www.scopus.com/inward/record.url?scp=0032598558&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032598558&partnerID=8YFLogxK

M3 - Paper

SP - 449

EP - 468

ER -

Smith B, Millar W, Dunphy J, Tung YW, Nayak P, Gamble E et al. Validation and verification of the remote agent for spacecraft autonomy. 1999. Paper presented at Proceedings of the 1999 IEEE Aerospace Conference, Snowmass at Aspen, CO, USA, .