HOLLOMAN AIR FORCE BASE, N.M. -- In the desert of New Mexico the men and women of the 846th Test Squadron, 704th Test Group, Arnold Engineering Development Complex send test articles strapped to rocket sleds hurtling along rail tracks up to 10 miles long.
Some tests, such as ejection seat and target penetration testing, send debris across the desert; which is catalogued as part of the data collection for the test.
Project Zero is an effort by the 846 TS and the Strategic Development Planning and Experimentation Office, or SDPE, of the Air Force Research Laboratory, which aims to automate the task of identifying and tracking both planned and unplanned debris from tests at the Holloman High Speed Test Track, or HHSTT, with the use of small, unmanned aircraft systems, or sUAS, better known as drones, and machine-learning algorithms, a subset of artificial intelligence, or AI.
“If successful, Project Zero would reduce Airmen’s time searching and finding debris components, and increase safety by ensuring explosive ordnance disposal personnel spend less time in an area of dangerous wildlife, unexploded ordnances and desert heat,” said Maj. Ryan Middleton, C-130 navigator and experimentation lead with SDPE. “Project Zero may also improve data collection, providing a unique perspective to capture tests.”
The 846 TS has used sUAS to obtain a bird’s-eye view at the track, but it was done in a manual mode, as opposed to the automated, artificial-intelligent driven operations and data analysis being pursued through Project Zero.
"We are always open to automating manual processes and exploring solutions that may increase efficiency and safety,” said 2nd Lt. Aaron Runnells, a rocket sled test engineer with the 846 TS. “The idea of leveraging machine learning to train drones for use in the real world, or use in real-world scenarios, may unlock a new means of data collection without the hazards of having to be physically on location."
The effort at HHSTT provides an opportunity to compare the use of synthetic data against using real-world data for the training of models. The team is also studying how digital environments may be used to understand vulnerabilities and resiliency in machine-learning algorithms prior to being deployed.
"The partnership between the Air Force Test Center’s 846th Test Squadron and AFRL's SDPE division is a perfect match; we offer a relevant use-case, along with decades of data to train models, and SDPE provides the expertise and experimentation team to quickly determine how quickly we can field this capability," Runnells said.
The test case scenario for Project Zero involves testing ejection events on a variety of aircraft. When the ejection system is used, the canopy is fractured to allow for the pilot to be ejected from the aircraft. This creates a field of debris that must be identified, located and collected.
Project Zero is digitally recreating the HHSTT with open-source tools from the entertainment and video-gaming industry. Within the digital HHSTT, ejection system testing will be recreated based upon physics models and using video-gaming engines, and utilized to train machine-learning models to identify, track and report location of the debris.
“The digital environment allows for efficient training by modeling different scenarios – environmental conditions, test scenarios, malfunctions – to increase the resiliency of the models,” Middleton said. “And our open-source approach is how we move fast – we tap into the world-wide community of software developers for updates, bug fixes and new tools.”
Conducting research to find scalable ways of ensuring artificial-intelligence and machine-learning models are safe and resilient, such as Project Zero, is in line with the Department of Defense strategy for responsible AI.
“The 846th Test Squadron, with its unique mission, multidisciplinary team and range, offers the proving ground for machine learning experimentation with real-world use cases,” Middleton said. “SDPE is excited to partner with the 846th Test Squadron, not just for the experimentation opportunity, but also that Project Zero may improve data collection and improve safety for the 846th Test Squadron. The open-source, digital tools that create the environment can be used by other organizations in the DOD to build their own events to train ML [machine-learning] models.”
Runnells agreed.
"If we get this right, we will demonstrate the ability to train a drone with machine-learning in a digital environment where simulations can be repeated multiple times saving time and money,” he said.
Middleton also noted how efforts such as Project Zero provide opportunities for DOD to obtain support from industries which are not traditionally seen as part of the defense industrial base.
“This is one example of how to attract new and non-traditional talent to support AEDC mission sets,” he said. “The innovative vision of the 846th Test Squadron leadership allows for new partnerships with industry, and the technical competence of AEDC engineers excites industry engineers to support critical DOD mission sets.
“Project Zero includes virtual effects artists from Hollywood’s entertainment industry as well as roboticists and machine learning experts – some who would not traditionally consider working with the DOD. They were drawn to this effort given the complex engineering challenges of the HHSTT, and the idea that outcomes could increase safety for pilots and explosive ordnance disposal personnel.”