There is currently a lack of information about landing sites for helicopters and other rotorcrafts, and acquiring, verifying, and updating this database of information is no small task for organizations such as the FAA that need it. A project from UTC partners at Rowan University is working to develop an AI-based algorithm that will automate the process of identifying landing sites from video data as well as satellite images—increasing safety and efficiency in the NAS.
A helicopter flies over Colts Neck, N.J. Photo ©Wikimedia Commons.
As companies such as Uber look to the National Airspace System (NAS) as the future of transportation with Uber Air along with other ridesharing technologies, it brings up a number of questions about safety.
Currently, air transportation has an extremely low number of fatalities and injuries each year, but if air mobility and transportation grows exponentially, then it could bring with it potential safety issues due to thousands of new air-delivery drones and air taxis being added to an already congested NAS. Before we get there, Artificial Intelligence (AI) technology will need to advance to certain levels to make these new modes of transportation safe and secure.
But, with thousands of helicopter and rotorcraft pilots already flying in the United States every year, safety is already the number one priority for organizations such as the Federal Aviation Administration (FAA). One area in particular is identifying and maintaining information on helipads, heliports, and landing site infrastructure. A new project from University Transportation Center (UTC) partners at Rowan University aims to address these concerns by developing an AI-based identification system for rotorcraft landing sites.
“This research aligns with the DOT Strategic Goal of Reducing Transportation-Related Fatalities and Serious Injuries across the Transportation System,” said Cliff Johnson, FAA’s lead for Vertical Flight Safety Research at the FAA William J. Hughes Technical Center. “By identifying locations of heliports, helipads, landing zones, and other infrastructure from various data sources, the research will increase safety for helicopter pilots and their flight crews/passengers. The technology we are exploring with Rowan University through the use of AI/ML offers the possibility to one day allow a pilot of an rotorcraft/eVTOL to be able to continue an approach and land by enhancing their visual acquisition of the landing environment (either via supplemental or autonomous means) in both normal and low-visibility operations.”
“Through our partnership with Rowan University and industry/government partners such as LZControl and NJDOT, we are leveraging existing databases and developing new techniques that support the work of FAA and industry safety groups, including the United States Helicopter Safety Team, which has identified a need to improve and enhance low-altitude infrastructure,” he continued.
Led by Dr. Ghulam Rasool, an Assistant Professor at Rowan University’s College of Engineering; Dr. Nidhal Bouaynaya, a professor and associate dean of Research and Graduate Studies at Rowan; and Dr. Mohammad Jalayer an assistant professor at Rowan University, the project began this winter with gathering video and image data from the FAA and Google Earth satellite image database.
“Updated information about the location and type of rotorcraft landing sites is an essential asset for the FAA and the Department of Transportation,” Dr. Rasool said. “However, these organizations currently do not have an updated and accurate registry of these landing sites. That is why we want to automate the process using AI and multiple data sources, ultimately simplifying the process while mitigating risk factors and promoting safety.”
The end goal is to develop AI algorithms that will automate the process of identification of landing sites from video data as well as satellite images with accuracy equal to or higher than a trained human operator—at a fraction of the time and resources, he said.
Once developed, the AI system would allow the FAA to update its databases of landing sites regularly without any delays. A key component of the effort relies heavily on the team’s partnership with LZControl. In 2019, the FAA and LZControl signed a cooperative research and development agreement to conduct testing for the purposes of improving the U.S. airport master record (5010) database for heliports and landing zones. That agreement covers the work undertaken with Rowan University. As part of this effort, LZControl is providing access to landing sites via its database and the research team is developing algorithms that would further enhance the safety and accuracy of these landing sites across the U.S.
“With the issuance of the January 2019 NASA Safety Report, ASRS No.1599969, that indicated upwards of some 2,000 heliports were unaccounted for in the U.S., LZControl has been working diligently to help facilitate a more comprehensive and reliable database system both in the U.S. and internationally,” said Jonathan Godfrey, COO at LZControl. “Working with Rowan University allows us to engineer improvements to the collection, accuracy, and identification of these sites as we work collaboratively with the FAA to improve the safety of rotorcraft and vertical lift operations.”
“I see this effort being a major opportunity in not only helping coalesce the large number of private-use facilities currently unaccounted for in the U.S. but also for paving the way for the future of Urban Air Mobility, where Data-Integrity for infrastructure will be paramount to safety and success”, said Rex Alexander, SVP/Aeronautical Integration at LZControl. “These efforts could one day provide the necessary oversight to the Secretary of Transportation to achieve their mandate by U.S. Code for the collection and dissemination of airport safety data beyond just public-use facilities to encompass all private-use facilities as well. Working with the FAA and Rowan University allows us to accomplish these objectives.”
Dr. Rasool explained that the acquisition, verification, and regular updating of this data traditionally has been a manual task that is both time-intensive as well as error prone. The team is developing a pioneering and transformative technology that would integrate with FAA’s current workflow and allow them to update their landing infrastructure databases regularly with minimal human intervention. The team said that the current AI system are black-box solutions, that is, these models and algorithms cannot explain their decisions. This drawback severely limits the applicability of these modern AI systems in real-world scenarios.
So to start, researchers are gathering videos from rotorcrafts, satellite and aerial imagery from Google Earth, as well as data entered by helipad owners, operators, or pilots. They are now cleaning and labeling this data and doing preliminary investigation into promising state-of-the-art algorithms that will eventually be used to create AI models.
“Our preliminary results show that we can identify landing site infrastructure with more than 90% accuracy using Google Earth satellite images. More importantly, we can explain the decisions made by the AI system. This is Explainable AI: cracking open the black box of AI,” Dr. Bouaynaya said.
Explainable AI – Detection of helipad from satellite images. (Left) Input image to the AI algorithm. (Right) The output of the AI algorithm showing the areas in the image that influenced the algorithm to make the decision about the presence of a helipad. Red pixels highlight strong contribution while blue pixels indicate weak contribution to the decision about the presence of the helipad. Photo ©Rowan University.
“Safety is critical across all modes of transportation, but especially when it comes to aviation,” Dr. Rasool said. “With this research we plan on developing a technology that can help the FAA further achieve its safety goals through automating the process of identifying rotorcraft landing sites from various datasets. This will help helipad owners, operators, or pilots operate with more confidence and safety.”
