Eyes up and out: Advancing situational awareness in helicopter avionics

Story

March 12, 2020

Basic physics still dictates much of what makes helicopter flight successful, but military airborne platforms are constantly faced with environments civilian rotary- and fixed-wing aircraft simply don’t encounter: Degraded visual environments, a need for reduced workload, and improved pilot-vehicle interface drive military helicopter avionics upgrades and remain at the top of customer design requirements. In response to these military-user needs, companies are attempting to uncomplicate helicopter cockpit designs while using the most advanced electronics available.

One of the last things a helicopter pilot wants is to have to look down – ­especially in combat. Mission effectiveness requires the undivided attention of military helicopter pilots and crews, and recent avionics advancements intend to ensure that warfighters maintain the highest degree of situational awareness. Achieving that focus calls for the implementation of technology that prevents the pilot and crew from being consumed by arduous data processing, low-visibility environments, and critical yet intensive decision-making.

Size, weight, and power (SWaP) constraints; cybersecurity concerns; and standardization efforts can act as both catalysts and obstacles for emerging helicopter avionics. Standards-promulgating consortia including the Future Airborne Capabilities Environment (FACE), Sensor Open Systems Architecture (SOSA), and Modular Open Systems Architecture (MOSA) aim to alleviate the challenging design demands for companies, which are starting to treat these standards as de facto requirements rather than guidelines.

What both the warfighter and the manufacturer agree on, however, is that overarching all in the aerial battlefield is situational awareness. End users are often in environments where they not only need to fly the aircraft effectively, but simultaneously control other aerial systems, switch from piloting to mission management, and maintain vision even in degraded environments.

Situational awareness goes beyond sight

Improved situational awareness begins at a display level. Pilots are often equipped with heads-up, augmented-reality, helmet-mounted displays to project critical information directly into the pilot’s line of sight. Helicopter crews are supplemented with heads-down displays that can be programmed with speech recognition or resistive touch, while large-format displays aid in managing content from a variety of positions within the cockpit.

When flying in environments degraded by fog, smoke, rain, and other obscurants that affect visibility, these displays are then used to project augmented depictions of the helicopter’s surroundings so as to keep the pilot and crew aware and in the air. Called synthetic vision, this capability persists as a must-have for military helicopter avionics, as it provides a basis for situational awareness and introduces various levels of sensor processing (Figure 1).

Figure 1 | By combining synthetic vision systems and its Avoidance Re-Router cognitive decision-aiding application, the Collins Aerospace software tool enables pilots to quickly react to stationary or moving threats encountered along the flight path, including in cases of low visibility or unfamiliar territory. Image courtesy Collins Aerospace.

“Synthetic vision is only as good as the database that it pulls from. With in­creased fidelity in terrain maps, available datasets, real-time information, contextual data, and ability to employ active sensors on platforms, we can synthesize a very robust digital world model view,” says Jon McMillen, who works with mission systems business development at Lockheed Martin Rotary and Mission Systems (Washington, D.C.). “This information can be provided to pilots in the form of synthetic vision to inform them of their surroundings or can also be used for advanced flight modes all the way to fully autonomous flights.”

Companies are pushing toward multispectral, adaptable solutions to adhere to various conditions and retain upgradability as requirements shift and technology advances. Situational awareness also includes having full control over any aerial systems during missions that require advanced teaming, the perpetuation of a manageable workload, and the use of cognitive decision-making.

“It’s not only having the vision in terms of what their eyes see, but it’s also indicators, symbology, and cues that will guide them to additional decisions that they may need to make and give them options rather than overloading them with a ton of information and then asking them to figure it out while they’re in the middle of a mission,” says Dave Schreck, vice president and general manager for military avionics and helicopters at Collins Aerospace (Cedar Rapids, Iowa).

Manufacturers have also commented on the need to maintain reconfigurability between spectrums to better achieve on-the-fly switches between sensors. Ensuring that situational awareness capabilities provide multifunctionality is paramount not only because every pound counts when in flight, but because standardization efforts are quickly catching up with legacy platforms.

Standardization and system security

FACE, MOSA, and SOSA have been influencing military helicopter avionics since the inception of the consortia several years ago. Current airborne platform purchases are intended to serve for decades to come, with mission equipment refreshes to take place at a quicker rate than that of the helicopters. This reality means that manufacturers are seeing a demand for open systems, which enable their users to obtain the equipment and the vehicles independently.

“The services are looking at open architecture for two reasons: one, to reduce cost. Because if you can buy a set of applications, or an application once and apply it multiple times across multiple platforms or multiple services, then there are certainly costs to be lowered there,” Schreck says. “But more importantly, they’re looking for ways to separate the mission-critical, flight-critical pieces of the platform from the mission applications. That way you aren’t constrained by breaking open an operational flight platform.”

Standards conformance becoming more important

Manufacturers are more frequently considering FACE conformance as a requirement, especially when considering the cybersecurity risks that avionics systems can become vulnerable to when multiple sensor-driven capabilities are used. Standards like DO-178C [Software Considerations in Airborne Systems and Equipment Certification] provide the guidelines to secure an airborne platform’s software while preserving the system’s open architecture. Helicopter avionics companies are also recognizing the real possibility of cyberattack and implementing their own solutions.

“Our approach to avionics system security, referred to as Program Protection, is designed to protect technology, components, and information from compromise through the cost-effective application of countermeasures,” says Jim Conroy, vice president of navigation, targeting, and survivability for Northrop Grumman (Falls Church, Virginia). “We consider many factors in developing secure systems, including integrating, mitigating, and managing risks to advanced technology and mission-­critical system functionality from foreign collection, design vulnerability or supply chain exploit, battlefield loss, and unauthorized or inadvertent disclosure throughout the acquisition life cycle.”

Because the threat environment is rapidly evolving, without reliable security, taking advantage of the capabilities that open and multispectral solutions offer would put helicopter platforms in a far-too-vulnerable a position. Demonstrating the digital interoperability of an advanced yet comprehensible pilot-vehicle interface is driving not only standardization efforts, but also general trends in helicopter avionics.

Simplifying pilot-vehicle interface

According to industry officials, the next major step in military helicopter avionics will be a shift from piloting to mission management. With that change, new ways to interact with the system will be introduced. Manufacturers are working to design innovative methods to present data to the pilot as well as ways to receive pilot inputs to provide operators with the ability to transition in the middle of the flight from being the pilot to being the mission operator.

“Many customers are looking for ways to address changing mission requirements by adding functionality to aircraft,” McMillen says. “Leveraging the latest in open system design, we’re able to add specific mission equipment packages and provide additional information to the pilots in the cockpit. As we increase the level of automation in helicopters, open systems allow rapid adaptation in the way information is presented to effectively manage mission objectives.”

This approach creates a streamlined human-machine interaction, improved speed of information access, and fusing of that information. The data is then presented through the system as an actionable next step for the pilot as opposed to providing them with different pieces of information that they then need to take the time to process, integrate, and react to. “There is a lot of data to consume, so the key is to be able to pull the necessary information from the consolidated-world model and then present only the data that will aid in decision-making to reduce information overload,” McMillen says. “The goal being that we provide all the necessary information to the pilot or onboard system at all times to eliminate controlled flight into terrain (CFIT), but not increase workload.”

Consequently, artificial intelligence (AI) and machine learning (ML) have become promising solutions when considering ways to further methodize the pilot-vehicle interface on military helicopters. Expanding workload reduction capabilities with autonomous functionalities is becoming a necessity for the end user, and manufacturers are taking notice.

Avoiding task saturation with AI and ML

“The requirement sets for artificial intelligence and machine learning are being developed in terms of how that can be most leveraged in military helicopter avionics,” Schreck says. “Workload-reduction systems and taking multiple inputs from different systems and subsystems in the cockpit and then providing those in a fused picture to the pilot are a couple of ways that we are already starting to venture down the path of AI and ML.”

The use of AI and ML appears to be most beneficial when incorporated into mission-support applications: Leveraging those autonomous capabilities in workload reduction efforts offer lower risk from a flight-critical nature, while still providing significant value and assistance for the end user.

“We continue to drive crew workload down to avoid task saturation and better enable the warfighter. This is particularly important when a crew or aircraft is in an urgent situation,” Conroy says. “Our cognitive decision-aiding and machine augmentation capabilities are designed to assist the crew in combat, and if necessary, automatically control the aircraft when required.” As military helicopter avionics push deeper into military missions, Conroy asserts that Northrop Grumman also intends to implement AI and ML into their avionics mission solutions in ways that include sensor data processing and exploitation, augmented situational awareness, threat intent prediction, and response planning and execution.

Optionally piloted airborne platforms are also at the vanguard of AI- and ML-powered helicopter avionics. At the 2019 Paris Air Show, Sikorsky (a Lockheed Martin subsidiary), together with Collins Aerospace, demonstrated technology that the companies said would enable fielding of a pilotless UH-60 Black Hawk. (Figure 2.) The soon-to-be-unmanned vehicle will take to the skies for the same reasons autonomous workload reduction systems exist: to let pilots and crews commit more time to the mission. But that in no way means that autonomous advancements will stop there.

Figure 2 | Sikorsky tests its fly-by-wire technology that the company says will enable autonomous flight for rotary- and fixed-wing aircraft. Lockheed Martin photo.

“When you start to focus on how the machine does something more predictably based on machine inputs and a variety of inputs that may change and vary over time, that’s where you really get into the more challenging areas,” Schreck says. “That’s where we’re working with the customers and the rest of the industry, looking at how far you can really push that line when you’re talking about the avionics, the flying piece of the mission.”