Augmented-reality technologies are here: Defense organizations must acknowledge the security concerns

Story

November 20, 2019

Duncan McSporran

Kognitiv Spark

Augmented-reality (AR) and mixed-reality (MR) technologies present unique opportunities in the defense and aerospace world but at the same time can introduce security risks. AR/MR solutions enable soldiers, technicians, subject matter experts, military strategists, and the like to overlay visual representations – commonly referred to as holograms or heads-up displays – of data, systems, machines, and other information into their real-world space. These technologies are enabling organizations to facilitate more robust and complex training scenarios, increase the speed of repair and maintenance, and increase knowledge transfer, among other benefits.

Augmented-reality (AR) and mixed-reality (MR) technologies are reaching maturity, as a number of the world’s militaries adopt various AR/MR solutions. Organizations in industries that require a high level of security and data sovereignty need to be informed and examine these solutions critically before piloting or deploying these technologies, however, as launching a nonsecure AR/MR application in a military environment could expose the collected data to risk. Defense organizations must consider certain security considerations as they move to adopt and implement these revolutionary technologies.

An overview of AR for defense

Whether it’s the internet, GPS, or virtual reality (VR), defense organizations have a long-standing history of creating or leading the adoption and development of emerging technologies. AR and MR is no exception.

These technologies enable digital information to be superimposed over an end user’s field of view or real-world environment. This is done through an AR/MR hardware and accompanying software designed for a specific use or set of uses. The ability to deliver data instantly to remote end users can dramatically improve situational awareness, task comprehension, and knowledge retention. On the other end, mission coordinators, subject matter experts (SMEs), and strategists can better support end users while remaining at a safe, remote location.

Applications of AR and MR solutions are diverse. They can help a factory worker in an industrial park in North Dakota, a weapons technician in the middle of the Pacific Ocean, or a soldier on a front line. When accessed through a head-mounted spatial computing system like the Microsoft HoloLens or the Magic Leap, AR/MR solutions ensure end users are heads-up, eyes out, and hands free. These factors increase reaction time in critical situations and deliver data while keeping users situationally aware, enhancing decision action cycles and reducing cognitive stress on the user.

AR/MR use in defense

AR/MR technologies are currently in use by a number of defense organizations including the U.S. military and the Canadian Department of National Defense. Many other defense and aerospace organizations are looking to adopt these technologies for use in three key areas: active use, operational readiness/efficiency, and training. Use cases and specific AR/MR solution vary from organization to organization.

Active use: Active use of AR is identified as any solution that can be used in combat scenarios or operational environments. This application of AR in defense has yet to materialize as standard equipment for soldiers; however, the U.S. military is currently developing an MR system for this use case known as the Integrated Visual Augmentation System (IVAS). IVAS is based on the Microsoft HoloLens 2 and will feature capabilities including night vision, thermal imaging, hostile and friendly target identification, navigational data, and more. This system revolutionizes the way soldiers and command operators share data, delivering mission-critical information directly into a soldier’s field of view. Active-use solutions have impact beyond how combat operations are conducted: For instance, through “see-what-I-see” solutions, combat medics and bomb-disposal teams could receive real-time advice, supporting data, and instructions from SMEs located off-site.
Operation readiness/efficiency: When a complex military system or expensive piece of equipment goes down, it could expose the associated operation to risk. For example, if a naval frigate’s infrared search-and-track system goes down, the vessel loses its main method of tracking incoming aircraft and ships. This loss weakens the operational readiness of the ship, its crew, and the fleet. AR/MR solutions can help in cases such as this by enabling rapid and timely temporary fixes until the ship is dockside for more extensive repair.
AR/MR has the ability to connect SMEs to in-the-field technicians when they encounter a complex or unfamiliar problem. SMEs can see what the technician sees and provide voice instructions, annotations, and holographic content to help with the task, depending on the solution. These solutions can ensure that the equipment is repaired quickly and correctly while simultaneously educating the technician. The Royal Canadian navy, Royal Canadian army, and Royal Canadian air force all use AR/MR solutions to identify applications in order to maintain a high level of operational readiness. (Figure 1.)

Figure 1 | AR/MR can connect subject-matter experts to field technicians who may encounter complex or unfamiliar repair or maintenance problems. Photo courtesy KognitivSpark/U.S. Air Force.

Training: Training, the final key application for AR/MR technology in defense, is a use case traditionally dominated by VR solutions. These technologies differ in that VR creates a virtual world in which users can perform a variety of tasks, depending on the software, while AR/MR solutions allow for a more classroom-based education experience. End users can wear spatial-computing headsets and view holograms while remaining in their real-world environment.
The Canadian army uses AR/MR technology for training scenarios on limited-availability equipment, specifically, a light armored vehicle (LAV). Multiple junior technicians are guided by an SME in a classroom. They all view a highly detailed, to-scale hologram of the LAV while the SME talks them though specific components of interest. Though the use of AR/MR, junior technicians can get a deep understanding of how the vehicle operates well before they ever set eyes on one. This scenario also reduces the burden on a limited training fleet and removes equipment availability as a constraint in training pipelines.

Security risks of AR for defense organizations

AR/MR solutions – while offering a unique set of benefits for organizations and end users alike – if not vetted properly can introduce risk. Before implementing these technologies, an organization must look critically at a solution, its limitations, vulnerabilities, available computing options, data-management policies, the organization’s own security protocols, and the environments in which the AR/MR solution will be tested or deployed.

There are two main factors that an organization should take into account before testing or deploying an AR/MR solution. These factors will dictate the level of security required to use AR/MR solutions.

Environment

The first is the environment in which the solution will be used. What are the security requirements of this location and do those requirements fall within the capabilities of the technology? The nature of AR/MR hardware requires the use of one or more cameras or spatial mapping sensors. Holographic data can’t be superimposed over a user’s field of view or respond to a user’s environment if the hardware can’t detect that environment through its own sensors. Considering that AR/MR solutions require the environment to be viewed by the hardware and then likely transmitted, data management and sovereignty can be a concern for defense organizations.

Frequently in the defense and aerospace sectors, environments (buildings, labs, training areas, etc.) feature tiered security. A visitor can’t walk into an equipment service bay without the proper clearance because the equipment and data in that environment may be beyond the scope of the visitor and endanger an operation or project. Data or content leaked from high-tier defense environments can be catastrophic for the organization or nation.

Naturally, this calls into question the viability of AR/MR technologies in these environments. Bringing a head-mounted spatial computing system that features cameras, spatial mapping sensors, and the ability to connect to internet networks could be problematic if the solution’s data-security layers, failsafes, and encryption infrastructure are flawed, weak, or nonexistent. There is no workaround for having cameras on-site if you’re looking to use AR/MR technologies. Therefore, it’s important to critically assess both the AR/MR use case (and its corresponding environment) as well as the solution’s data-security infrastructure. If an organization is looking to test AR/MR tech, considering the environment is paramount. The recommendation is to reduce risk in pilot programs initially; the lower the security requirements during the pilot phase, the better. This approach enables an organization to adapt their process to the new technology while monitoring the security of the solution, ensuring its suitability for higher-tier deployments.

Data management and data sovereignty

When considering the environment in which you wish to test or deploy AR/MR tech, it’s important to simultaneously analyze how the AR/MR service provider transmits, manages, and stores the collected data. This is the most crucial implementation step for defense organizations, as so many operations in defense hinge on cybersecurity and the protection of the organization’s data. As the AR/MR industry is still relatively new, every solution has a different security infrastructure and there is not yet an industry standard or certification for these solutions to operate in the defense space. The most important thing an organization can do to ensure the solution they select will comply with their security protocols is to explicitly understand their own requirements and vet each AR/MR software based on these requirements.

An important differentiator between AR/MR solutions is the difference between a cloud solution and an on-premise solution. Cloud solutions send data to the public cloud, which is in essence a network of global servers where data is received, processed, transmitted, and stored. Cloud solutions have touchpoints with the public internet. On-premise solutions act much the same, except the server is located on-site and has no touchpoints with the public internet. These solutions can be installed virtually anywhere and on all AR/MR devices to manage data locally. On-premise solutions have high inherent security unless they are physically tampered with.

For organizations that are concerned about data sovereignty, on-premise computing solutions are a good way to ensure data doesn’t leave the country or direct supervision of the organization. Not all AR/MR solution providers allow for on-premise computing; an organization should be sure to ask the AR/MR provider if they can provide this form of highly secure computing.

Security questions to ask before testing or deploying AR/MR

Organizations looking to use AR/MR technologies should ask AR/MR solution providers or their internal IT or cybersecurity teams the following questions before using these technologies on-site:

Environment

What are my organization’s security requirements when operating in the area in which I wish to conduct an AR/MR pilot project?
Are cameras (cellphone, physical camera, tablet) allowed in this environment?
What is the internet connectivity quality of this environment? Will the AR/MR solution function under this bandwidth quality?
Is there a lower-security area in which this pilot program can be conducted that will still produce quality data that can inform future deployments?
What information or equipment in this environment could expose our operations to risk and how can that risk be mitigated?

Data management

Considering the environment and data policies, will a cloud solution or on-premise solution be required? Can the solutions provider deliver on this requirement?
What level of data encryption does the solution provide?
In the case that a third party has gained access to data captured by the solution, does the provider offer tamper detection, firewall monitoring, and tamper-blocking infrastructure?
Can the solution provider access client-captured data and, if so, what does their internal security infrastructure look like?
Is video and audio data collected by the solution stored on servers after the solution has been used? Who has access to this data?
Does my solution require data sovereignty? Can the solution provider ensure data collected through the application stays in the organization’s country of operation?

Duncan McSporran is the cofounder, COO, and VP of Aerospace and Defense at Kognitiv Spark, an industrial mixed-reality solution provider. McSporran is a former military officer in the British Army, including seconded service with both the U.S. Army and Canadian armed forces.

Kognitiv Spark
www.kognitivspark.com