Gaming tech: Shaping the reality of military training

Story

September 28, 2017

The U.S. military leadership still pushes the mantra “train like you fight,” but the technology necessary to meet that goal is getting more sophisticated and more dependent on their relationship with the commercial gaming world and concepts such as virtual reality (VR), augmented reality (AR), and mixed reality. This has become especially beneficial as many young recruits are digital natives and find this simulation technology intuitive.

Technologies like AR, VR, and mixed reality in military training today are not uncommon as “training and education theories evolve over the years,” says Tony Prause, Portfolio Manager for the Products Life Cycle Support portfolio at Engility Corp.’s Chantilly, Virginia location.

More realistic simulation technology also enables more remote training and less use of expensive live platforms, thereby saving the DoD millions of dollars.

“What we’re seeing around the world are two things: The first one is an ever-tightening of military budgets, so that’s basically universal, and the second one is a reduction in the amount of real-world space where militaries can conduct training exercises,” says Pete Morrison, co-CEO of Bohemia Interactive Simulations (BISim) in Winter Park, Florida. “Both of these facts, or both of these items, will lead military organizations toward using simulation for training.”

“The military faces many challenges as it relates to training,” states Andre Balta, chief technology officer of Cubic Global Defense in Orlando, Florida. They continue to focus on increasing throughput while increasing the quality of training – whether through point of need training (distributed training) or accelerating learning through learning science, he adds.

Much of their current training leverages similar technology to the games they played in high school, improving skills, ranging from complex decision-making to simple daily maintenance tasks.

Even as many parties agree on the usefulness of VR and AR in military training, there are not yet requirements in place to facilitate companies’ proposing and fulfillment of such programs. “It’s a challenge to identify what the training requirements are today, but even harder to do so in five years because you don’t know what will change,” Prause notes.

Despite the lack of requirements, commercial gaming technology has a foothold in military training. The recruit’s go-to forms of entertainment are often video games, as they grew up with games like Call of Duty. “Today’s sailors expect that same immersive training environment that not only includes the technical aspect – such as the systems the military uses – but also the scenario storyline,” explains Eric Phipps, program manager, U.S. Navy training programs at Engility Corp.’s San Diego facility.

It’s no surprise that “The entertainment industry, specifically around computer games, is huge,” Morrison says. “The latest estimate is that by the end of this year [2017], computer games will be a $100 billion industry.”

The military-simulation industry is small compared to that, and “what we’ve seen over the last couple of years is that the technology in the entertainment domain has really outpaced the technology, especially in terms of graphics and physics, in the military-simulation world,” he adds.

Military simulation leverages gaming concepts

Increasingly, gaming community advances are driving military-simulation designs. “More and more companies are leveraging the cultural and process elements that mainstream game studios use to create high-fidelity products at rapid rates,” Balta says. “These ‘cultural’ changes contribute significantly to the evolution of the training industry. These game engines – advanced, photorealistic, and cost-effective – also come with tool suites to generate content easily and readily. The defense industry benefits from all the best practices documented from the gaming industry.”

The content is just as important as the technology, but it’s more “of a holistic approach,” Prause says. “For example, an Electrician’s Mate (EM) or Electronics Technician (ET) in the Navy has to go through ‘A school’ and subsequently through different types of schools, including on-the-job training. Each of those different training scenarios are system-specific schools – and you have to take all that into account to build through that whole training life cycle,” Prause says. “That’s a challenge, but you have to take that bigger picture into account.”

The entire solution set needs to be realistic enough for the military. “From an Engility standpoint, we have to create a full backstory for every training scenario,” Phipps says. “During the crisis or scenario, orders have to be realistic. Coalition forces, geography, geo-political boundaries, all of that has to be developed to provide the fidelity that sailors expect.”

Mixing reality with a game

VR and AR are all part of the game with mixed reality sitting on the cutting edge. Morrison adds. “Mixed reality is where you have a headset with video cameras, which captures the real world, but we inject virtual elements.

“A pilot can sit in a mock-up of a cockpit, but that cockpit is created in the real world with all the dials and switches,” Morrison explains. “But then when they look out the window through their headset, they’re seeing the virtual world, which is generated by our software. This is called mixed reality, and the next frontier we’re working on with the U.S. Navy.”

The idea that the user’s mind must create a fake reality, yet can feel, see, and touch the tools and systems make this type of training more effective – and it’s simply mind-blowing.

This provides an ability to have a mixed reality solution, enabling interaction with the real world, vis-à-vis the cockpit controls or anything else an operator may want to do, explains Nick Gibbs, vice president and general manager, training solutions, for Rockwell Collins in Sterling, Virginia. “You could fire a weapon, it would be a real weapon and you could load and unload it while seeing your hands performing the task. You don’t need any simulated behaviors for that.”

Live, virtual, constructive (LVC) training – which integrates live and virtual constructs – also enables the military to bring together different domains from air to sea to land.

“An LVC environment is very scalable and repeatable. That’s another benefit the Navy wants take advantage of,” Phipps says. “They can train a single ship, crew, or aircraft, or scale up to just about any number of ships, crew, aircraft, and other forces. And they can repeat the training as necessary, ensuring that the objectives and learning points have been achieved.”

Eventually, the end result will align with the Navy’s vision, which is “to eventually take the synthetic environment and blend it with a LVC environment that will be seamless between real and constructive forces,” Phipps says.

Efficiency versus effectiveness

While adoption of a gaming-style type of training for today’s digital natives sounds natural for the military concerns over effectiveness remain. “Now that you’ve given the training, how do you determine how successfully that individual is now trained, or that collections of individuals are now trained, in the task that they were supposed to obtain from the course or from the simulations?” Gibbs says.

“Right now, there’s an effectiveness and an efficiency argument,” Morrison states. “When we talk about effectiveness, we’ve proven that doing simulated exercises, and it doesn’t matter what the domain is whether it’s land, sea, or air, if you do a task in simulation before you do it in the real world, and that simulation is suitable for the training task of course, then you’re going to essentially decrease the failure rates in the real world. You’re going to be better at that task.”

The mantra with this training is: practice makes perfect. “Our software gives them an opportunity to practice how to think,” Morrison says. “They can operate collectively in the virtual environment and practice the things that they need to do in the real world. Through practice in the virtual environment, it reduces the amount of training that they have to do live, and it also can help save their lives in theater because they’ve done it many more times in simulation that they would’ve gotten to do it live.”

In addition to practicing, AR and VR “increases the effectiveness, in particular the decision-making process,” Prause says. “It reduces the cost of training and becomes very attractive for users, not just for the cost, but also because the access to the training is easier.”

The aftereffects of gaming tech in military training

Gaming technology is attempting to close the gap in the training deficiencies that the military is currently experiencing. For example, the U.S. Navy’s Littoral Combat Ship (LCS) engineering casualties in late 2015 through 2016 forced military leadership to emphasize training.

In answer to that situation, “Cubic’s Immersive Virtual Shipboard Environment (IVSE) developed for the U.S. Navy’s Littoral Combat Ship is a game-based learning continuum using Unreal 4 Engine,” Balta explains. “It is designed to meet the Navy’s objective of ‘Train to Qualify’ (T2Q) and ‘Train to Certify’ (T2C). T2Q is the standard set as an individual measure of a sailor’s proficiency to ‘stand the watch’ as soon as he/she reports aboard their ship, while T2C is a similar measure of proficiency for the ‘watch team.’”

By using gaming technology “students are immersed in a high-fidelity 3-D replica of the actual ship within a virtual environment that contains all the spaces, compartments, systems, equipment, technical documents, instructions and tools necessary to qualify a student for their prospective watch station,” Balta says. (See Figure 1 video/image).

(Click graphic to zoom)

 

“Training is conducted in the controlled virtual environment, enabling multiple students to learn simultaneously, independently, or in teams,” Balta adds. “Training in the IVSE replicates on-the-job training just as sailors learn on board current ships of the fleet, without the ‘underway’ challenges of limited time and inconsistencies as sailors learn from various sources and senior personnel.”

The use cases are not hard to find. “Since 2004, Games for Training has been a program of record within the U.S. Army, so that’s essentially validation that there is forward recognition that gaming technology works, and that’s an investment by the U.S. Army in gaming technology. probably to the tune of maybe $50 to $100 million dollars a year, so there’s anecdotal evidence that given the level of investment, the U.S. military believes that it works,” Morrison points out.

“We’ve had a case in the U.K. where, a soldier credited game-based training for saving one of his soldier’s lives in Afghanistan,” He adds. “They had a vehicle rollover event, and because they’d rehearsed that so many times in simulation, they knew exactly what to do in the real world when it happened.”

From sea to land to air, “a great example is the joint-strike fighter, where it’s a single-seat plane,” Rockwell Collins’ Gibbs says. “There is no second seat version for training. All of the pilot’s training before first flight takes place in the simulator. We provide the helmet-worn display, the entire visual system along to Lockheed Martin for that training device.”

	(Click graphic to zoom)

 

Rockwell Collins’ high-fidelity training system, the EP 8100, has “FPGA [field-programmable gate array] technology, so it provides even more enhanced capability,” Gibbs adds. (Figure 2.)

The beauty of this type of training is that “users can create whatever scenario they need to train to in an immersive training environment,” Phipps says. “For example, users can’t always train in ballistic missile defense (BMD) against long-range high-altitude missiles in real life. It is expensive and there’s a high-risk factor involved with this scenario. There is also a decision-training matrix involved including the ships being trained, the authorities in charge, and an intelligence background needed. A synthetic background is ideally suited for this. You can network multiple ships and train for a BMD scenario without the restrictions of battlespace.”

Figure 1: Cubic’s Immersive Virtual Shipboard Environment (IVSE) developed for the U.S. Navy’s Littoral Combat Ship. Video/image courtesy of Cubic Global Defense. Click here to view video.

Figure 2:Imaging generated with Rockwell Collins EP 8100 used for the Apache helicopter visual system. Photo courtesy of Rockwell Collins.