Diminutive AI-powered drones outwit threats

Story

June 09, 2023

Robotic vehicles play increasingly important roles on the tactical battlefield. In addition to larger unmanned aerial and ground vehicles (UxVs) carrying armaments for open warfare, smaller man-portable vehicles are now being used for intelligence, surveillance, and reconnaissance (ISR) missions. With the addition of the onboard artificial intelligence (AI) needed to overcome the many threats thrown their way – both physical and electromagnetic (EM) – human-machine teaming is rapidly becoming a reality.

Tactical unmanned aerial systems (UASs) must work in hostile environments and must work to detect and avoid threats while operating without GPS. Fortunately, advances in artificial intelligence (AI) hardware and software are making possible smaller nano drones with greater intelligence. Compact AI technology and supporting software is providing the capabilities needed to enable nano UASs as well as larger UxVs [unmanned aerial and ground vehicles] to overcome multiple threats on the battlefield from safe standoff distances. Systems are able to become smarter with each mission and to share these learned behaviors with one another, creating more efficient and capable platforms. AI-enabled UxVs are useful for many nonmilitary applications as well, enabling first responders, police, and firefighting professionals to help others while reducing risk to themselves.

Out of necessity, nano UASs carry smaller payloads but cannot afford to sacrifice key capabilities in tactical situations. The ability to communicate processed sensor data on-device, for example real-time video object detection made possible by AI and machine learning (ML), cannot be compromised due to a smaller airframe. In intelligence, surveillance, and reconnaissance (ISR) applications, where intelligence provides an advantage, gaining as much edge AI capability as possible is a benefit and can mean life-or-death outcomes.

Even a smaller UAS, when enabled with a powerful AI central processing unit (CPU) capable of running advanced AI and ML algorithms, can act as an extremely sensitive measurement probe within hostile operating environments. Adequate AI and ML processing power enable simultaneous analysis of multiple video and sensor inputs, such as potentially hazardous chemical mists or classification of visible and thermal images of unknown objects, by comparing them to known references stored locally in computer memory.

As miniature UASs are being developed for clandestine ISR, electronic warfare (EW), and other tactical operations, common component use simplifies deployment while AI capabilities enable navigating in hostile operating environments; one example is a situation in which GPS satellite signals are jammed or unavailable. A nano-sized AI-enabled UAS can now operate reliably inside buildings as well as outdoors. Within a building, these small craft face the challenges of no GPS, unknown and unstructured environments, multiple threats, and complex communication due to reflections and jammers. The ability to carry a full sensor suite, including optical and thermal cameras and onboard machine vision (MV) software, can provide dependable unpiloted navigation through an unknown building while providing a real-time map for others to follow. Multiple sensors drawing upon the assistance of AI algorithms for navigation and direction, using techniques such as simultaneous localization and mapping (SLAM), is used by the UAS to build a map and localize itself in that map at the same time. SLAM algorithms enable the vehicle to map out unknown environments and carry out tasks such as path planning and obstacle avoidance.

Ever-shrinking systems

For years, military UASs have been evolving towards smaller electric-powered systems that can be carried by a single soldier. In addition to fixed-wing aircraft operating from a landing strip, portable rotary-wing drones that can be folded have been developed for ease of transport. Modern vertical takeoff and landing (VTOL) aircraft – including rotary-wing aircraft, fixed-wing aircraft, and hybrid combinations of the two aircraft structures – do not require runways for takeoff or landing operations.

Tactical AI-enabled portable VTOLs and cargo-pocket-sized UASs are often referred to as nano drones and can now be considered for in-building as well as outdoor surveillance and full ISR missions. Payload electronics miniaturization enables nano UASs to perform specialized tactical sensing missions without detection. They can operate with almost imperceptible visual and audio signatures for secretive missions in buildings, confined areas, and outdoor urban environments. Depending upon the mission, the payload capabilities may match those of larger UASs used for similar missions, but from hardware a fraction of the size of the larger UASs. At present, smaller UASs and their ground-control systems (GCSs) are being designed to fit into cargo-pocket-sized cases that can be transported to a battlefield (or field of interest) on board-standard all-terrain tactical vehicles or by dismounted soldiers.

When flying a tactical payload into a battlespace or contested airspace of any kind, smaller drones must achieve functions essential to all tactical UASs, such as real-time communication of sensor data from the UAS to GCSs and authorized recipients. UxVs can now be controlled by compact GCSs such as a body-worn end user device or as large as a multiuser command post system; in any case, the need for reliable, instantaneous information from the UxV is critical in all cases.

Operating conditions may vary, but most UxVs must communicate wireless video and data from onboard cameras and other sensors. Latency is critical to be effective in keeping UxV operators and troops out of danger. Operators remain at a safe standoff distance from a few meters to more than 10 km (6.2 miles). Nano UAS endurance is determined by the nature of the mission as well as AI loading and the number of onboard electronic devices drawing power from the rechargeable battery cells. Minimizing size, weight, and power (SWaP) of all onboard components is especially critical in small and nano UAS systems and imposes limits on the size, power consumption, and capabilities of the UAS’s payload components. As part of flight navigation, for example, collision avoidance can be achieved through AI-driven time of flight (TOF) cameras, light detection and ranging (lidar), and sonar systems. For counter-UAS purposes, image resolution for detection and recognition must be of sufficient quality to differentiate a troop’s drones from an adversary’s UASs.

With sufficient onboard computer processing power, the AI process of comparing images of detected objects to known references can take place on device. When desirable, AI enabled UASs can orchestrate and coordinate movement between themselves in drone swarms to achieve a result. The use of a modular open systems approach (MOSA) simplifies interoperability while common interfaces enable payloads to be utilized across a wide range of platforms.

Strong networking capabilities in hostile radio-frequency environments is critical to AI-enabled UxVs being able to coordinate with one another and share real-time video and data with troops that depend on it. Radio electronics miniaturization enables mobile ad hoc network (MANET) communication systems to be incorporated on nano UASs, expanding the “edge” of the network “cloud.” Software-defined radio (SDR) technology provides the radio communications flexibility to adapt to jammers and changing communications environments. Coupled with advanced antenna technologies – including low-power, electronically steered and mesh antennas – multiple-input, multiple-output (MIMO) SDRs enable reliable control and monitoring of even large swarms of nano drones under hostile conditions.

Real-time solutions

Compact integration is one key to fitting required AI and ML processing power, servo controllers, sensors, and communication systems into the smaller spaces of shrinking UASs. The Artificial Intelligence Mission (AIM) system (AIMind) robotic control unit for UxVs including nano drones is the result of densely packed, vibration isolated components while paying close attention to the placement of active devices to minimize power consumption and thermal footprint but to also limit electromagnetic interference (EMI).

The AIM system is comprised of VOLT, a lightweight power “wedge,” GCS, a body-worn end-user device like the Samsung Galaxy S20 TE; and the wearable AITN module (Figure 1), a tactical radio with the AIMind computer. UxVs equipped with the 89 g (approximately 3 ounce) AIMind robotics control unit are controlled by GCS running the Android Tactical Assault Kit (ATAK) and the ATAK UAS Tool plugin. With the AIMind nano UAS, AIM weighs less than 2 pounds and can easily be carried with other gear. By utilizing native ATAK UAS control software, live-streaming video and data is available across the secure ATAK network.

[Figure 1 ǀ This module packs extensive AI/ML processing power onto a single PCB along with multiple radio options.]

AIMind UxVs with AI/ML technology and control software can be used to create indoor and outdoor UxV swarming missions. Payload sensor data is processed on-device by AI/ML algorithms with or without GPS or connectivity to ground control. Modular payloads are designed to fit the smallest tactical drones and include scanning lidar, 12-channel chemical detection, ultrasonic sensors, and EO/IR [electro-optical/infrared] cameras. The payload gains impressive control and analysis capabilities from the on-device robotic control unit. The software provides fully integrated autonomous UAS navigation using multiple sensors for AI-enhanced object detection and classification. Working with integrated SDR radios, enabled UxVs coordinate using MANET communications at data rates to 92 Mb/sec over distances from 2 km to 10 km (1.24 miles to 6.2 miles).

UxVs equipped with AI technology can perform precise mapping and guidance through threat-filled indoor environments even where GPS signals are not available and enable the application of visual-augmentation system software (VAS) to manage virtual- and augmented-reality (VR/AR) algorithms. Navigation without light and visible cameras is possible using 40 KHz ultrasonic pulses and TOF lidar to create a 3D point cloud that represents the flight environment. With extensive onboard signal processing, there is no need to send data back and forth between a remote computer controller and a drone to achieve accurate object recognition; it can be executed onboard the UAS. With the signal-processing and AI capabilities handled by software, smaller UASs can provide reliable surveillance in both indoor and outdoor environments.

Such a modular approach to adding and integrating AI/ML computing to a UAS provides flexibility and adaptability as new, smarter UxVs are developed. A modular UxV design approach simplifies the task of creating mission-specific platforms for each mission.

For example, the AIMind control unit with as many as eight cameras, lidar, audio microphones, and chemical sensors, along with the onboard AI/ML processing power can be rapidly integrated onto any PX4 [open-source autopilot software] UxV frame, whether aerial, ground, or surface of submersible. For example, flight times of more than three hours are possible with this system on an electric VTOL by careful choice of efficient wing design, rechargeable battery size, and number of sensor and other modules.

Of course, even a small UAS with the densest, most advanced AI computing engine requires consistent communications links to command stations to share valuable ISR data collected. Whether for control of single drone or in swarms, the radio links must often overcome hostile adversaries and environmental conditions, even the movement of the troops and the command post as the UASs are collecting data. Miniature concealment antennas (such as those developed by Southwest Antennas, https://www.southwestantennas.com/) when onboard UASs and integrated as part of body-worn communication equipment have proven capable of maintaining MANET communications even when faced with demanding electrical, mechanical, and environmental requirements (Figure 2).

[Figure 2 ǀ Concealment antennas such as the shown dual-band MIMO unit form part of the communications links for smaller tactical drones in the field.]

Such dual-band antennas are needed for transmission and reception between drones in a swarm as well as from a lead drone to body-worn communications equipment. An antenna built for MIMO use with vertical polarization from 2.1 to 2.5 GHz and 4.4 to 5.0 GHz frequency bands is such a candidate since it measures just 1.9 by 0.75 by 0.15 inches but can operate effectively even when closely mounted to a soldier’s body. The antennas can also be used in a slant-left/slant-right configuration if mounted at 45-degree angles inside the drone. Two would be required in opposite-slant orientations to get 2 by 2 MIMO performance. These antennas have a unique onboard adjustable matching circuit, enabling adjustment of the antenna feed point tuning for custom application needs and integration into a wide variety of enclosures of different sizes and configurations. Its rugged enclosure even makes it waterproof for most practical applications.

AI is making possible reduced-SWaP for small UxVs with their functional diversity and high levels of integration. At the same time, MOSA and PX4 modularity is enabling the rapid upgrade of mission-specific UxVs with increased AI and ML capabilities. The result: Robotic teammates that help their human partners maintain safe standoff distances from danger while collecting as much sensory data as possible for successful missions.

Seth Spiller is president and CEO at Orion Technology Group. He has extensive engineering and international business development leadership experience working with artificial intelligence (AI) and autonomous robotics platforms. He is the co-inventor and holder of the first AI patent in 1997, technology that has been leveraged worldwide and successfully integrated in more than 30% of the Fortune 100 companies. His career experience includes engineering, operations, and LEAN manufacturing with leading global technology and defense companies. Spiller attended the University of Maine where he earned a BS in electrical engineering.

Orion Technology Group     https://oriontechnologygroup.com/