Certifying embedded COTS software for military systems

Story

March 13, 2020

Rich Jaenicke

Green Hills Software, Inc.

Commercial off-the-shelf (COTS) software rarely goes through any type of certification process with independent verification of functionality, API [application programming interface] compliance, or security, with the main exception being software targeted at applications that require safety certification. Part of the issue is that most software standards do not specify certification test suites or a formal certification process. This is starting to change, however, for some military embedded systems, driven partly by processes defined by the aviation community but that can be applied more generally beyond avionics.

Software certification can be beneficial even for applications where certification is not required for regulatory compliance. The most significant benefit may be the confidence that certification confers to OEMs, integrators, and end users that the software operates as expected. A close second could be avoiding deliverable disputes due to poorly defined requirements, given that obtaining certification is a very clear deliverable. Other benefits of certification include reducing the likelihood of a product recall by using an independently tested product and lessening legal liability by demonstrating that the company went the extra mile to use a certified product.

API conformance

One type of certification is API [application programming interface] conformance to a standard that includes validation test suites and a method of independent validation. An example of a standard with a test suite for a broadly used piece of software is POSIX certification by IEEE and The Open Group, where The Open Group is the certification authority. The POSIX certification test suites test both API conformance and functional testing.

A safety-specific example of a standard with a conformance test suite is ARINC 653, Avionics Application Software Standard Interface: ARINC 653 Part 1 defines a general-purpose API between the operating system and the application software that enables hosting multiple applications at different assurance levels on the same hardware. ARINC 653 Part 3 is the Conformity Test Specification, which includes substantial functional testing within an application as well as API conformance testing. The latest revisions of ARINC 653 Part 1 – which are of particular interest for systems that include multicore processors – include the requirement that software architectures need to make sure that an application with multiple threads can run across multiple processor cores in parallel.

For the broader market of military embedded systems, the FACE [Future Airborne Capability Environment] Technical Standard (Figure 1) is designed to make military computing operations more robust, interoperable, portable, and secure. The standard enables developers to create and deploy an extensive catalog of applications for use across the entire spectrum of military electronic systems through a common operating environment. The FACE Technical Standard includes conformance testing for each of the architectural segments: Operating System, I/O Services, Platform-Specific Services, Transport Services, and Portable Components.

Figure 1 | FACE Architectural Segments.

Although the FACE Technical Standard was conceived for airborne systems, the software standard is applicable to all environments. The Operating System Segment (OSS), for example, includes the definition of general-purpose profile as well as a safety profile and a security profile. Conformance to at least a defined subset of POSIX is a requirement of each OSS profile; the safety and security profiles also require ARINC 653 support. Other FACE segments are restricted to using only the APIs contained in the OSS profile selected. Conformance test suites exist for each profile in each segment, and the certification process includes verification testing by an independent verification authority followed by review of the test and verification results by a certification authority.

Safety-assurance certification

Software safety assurance usually is thought of in terms of specific industries that require a high degree of safety. From a general real-time systems perspective, software safety assurance implies that a system performs its intended function with the highest degree of determinism. Software safety assurance largely involves an enhanced product life cycle to discover and eliminate design errors and omissions. The level of scrutiny and documentation required is based on the predicted impact of a failure, often described in terms of injury or loss of life.

The international base standard for functional safety in electronic systems is IEC 61508, which applies to a broad set of industries. That standard – promulgated by the International Electrotechnical Commission – defines four safety integrity levels (SILs), with the strictest being SIL 1, with a probability of a dangerous failure being <10-5 per hour of continuous or frequent operation. As a means of assessing compliance with IEC 61508, the Conformity Assessment of Safety-related Systems (CASS) methodology can be used by external auditors for accredited certification. Safety standards for specific industries often are derivatives of IEC 61508, including automotive software (ISO 26262), rail software (IEC 62279), and nuclear power plants (IEC 61513).

One of the strictest applications for software safety assurance is flight safety, including DO-178C/ED-12C for airborne software and DO-278A/ED-109A for ground-based software, specifically communications, navigation, surveillance, and air traffic management (CNS/ATM). Software is developed and verified to a design assurance level (DAL) based on the effects of a failure condition, with DAL A “Catastrophic” being the highest, down to DAL D “Minor.” Applications that have no safety-assurance requirements are labeled DAL E. As the aviation certification authority in the U.S., the Federal Aviation Administration (FAA) approves systems based on design, test, and verification artifacts delivered by the system provider.

Possibly the biggest hurdle to achieving compliance at a high DAL is the shared resource contention that occurs in multicore processors. That contention can cause an application running on one processor core to interfere with a different application running on another core, negatively affecting determinism, quality of service, and – ultimately – safety. Directly addressing the issue of multicore interference, the Certification Authority Software Team (CAST) has published guidance for multicore systems in a position paper called CAST-32A. CAST-32A includes 10 objectives that need to be satisfied to address the concerns with the use of multicore processors.

Mitigating multicore interference is also a prerequisite for using multicore processors in integrated modular avionics (IMA). One of the key characteristics of an IMA application, as specified in DO-297, is that applications are independently modifiable. Without a general solution for mitigating multicore interference, interference from a modified application could cause an existing application not to execute correctly or vice versa.

Security certification a huge concern

Security has become a more significant concern in almost every military electronics system. Although each military program has its own set of security requirements, software certified to a standard protection profile has a huge advantage. The international standard for computer hardware and software security assurance is the “Common Criteria for Information Technology Security Evaluation” (ISO/IEC 15408). Evaluations can be done to different levels of depth and rigor, called Evaluation Assurance Levels (Figure 2), with EAL1 being the least rigorous and EAL7 being the most rigorous.

Figure 2 | Evaluation Assurance Levels.

The framework for Common Criteria is very generic, allowing suppliers to define their own security requirements for the evaluation. The primary value of Common Criteria comes when the evaluation is done against a government-defined protection profile. In the U.S., the National Information Assurance Partnership (NIAP) defines protection profiles and manages the Common Criteria Evaluation and Validation Scheme (CCEVS) validation body. When the evaluation is at EAL5 or higher, the National Security Agency (NSA) participates in the evaluation.

The level of security does not come directly from the evaluation assurance level, but rather from the security requirements in the protection profile. It is only when a high EAL is achieved with a very demanding protection profile that the best security is achieved.

Some examples of NIAP protection profiles for software are

Protection Profile for General Purpose Operating Systems (GPOS PP): This PP was not designed for a specific EAL, but is based mostly on EAL2 requirements.
Protection Profile for Separation Kernels in Environments Requiring High Robustness (SKPP): This protection profile was designed for EAL6 plus additional requirements (EAL6+).

The aviation industry now has security guidance as well: As a complement to DO-178C, there is a set of high-level specifications for airborne security, including DO-326A, DO-355, and DO-356. Those specifications contain security objectives at the system and aircraft level. Any new commercial aircraft system fielded must address the DO-326A requirements.

DO-326A does not specify how to implement the required security objectives; it only provides guidance on the process to identify threat vectors and make sure adequate mitigation measures are in place. Processes in DO-326A parallel DO-178C, such as requiring a plan for security aspects of certification (PSecAC) similar to the plan for software aspects of certification (PSAC).

Software products that are certified to Common Criteria at EAL5 or higher have a head start on meeting DO-326A because there is a significant overlap in the processes. The SKPP, in particular, has much more stringent security requirements and testing than is required for DO-326A.

Putting it all together

The certification process can be difficult and time-consuming, particularly if the product was not designed holistically from the start for the capabilities being tested. An example of a software product simultaneously designed to meet all requirements for API conformance, safety certification, and security certification is the INTEGRITY-178 tuMP real-time operating system (RTOS). INTEGRITY-178 tuMP was the first product to be certified to the latest FACE Technical Standard, edition 3.0. That OSS certification was for safety and security profiles, which include both POSIX and ARINC 653 conformance testing.

The INTEGRITY-178 tuMP RTOS enables a general solution to multicore interference mitigation, which can help a system integrator in meeting CAST-32A and DO-297. That solution uses DAL A mechanisms to monitor and enforce bandwidth allocations from each processor core to shared resources.

On the security side, the INTEGRITY-178 operating system is certified to the NSA-defined Separation Kernel Protection Profile (SKPP) for High Robustness and the only operating system ever certified to Common Criteria EAL6 or higher. Extending that design to multicore processors, INTEGRITY-178 tuMP continues to meet the SKPP’s rigorous set of functional and assurance requirements. Those safety and security assurance certifications are proven in real-world customer applications with over 80 DO-178B/C Level A/EAL 6+ unique customer-certification packages delivered across more than 30 different microprocessors.

Richard Jaenicke is director of marketing for safety and security-critical products at Green Hills Software. Prior to Green Hills, he served as director of strategic marketing and alliances at Mercury Systems, and held marketing and technology positions at XCube, EMC, and AMD. Rich earned an MS in computer systems engineering from Rensselaer Polytechnic Institute and a BA in computer science from Dartmouth College.

Green Hills Software
www.ghs.com