Static analysis improves efficiency, reduces downstream integration costs
StoryApril 26, 2010
Andy Chou
Coverity
With modern military systems increasingly relying on software, new techniques are being adopted to decrease costs and increase the chances for mission success. As a result, static analysis has been gaining traction in the software development community based on its deep analysis capabilities before runtime.
These static analysis tools - which augment but do not replace traditional test and debug methods - find integration errors long before integration occurs, eliminating costly late-stage integration issues.
As military equipment and vehicles become increasingly modernized, they inevitably become more technologically complicated as well. In many instances, these machines represent a delicate blend of hardware and software, both of which must interact perfectly. Since the software components of these machines must work as reliably as possible in the field, it is important to perform extensive debugging and testing using a combination of techniques to eliminate defects before they cause extended delays or overruns. This is often impractical since there is usually no way to reliably test code until it is actually implemented into the equipment on which it will eventually run. Since software seldom works perfectly the first time it is executed, going back and fixing errors with conventional development methods and tools hinders productivity and diverts valuable resources and personnel from other projects that need to be completed.
For instance, the Boeing 787 Dreamliner was delayed for two years because of a combination of hardware and software defects. Often, such problems are intertwined; in the case of the Dreamliner, one particular delay was the result of defects in the software that controlled the braking system. It is important to note that conventional testing did not reveal this defect before it actually became a problem and caused costly setbacks and other complications in the development process. Modern techniques like static analysis can increase the probability of problems like memory corruption and user-after-free being discovered sooner, as well as assist in achieving DO-178B’s Design Assurance Levels (DALs). A more efficient and cost-effective path than the traditional V-model, static analysis works in concert with traditional test and debug methods to ease integration woes and expense.
Static analysis and the development process
Software development tends to follow a specific life cycle. One example is the V-model (Figure 1), which is commonly used in aerospace engineering. The V-model represents a holistic approach to development that attempts to reconcile project definition and the testing process over a period of time. The model begins with establishing the scope of a project, including its concepts of operation, its requirements and architecture, and the specific details of its design.
Figure 1: The V-model represents a holistic approach to development that attempts to reconcile project definition and the testing process over a period of time.
(Click graphic to zoom by 1.2x)
|
|
This process starts out very abstract at higher levels and becomes gradually refined and detailed over the course of the design process. As the design is implemented, it becomes more expensive to fix problems later in the development cycle. Once the project is mostly complete and is being tested, going back and making repairs to its fundamental aspects becomes prohibitively expensive.
Ultimately, many software problems are caused and then worsened by an inefficient development model and imprecise debugging procedures. Development efficiency and cost-effectiveness can be improved by eliminating software defects as early as possible with modern techniques like static analysis.
Specifically, static analysis is a technique for finding defects in software without running it. It works by examining product source code, starting from individual functions, working its way up to modules, then ultimately the entire program. Static analysis can find many different kinds of defects, including memory errors in C/C++ programs. For example, static analysis can detect an error in this simple code fragment:
int a[10];
for(int i = 0; i < 10; i++); {
a[i] = 0;
}
Sometimes it is hard for human beings to see problems in software code because they see what they want to see instead of what is actually there. When this code is turned into an executable program by a compiler, the compiler reads the source code in a mechanical and precise way, ignoring human cues like indentation and spacing. A compiler will read the example in the following way:
int a[10];
for(int i = 0; i < 10; i++)
;
{
a[i] = 0;
}
Once the code has been reformatted as shown, an extra “;” character that causes the program to have a completely different meaning suddenly becomes much more visible. If this error remains uncorrected in the final program, the array access a[i] will execute only once when i = 10. The result is an assignment to a memory location past the end of the array, which might cause the program to crash.
Static analysis examines the code as mechanically and precisely as a compiler would. However, instead of blindly translating the code into an executable program, a static analyzer looks for paths where the code’s function diverges from how the developer originally intended it to work. Static analysis can be done by techniques as simple as pattern matching or as advanced as interprocedural dataflow analysis and Boolean satisfiability. Regardless of technique, static analysis is custom-built to look for cases that human developers are likely to overlook or get wrong. This presents a far more efficient alternative to comprehensive line-by-line code auditing, which is not cost-effective for larger software systems.
Combine static analysis and traditional debugging
Compared with a traditional method such as functional testing, static analysis presents a different set of trade-offs. Traditional testing can only detect errors in code that is actually tested, whereas static analysis can find defects in all code without any tests. Sometimes, errors found through testing are hard to reproduce and pinpoint to a specific problem in the source code. Static analysis can find problems in a repeatable, predictable fashion and will always point to a specific location in the code. On the other hand, traditional testing can find functionality errors that static analysis cannot, because static analysis does not attempt to compare the behavior of the program against an expected result. Static analysis can also lose precision when analyzing deep program properties, so it might miss some defects. Therefore, static analysis is meant to augment the effectiveness of traditional methods instead of replacing them outright.
Since static analysis works with existing toolsets and compilers, there is no need to change current development practices. Static analysis can be applied frequently (even on nightly builds) for the duration of the project starting immediately after the first line of code has been written. Essentially, if the project’s code base can be successfully compiled, static analysis can be used to debug it and eliminate problems long before the project is delivered to quality assurance personnel for final testing.
Integration benefits of static analysis
Static analysis also works well in a team-oriented environment. Developers can use static analysis to check each other’s contributions for consistency and check for conflicts caused by code written by different team members. This can work within a single project or even scale to larger scenarios where integration between several projects is required to create a complicated system.
Static analysis can also do more than find simple code defects; as mentioned, it is capable of analyzing the interactions between different components before they are integrated together. In the V-model of software development, testing and verification start with individual components. When these components meet their low-level specifications, they are integrated together for higher-level testing against system requirements. If software integration is performed strictly according to this ideal, the integration phase introduces a high degree of risk because the individual components will be interacting for the first time. These interactions are likely to expose problems with the original project specifications, particularly regarding how the system requirements and architecture are translated into detailed design requirements and source code. Static analysis tools can analyze the interactions between software components as soon as they are written, catching some of these expensive integration problems during the implementation phase. The tools accomplish this by finding problems in Application Programming Interface (API) usage by analyzing code across procedure boundaries even if the procedures are in different software components.
Maximizing efficiency with static analysis
Software is being used to improve military systems such as smart bombs and unmanned drones in ways that were once impossible. Static analysis tools, such as those offered by Coverity, are also a part of this trend. Static analysis tools – when combined with traditional test and debug methods – effectively analyze software and detect code errors before runtime, minimizing risks and costs while maximizing the value of investment in software development.
Andy Chou is chief scientist and cofounder of Coverity. He is responsible for advancing source code analysis technology as well as furthering the state-of-the-art in software quality and security industry-wide. He received his Ph.D. in Computer Science from Stanford University and his B.S. in Electrical Engineering from UC Berkeley. He can be contacted at [email protected].
Coverity 415-321-5200 www.coverity.com
