As shown by a recent study by the American Automobile Association, safety-critical automotive applications are not exactly free from software defects.
As shown by a recent study by the American Automobile Association, safety-critical automotive applications are not exactly free from software defects. The AAA reported that advanced driver assistance systems (ADAS) that automate steering and braking in a growing number of vehicles are not providing reliable safety benefits, and recorded disruptions and disengaged roughly every eight miles. See related article, How Safe is Safe Enough? Explore ADAS Benchmarking for Vehicle Safety.
Since ADAS systems consist of complex interconnected software components, often built by different vendors, extensive code testing must be performed during the various development phases to ensure they work as intended in real world driving conditions. One of the ways to eliminate software defects that can lead to injury, death, or expensive product recalls is by adopting Dev(Sec)Ops and CI/CD tooling.
DevOps is a set of practices that combines software development (Dev) and IT operations (Ops). It aims to shorten the systems development life cycle and provide continuous delivery with high software quality. Meanwhile, CI/CD tooling refers to the combined practices of continuous integration and continuous delivery or deployment. CI/CD bridges the gaps between development and operation activities and teams by enforcing automation in building, testing, and deployment of applications.
The biggest challenge to adopting this approach in safety critical software like automotive applications is the need to satisfy audits, validation and certification requirements. For example, embedded automotive applications such as ADAS must support standards including MISRA and ISO 26262 for safety and security. These standards provide guidance for defining automotive safety lifecycle, functional safety aspects of the entire development process, as well as requirements for validation and confirmation measures necessary to ensure that a sufficient and acceptable level of safety is being achieved.
Many Agile methods and DevOps processes and techniques aren’t defined in detail in these standards, allowing teams to develop code as they see fit. However, safety and security standards require a rigorous, well-defined process. While the use of Agile and DevOps techniques is not incompatible with achieving safety certification requirements, it does require software teams to define and document their DevOps tools, processes, and techniques in precise detail.
One important example of this is traceability. Proving requirements are satisfied with validation evidence is required to confirm system functionality and airworthiness. Any DevSecOps process would have to manage traceability precisely.
Applying a hybrid approach in V-model development
The traditional waterfall software development method implies that each major stage of the development life cycle be completed from start to finish before the next phase starts. In safety critical systems this is often characterized as the V-model.
The V-model is commonly used in safety critical software since it defines the dependencies of each phase and the traceability links needed from phase to phase (e.g. requirements drive system architecture, system testing validates requirements, etc.) This is seen as the antithesis of Agile but adopting a hybrid approach that leverages continuous processes such as CI/CD and Dev(Sec)Ops is still possible.
While the V-model might imply waterfall development, safety critical software standards like MISRA and ISO 26262 do not dictate any process. Therefore, they do not prohibit the use of Agile and iterative techniques. In fact, many development teams building software with high safety requirements have incorporated iterative and agile techniques in their process.
For example, many automotive systems have requirements defined up front as part of the request for proposal (RFP) process during vendor selection. It is also common for milestones to be established as part of large-scale projects where deliverables are well defined. In such cases, planning around these requirements and milestones is necessary.
So while these requirements feed the design and implementation phases they can still be iterative, agile processes. That’s because the software development method chosen during the design, implementation and testing of code is left up to the manufacturer. As long as it meets the basic criteria of good engineering practices with traceability, safe and secure practices, and reporting and documentation evidence for results (to name a few).
Agile and iterative methods can work well in this phase despite the entire lifecycle not necessarily working within the Agile framework. In fact, the approach leads to better results by shifting many important parts of development earlier such as testing.
Most developers would agree that the biggest issue with waterfall development is related to integration. Often called “Big Bang integration,” many undetected issues emerge during this phase which result in rework and bug fixes. One of the key benefits of CI/CD is reducing these Big Bang issues by integrating software much earlier — and continuously — than is typical in the waterfall process.
In conjunction with an iterative/agile process, CI/CD requires developers to create incremental functionality that is deployed and tested thoroughly before moving onto the next “chunk.” By implementing a full DevOps approach, many of the issues in the software deployment and release chain are detected earlier, in other words they “shift left” on the software development timeline.
Despite the fact that implementing CI/CD and DevOps is more challenging in embedded safety critical systems, more and more automotive projects are getting closer to full CI/CD and DevOps. Both for delivery to end customers for early validation of subsets of requirements and cybersecurity testing. While these deliveries seldom include a full drivable vehicle, they do include integrated subsystems that can be tested on test benches.
Static analysis and hybrid AppDev
Since static analysis tools are often used locally by developers at their desktop for real-time feedback and to analyze complete builds, they are designed to integrate well with software automation tool chains and development methodologies and processes. In addition, these tools are completely autonomous because they require no interaction with testers or developers. As a result, they can facilitate the implementation of hybrid iterative and agile development processes by checking for bugs and security vulnerabilities in code at any of the following stages:
Development: Finding security vulnerabilities as new code (or test cases) is written, even before it’s submitted to a build or software control system. Bugs can be presented immediately in a developer’s IDE just like a compiler warning, along with recommended corrective actions. In addition, these tools can help ensure secure coding standards are met by flagging any violations for investigation.
Test: This is self-explanatory.
Deploy: Analysis of binaries and libraries can detect the same types of vulnerabilities as source analysis. This will identify bugs introduced during building and deployment of deliverables.
Review: Static analysis findings can be used to investigate individual defect warnings and also perform higher-level assessments of application quality at each phase of the development lifecycle.
Although implementing a hybrid iterative and agile approach for embedded and safety critical software development poses some challenges, the payoffs can be significant in terms of product quality, security and safety, as well as speed. One way to automate validation and traceability requirements, especially in regulated industries, is through the use of static analysis tools for detecting code bugs as they occur.
一 Vince Arneja is Chief Product Officer for GrammaTech