Open-standard Acceleration APIs for Safety-critical Graphics, Vision, and Compute

Article By : Neil Trevett

The need to deploy accelerated processing, combined with the complexities of safety-critical certification, has created a confusing landscape of processors, accelerators, compilers, APIs, and libraries.

The need for graphics and compute acceleration in embedded markets is growing. Cameras and sensor arrays are increasingly central to many use cases in diverse industries, ranging from automotive to industrial, and are generating increasingly rich data streams that require sophisticated processing. At the same time, advanced user interfaces are being developed using high-quality 3D graphics and even augmented-reality technology. However, the need to deploy accelerated processing, combined with the complexities of safety-critical certification, has created a confusing landscape of processors, accelerators, compilers, APIs, and libraries. That has driven up integration costs for embedded accelerators, which in turn has constrained innovation and time-to-market efficiencies.

Open standards have an important role in helping hardware and software vendors navigate this complex technology environment. Acceleration standards for the embedded market can enable cross-platform software reusability, decouple software and hardware development for easier deployment and integration of new components, provide cross-generation reusability, and facilitate field upgradability. Such standards reduce costs, shorten time to market, and lower the barriers to using advanced techniques such as inferencing and vision acceleration in compelling real-world products.

Khronos acceleration standards

For more than 20 years, Khronos has been creating open, royalty-free API standards that enable software application libraries and engines to harness the power of silicon acceleration for demanding use cases, such as 3D graphics, augmented and virtual reality, parallel computation, vision processing, and inferencing.

Khronos’s open, royalty-free interoperability standards
Khronos’s open, royalty-free interoperability standards

There is increasing interest in parallel computation as the industry prepares for what many experts warn is the coming end of Moore’s Law. Parallel computation shifts the focus of acceleration from frequency scaling of single cores to processors wielding large arrays of cores that can be leveraged to accelerate data-parallel algorithms.

Broadly, Khronos acceleration standards for parallel computation can be divided into two groups. High-level programming frameworks, such as SYCL and OpenVX, focus on streamlined development with effective performance portability across multiple hardware architectures. By contrast, low-level APIs such as OpenCL and Vulkan provide direct, explicit access to hardware resources for maximum flexibility and control, using the SPIR-V intermediate representation for kernel and shader programs.

The Khronos family of compute acceleration standards
The Khronos family of compute acceleration standards

SYCL (pronounced “sickle”) enables code for heterogeneous processors to be written together with the host application code in a “single source” file using standard C++ for portable acceleration on a wide range of hardware. SYCL will typically compile the part of the C++ application to be offloaded onto an acceleration processor into a lower-level API, such as OpenCL.

OpenVX (VX stands for vision acceleration) provides a graph-level abstraction to construct imaging and inferencing functionality by connecting a set of functions, or “nodes.” The high level of abstraction enables silicon vendors to optimize their OpenVX drivers for efficient execution on almost any processor architecture. OpenVX graphs can combine vision-processing and neural-network nodes for global optimization and can directly ingest complete trained networks in Khronos’s NNEF format, which provides a stable specification with open-source converters from diverse neural-network frameworks.

Vulkan is the only open standard for high-efficiency, cross-platform access to modern GPUs. Vulkan can accelerate compute operations on any compatible GPU using compute shaders alongside graphics rendering. Among the benefits that Vulkan brings to developers of next-generation 3D graphics and compute platforms are simpler drivers; direct and explicit GPU control; multiple graphics, command, and DMA queues for multithreaded applications; and offline front-end compilers for multiple shading languages.

The history of Khronos safety-critical 3D APIs
The history of Khronos safety-critical 3D APIs

OpenCL is a low-level standard for cross-platform, parallel programming of diverse heterogeneous processors found in PCs, servers, mobile devices, and embedded devices, including GPUs, DSPs, and FPGAs. OpenCL provides C- and C++-based languages for programming kernel programs that can be compiled and executed in parallel across any processors in a system, with explicit control over which kernels are executed on which processors. OpenCL complements GPU-only APIs by providing a simpler programming model, relatively lightweight runtimes, increased language flexibility, and more tightly defined precision for compute operations.

Streamlining functional safety certification

Safety certification is essential in markets such as automotive, autonomy, medical imaging, transportation, industrial, energy, and avionics. Systems in which a compute or display subsystem failure would pose a significant safety risk require system-level functional safety certification and compliance with safety standards such as RTCA DO-178C Level A/EASA ED-12C Level A (avionics), ISO 26262 ASIL D (automotive), IEC 61508 (industrial), and IEC 62304 (medical), as well as adherence to guidelines from industry groups such as the Motor Industry Software Reliability Association (MISRA) and Automotive Open System Architecture (AUTOSAR).

The demand for advanced accelerated graphics and compute is growing in these markets, and the accelerators must be included in system-level safety certifications. The high cost of safety certification is motivating the development of open-standard acceleration APIs that streamline the system-level certification process. These approaches reduce the API surface area by eliminating functionality that is unnecessary for embedded markets, increase runtime determinism, and provide robust error handling.

Khronos has a 20-year history of adapting mainstream acceleration APIs for safety-critical markets, starting with OpenGL SC 1.0 in 2003. These initiatives leverage proven APIs that are supported by multiple shipping silicon implementations and that are already familiar to developers.

The recently announced Vulkan SC 1.0 streamlines the Vulkan SC 1.2 API and increases runtime determinism. Vulkan SC pipelines can be compiled offline and statically analyzed to understand data flow and memory usage. The memory needed for pipeline execution can then be reserved at device creation time as fixed-size pools, both to minimize overall memory usage and to avoid the need for runtime memory allocation. Vulkan SC enables system implementers deploying GPU-accelerated
graphics and compute to meet safety-critical obligations and provide certification evidence packages with reduced cost and effort. It can also be invaluable for real-time embedded applications, even if not formally safety-certified.

OpenVX’s safety-critical profile enables the rapid deployment of trained neural-network models, using OpenVX as an inferencing engine that is ideal for safety-critical markets. The OpenVX safety-critical profile minimizes runtime API surface area and runtime size by separating the development of OpenVX graphs from their execution. This is accomplished by enabling the runtime to ingest a binary graph definition that has been precompiled offline. The OpenVX specification has also been annotated with functional requirement tag numbers, and OpenVX headers are compliant with MISRA C guidelines.

Khronos recently formed the SYCL Safety-Critical Exploratory Forum to investigate industry requirements for a general parallel-programming API for accelerated compute using SYCL’s standard C++ single-source programming model in safety-critical markets. Hardware-centric API designs are increasingly converging with sophisticated compiler technology, bringing new levels of programmable flexibility and portability when tapping into silicon acceleration for compute acceleration. The SYCL SC Exploratory Forum is in the discovery phase to understand and build consensus on industry requirements before starting standardization efforts. Any company is invited to join the Exploratory Forum at no cost and with no IP licensing obligations.

Standards organization working together

There are many standards development organizations (SDOs) in the industry, each with its own area of expertise. It is therefore important that SDOs have productive liaison relationships, to avoid duplicated effort and to ensure that diverse standards from different organizations work well together.

Khronos recently announced a liaison with the AUTOSAR global development partnership of vehicle manufacturers, suppliers, service providers, and companies from the automotive electronics, semiconductor, and software industries. The cooperative agreement encourages a productive flow of information while respecting the confidentiality and IP frameworks of both organizations. It enables Khronos members to receive information and insights about AUTOSAR activities and vice versa. The end goal is for AUTOSAR use cases and requirements to guide and influence the evolution of Khronos standards and to enable the AUTOSAR platform to utilize those Khronos APIs effectively.

Khronos has also worked closely with the European Machine Vision Association (EMVA) over the past two years to establish the Khronos Camera Working Group, which is now building an open-standard API for controlling embedded cameras.

The need for an open-standard camera API
The need for an open-standard camera API

The development of sophisticated optical systems, image sensors, and vision processors is accelerating, and embedded systems often tightly integrate camera sensors with image, vision, and inferencing accelerators in self-contained systems. While the Khronos APIs outlined above can be used for vision and inferencing acceleration, there is no widely adopted, open-standard, cross-vendor API to ease the software integration of new camera subsystems. That lack has resulted in unnecessarily high integration costs for camera technologies.

The camera API that Khronos and EMVA members are designing will give applications, libraries, and frameworks explicit control over camera runtimes, through a precisely defined interface that will enable cross-vendor portability of application code and easier system integration of new cameras and sensors. The new API will also preserve application code across multiple generations of cameras and sensors and will enable sophisticated control over the generation of sensor streams to increase the effectiveness of downstream processing.

The scope of the Khronos camera API currently in design
The scope of the Khronos camera API currently in design

The increasing use of compute, vision, and inferencing acceleration in embedded products is resulting in significant new business opportunities, and Khronos is developing a growing family of open, royalty-free API standards to meet the rapidly evolving needs of embedded and safety-critical markets. Khronos is committed to providing a safe space for companies to collaborate on open standards that benefit their own businesses and the wider industry.

If your company would like a voice and a vote in any of these standardization activities, or if you wish to implement a Khronos standard on your silicon or use a Khronos standard in your system design, Khronos welcomes your participation.

This article was originally published on EE Times Europe.

Subscribe to Newsletter

Leave a comment