RISC-V Forum: Vector & Machine Learning

In this presentation, Marc and Leonidas, talk about a novel approach in the acceleration of ML operations in area and power-constrained RISC-V processors. As an alternative to conventional approaches which build on the V-extension and require an additional area-hungry vector register file and the implementation of several instructions for compliance, they propose an open source short SIMD module for Machine Learning acceleration which reuses the integer register file, resulting in considerable area savings. The implementation complexity is minimal since it only requires 17 new highly configurable custom instructions which combine classic SIMD operations with subsequent reduction operations over a two stage design similar to a MAC unit, which can implement 200 different combinations of commonly used ML operations. The design has been driven by the analysis of the most common operations found in ML applications. For increased versatility, the instructions have been complemented with optional saturation and GPU-like features like swizzling and masking, which in conjunction with the register file reuse, remove the need of custom load store instructions. The module is portable among RISC-V processors and has been demonstrated on the open source RISC-V space processor NOEL-V, distributed by Cobham Gaisler. After integrating the module in the smallest NOEL-V processor configuration, the results show a 3-7x improvement for commonly used ML programs with only 25% area increase.

RISC-V Vector (RVV) is an instruction-set architecture extension that provides RISC-V vectorization support. It enables RISC-V processors to process data arrays, alongside traditional scalar operations in the SIMD manner to enhance the performance via data-level parallelism. However, the complexity of vector processing units can lead to the ambiguity of the vector specification. As such, a new Domain-Specific Language (DSL) is desired to build a golden reference model that accurately describes the behavior of the instructions and registers of a given ISA. Driven by this motivation, RIOS Lab designs the golden model of RVV in Sail, which is a DSL designed for expressing the ISA semantics of distinct computer architectures. This model will be released by the end of this year as a part of the standard RVV release of RISC-V foundation. In addition, as it is necessary to verify the behavior of RVV instructions implemented in our Sail model, RIOS Lab will also introduce an open-source random architectural test generator named “Talon”. Talon is portable with the RISCOF infrastructures and the generated instruction patterns are configurable via YAML. This presentation will show the design of the RVV Sail model and current progress. It will also demonstrate our random test generator as well as the new characteristics in future work. This proposal yields a golden model of RVV program executions and takes an active role in supporting the growth and evolution of the RISC-V ecosystem.

When deploying a neural network (NN) as part of a low-power edge application such as mobile or IoT devices, designers must trade-off flexibility and power efficiency. Hard-wired accelerators are often chosen for their inherent parallelism and performance; however, accelerators are rigid, may be difficult to program, and are not necessarily suited for some NNs that can’t take advantage of the parallelism provided. Conversely, general-purpose processors, ubiquitous in edge applications, may lack the compute efficiency needed under a strict power budget. A third option is a processor, optimized for parallelizable workloads, that can scale to many-core to deliver the necessary performance of multiple tera-ops per second that machine learning algorithms may require. Consequently, demand for vector-enabled, general-purpose processors that are compiler-friendly is rapidly growing. Using the widely deployed MobileNet CNN as an example, SiFive will demonstrate a TensorFlow Lite solution optimized for RISC-V vectors and then further boosted by more than 10X in some cases with SiFive Intelligence Extensions.

Neural Networks are foundational AI constructs for recognizing relationships in data requiring processing massive datasets in the form of tensors. Tensor processing is central to AI and machine learning applications. The RISC-V Vector (RVV) extension provides the capacity to accelerate computation of these tensor datasets in parallel on multi core processors. However, the extension alone provides only a part of the solution with software developers needing a standard programming interface targeting RVV for data intensive operations and host CPU for latency-sensitive operations. This presentation will dive in detail how our team accelerated the execution of a tensor-based neural network on the Spike simulator using open source and open standard software. The journey begins at the driver level where we implemented low-level abstractions, then moves to modifications made to extend the LLVM compiler, on to an interface designed to enable SYCL. We will then explore how the demo application was built using the open source Eigen and SYCL-DNN libraries with the ResNet50 neural network commonly used for processor benchmarks. Finally, we will demonstrate this working example neural network running in the Spike simulator.

The RISC-V Vector Extensions offer developers a wide range of options and configurations to provide hardware acceleration for the key algorithms for compute intensive applications such as Machine Learning. In addition, the RISC-V ISA (Instruction Set Architecture) supports custom instructions and heterogeneous multicore arrays that can scale to address the most demanding requirements. This talk will highlight the eSol project that successfully mixed an Arm Corex-A57 and 16 RISC V cores to accelerate the ALEXNET image recognition deep neural network as a custom hardware accelerator. Using standard training datasets, the workload requirements across the processor array was established for the key metrics for performance and optimization. Further software development helped to fine tune the application and provide test cases that can be used throughout the hardware design and verification process. While the investment in custom hardware accelerators is significant, the support for future software developers is essential for adoption. This early phase software-based analysis helps to jump-start the software optimization, lifecycle management, and the increasing importance of digital twins as operational references.

The FABulous open-source framework for building embedded FPGA had been used in different projects to integrate reconfigurable custom instructions directly into the CPU core of a RISC-V core (e.g., in the FlexBex). This talk will show how the reconfigurability of an eFPGA can be used for vector processing in ML applications. This includes custom data types, like short integers, non-linear arithmetic and working with compressed data formats.

The RISC-V vector extensions represent the most extensive and complex extensions yet for the RISC-V ISA. As a flexible and modular ISA (Instruction Set Architecture) the RISC-V standard extensions offer developers significant flexibility to configure and tune an implementation to the target application needs and requirements. The RISC-V Vectors Extensions offer extensive options and configurations to address a wide range of hardware accelerator requirements. Thus, designs span from edge devices to cloud based accelerators, across the spectrum of compute requirements. Working with the team at BCS (Barcelona Supercomputing Center) the complexity of the Vector accelerator engine has been supported with the adaption of the step-and-compare verification methodology to address the coverage and score-boarding requirements for the RISC-V Vector Extensions. Using a specification envelope model allows the testbench to cover the full range of the options including asynchronous and debug events, plus testing the regular scalar base core. As a general flow the test-bench also supports custom Vector instructions to anticipate future enhancements.

The RISC-V Vector Processing Unit supporting RVV extension is a powerful computational unit delivering unparalleled and scalable compute capability. However, there is a key functionality that must be enabled to ensure VPU can deliver the performance needed. The real challenge is the data movement. Many traditional solutions exist to solve data movement problem. These include extensive prefetching, large dedicated memories and caches. These improvements come at the cost of power and size. Andes Technology’s innovative Streaming Port implemented with Andes Custom Extension solves the data movement problem with minimal power and gate overhead. In addition, ACE can be used for pre- and post- processing data – like format conversion – to further increase VPU performance as well as controlling and managing external accelerators. RISC-V Vector Processing Units are great, but innovative solutions like Andes’ Streaming port make them better.