1# CMSIS NN 2CMSIS NN software library is a collection of efficient neural network kernels developed to maximize the 3performance and minimize the memory footprint of neural networks on Arm Cortex-M processors. 4 5## Supported Framework 6The library follows the [int8](https://www.tensorflow.org/lite/performance/quantization_spec) and int16 quantization specification of TensorFlow Lite for Microcontrollers. 7 8## Branches and Tags 9There is a single branch called 'main'. 10Tags are created during a release. Two releases are planned to be done in a year. The releases can be found 11[here](https://github.com/ARM-software/CMSIS-NN/releases) . 12 13## Current Operator Support 14In general optimizations are written for an architecture feature. This falls into one of the following categories. 15Based on feature flags for a processor or architecture provided to the compiler, the right implementation is picked. 16### Pure C 17 There is always a pure C implementation for an operator. This is used for processors like Arm Cortex-M0 or Cortex-M3. 18### DSP Extension 19Processors with DSP extension uses Single Instruction Multiple Data(SIMD) instructions for optimization. Examples of 20processors here are Cortex-M4 or a Cortex-M33 configured with optional DSP extension. 21 22### MVE Extension 23Processors with Arm Helium Technology use the Arm M-profile Vector Extension(MVE) instructions for optimization. 24Examples are Cortex-M55 or Cortex-M85 configured with MVE. 25 26 27| Operator | C <br> int8 | C<br>int16 | DSP<br>int8 | DSP<br>int16 | MVE<br>int8 | MVE<br>int16 | 28| --------------- | ----------- | ---------- | ----------- | ------------ | ----------- | ------------ | 29| Conv2D | Yes | Yes | Yes | Yes | Yes | Yes | 30| DepthwiseConv2D | Yes | Yes | Yes | Yes | Yes | Yes | 31| Fully Connected | Yes | Yes | Yes | Yes | Yes | Yes | 32| Add | Yes | Yes | Yes | Yes | Yes | Yes | 33| Mul | Yes | Yes | Yes | Yes | Yes | Yes | 34| MaxPooling | Yes | Yes | Yes | Yes | Yes | Yes | 35| AvgPooling | Yes | Yes | Yes | Yes | Yes | Yes | 36| Softmax | Yes | Yes | Yes | Yes | Yes | No | 37| LSTM | Yes | NA | Yes | NA | Yes | NA | 38| SVDF | Yes | No | Yes | No | Yes | No | 39 40 41## Contribution Guideline 42First, a thank you for the contribution. Here are some guidelines and good to know information to get started. 43 44### Coding Guideline 45By default, follow the style used in the file. You'll soon start noticing a pattern like 46* Variable and function names are lower case with an underscore separator. 47* Hungarian notation is not used. Well, almost. 48* If the variable names don't convey the action, then add comments. 49 50### New Files 51One function per file is followed in most places. In those cases, the file name must match the function name. Connect 52the function to an appropriate Doxygen group as well. 53 54### Doxygen 55Function prototypes must have a detailed comment header in Doxygen format. You can execute the doxygen document generation 56script in the Doxygen folder to check that no errors are introduced. 57 58### Unit Tests 59For any new features and bug fixes, new unit tests are needed. Improvements have to be verifed by unit tests. If you do 60not have the means to execute the tests, you can still make the PR and comment that you need help in completing/executing 61the unit tests. 62 63### Version & Date 64Each File has a version number and a date field that must be updated when making any change to that file. The versioning 65follows Semantic Versioning 2.0.0 format. 66 67## Building CMSIS-NN as a library 68It is recommended to use toolchain files from [Arm Ethos-U Core Platform](https://review.mlplatform.org/admin/repos/ml/ethos-u/ethos-u-core-platform) project. These are supporting TARGET_CPU, which is a required argument. Note that if not specifying TARGET_CPU, these toolchains will set some default. The format must be TARGET_CPU=cortex-mXX, see examples below. 69 70Here is an example: 71 72``` 73cd </path/to/CMSIS_NN> 74mkdir build 75cd build 76cmake .. -DCMAKE_TOOLCHAIN_FILE=</path/to/ethos-u-core-platform>/cmake/toolchain/arm-none-eabi-gcc.cmake -DTARGET_CPU=cortex-m55 77make 78``` 79 80Some more examples: 81 82``` 83cmake .. -DCMAKE_TOOLCHAIN_FILE=</path/to/ethos-u-core-platform>/cmake/toolchain/armclang.cmake -DTARGET_CPU=cortex-m55 84cmake .. -DCMAKE_TOOLCHAIN_FILE=</path/to/ethos-u-core-platform>/cmake/toolchain/arm-none-eabi-gcc.cmake -DTARGET_CPU=cortex-m7 85cmake .. -DCMAKE_TOOLCHAIN_FILE=</path/to/ethos-u-core-platform>/cmake/toolchain/armclang.cmake -DTARGET_CPU=cortex-m3 86``` 87 88### Compiler Options 89Default optimization level is set at Ofast. Please change according to project needs. Just bear in mind this can impact 90performance. With only optimization level -O0, *ARM_MATH_AUTOVECTORIZE* needs to be defined for processors with Helium 91Technology. 92 93The compiler option *'-fomit-frame-pointer'* is enabled by default at -O and higher. When no optimization level is specified, 94you may need to specify '-fomit-frame-pointer'. 95 96The compiler option *'-fno-builtin'* does not utilize optimized implementations of e.g. memcpy and memset, which are heavily used by CMSIS-NN. It can significantly downgrade performance. So this should be avoided. The compiler option *'-ffreestanding'* should also be avoided as it enables '-fno-builtin' implicitly. 97 98### Supported Compilers 99* CMSIS-NN is tested on Arm Compiler 6 and on Arm GNU Toolchain. 100* IAR compiler is not tested and there can be compilation and/or performance issues. 101* Compilation for Host is not supported out of the box. It should be possible to use the C implementation and compile for host with minor stubbing effort. 102 103## Inclusive Language 104This product confirms to Arm’s inclusive language policy and, to the best of our knowledge, does not contain any non-inclusive language. If you find something that concerns you, email terms@arm.com. 105 106## Support / Contact 107 108For any questions or to reach the CMSIS-NN team, please create a new issue in https://github.com/ARM-software/CMSIS-NN/issues
