1# CMSIS NN 2 3## About 4This page give a quick overview of the functions available and key differences between them. 5 6**Note:** The GitHub documentation does not follow the *develop* branch but rather the last official release in the *master* branch. Consequently, the group documentation linked to in the table table might not have the listed API. Please refer to the description in the [header](https://github.com/ARM-software/CMSIS_5/blob/develop/CMSIS/NN/Include/arm_nnfunctions.h) file instead. 7 8## Support / Contact 9For any questions or to reach the CMSIS-NN team, please create a new issue in https://github.com/ARM-software/CMSIS_5/issues 10 11## Legacy vs TFL micro compliant APIs 12There are two kinds of APIs available in the CMSIS-NN repository; One that supports a legacy symmetric quantization scheme[1] and one that supports TFL micro's symmetric quantization scheme. One of the main differences is how the quantization is performed. The legacy APIs have a fixed point format with power of 2 scaling. This simplifies the re-quantization to a cycle efficient shift operation. No new development is done on the legacy functions and all of the new development is on the functions that support TFL micro. The table below highlights some of the differences between the two formats for convolution related functions. The TFL micro compliant APIs in most cases have a _s8 suffix and is always specified in the API header file. 13 14Operation | Legacy APIs | TFL micro compliant APIs| 15|:-----------|:---------------------|:-------------| 16Core loop | No input or filter offset | Input and/or filter offset | 17Re-quantization | Shift and saturate in one instruction. ~ 5 cycles | Greater than 200 cycles for one output element 18Quantization | Per layer quantization | Per-channel quantization 19Output offset | No | Per-layer output offset 20Fused Activation | No | Yes 21 22## TFL micro compliant APIs 23Group | API | Base Operator | Input Constraints | Additional memory required for <br/> optimizations (bytes) | DSP Optimized | MVE Optimized | Other comments | 24|:----| :---| :------------ | :---------------- | :--------------------------------------------------------| :-------------| :------------- | :------------- | 25|[Conv](https://arm-software.github.io/CMSIS_5/NN/html/group__NNConv.html)||||| | || 26||arm_convolve_wrapper_s8()|CONV|dilation = 1|n.a.| Yes | Yes |The additional memory required depends on the optimal convolution function called| 27||arm_convolve_s8()|CONV|dilation = 1|4 * ker_x * ker_y * input_ch| Yes | Yes || 28||arm_convolve_1x1_s8_fast() | CONV | dilation = 1 <br/> ker_x = 1, ker_y = 1 <br/> pad = 0<br/> stride = 1<br/> input_ch % 4 = 0| 0 | Yes |Yes || 29||arm_convolve_1_n_s8() | CONV | dilation = 1 <br/> output_y % 4 = 0 | No |Yes || 30|| arm_depthwise_conv_3x3_s8() | DEPTHWISE_CONV | dilation = 1 <br/> depth_multiplier = 1 <br/> pad_x <= 1 | No|No|No| Preferred function for 3x3 kernel size for DSP extension. </br> For MVE, use arm_depthwise_conv_s8_opt()|| 31| | arm_depthwise_conv_s8() | DEPTHWISE_CONV | dilation = 1 | No|No|No|| 32|| arm_depthwise_conv_s8_opt()| DEPTHWISE_CONV | dilation = 1 <br/> depth_multiplier = 1 | DSP: 2 * ker_x * ker_y * input_ch <br/> MVE: 2 * DSP + 4 | Yes| Yes| Best case is when channels are multiple of 4 or <br/>at the least >= 4 | 33|[Fully Connected](https://arm-software.github.io/CMSIS_5/NN/html/group__FC.html)||||| | | | 34|| arm_fully_connected_s8() |FULLY CONNECTED & <br/> MAT MUL | None | 0 | Yes | Yes | | 35|[Pooling](https://arm-software.github.io/CMSIS_5/NN/html/group__Pooling.html)||||| | || 36|| arm_avgpool_s8() | AVERAGE POOL | None | input_ch * 2<br/>(DSP only) | Yes| Yes| Best case case is when channels are multiple of 4 or <br/> at the least >= 4 | 37|| arm_maxpool_s8() | MAX POOL | None | None | Yes| Yes| | 38|[Softmax](https://arm-software.github.io/CMSIS_5/NN/html/group__Softmax.html)||||| | || 39||arm_softmax_q7()| SOFTMAX | None | None | Yes | No | Not bit exact to TFLu but can be up to 70x faster | 40||arm_softmax_s8()| SOFTMAX | None | None | No | Yes | Bit exact to TFLu | 41||arm_softmax_u8()| SOFTMAX | None | None | No | No | Bit exact to TFLu | 42|[SVDF](https://arm-software.github.io/CMSIS_5/NN/html/group__SVDF.html)||||| | || 43||arm_svdf_s8()| SVDF | None | None | Yes | No | Bit exact to TFLu | 44|[Misc](https://arm-software.github.io/CMSIS_5/NN/html/group__groupNN.html)||||| | || 45||arm_reshape_s8()| SOFTMAX | None | None | No | No | | 46||arm_elementwise_add_s8()| ELEMENTWISE ADD | None | None | Yes| Yes| Reshape is not done in this function <br/> Only minor improvements are expected | 47||arm_elementwise_mul_s8()| ELEMENTWISE MUL | None | None | Yes| Yes| Reshape is not done in this function <br/> Only minor improvements are expected | 48||arm_relu_q7() | RELU | None | None | Yes| No| 49||arm_relu6_s8() | RELU | None | None | Yes| No| 50|[Concat](https://arm-software.github.io/CMSIS_5/NN/html/group__groupNN.html)||||| | || 51||arm_concatenation_s8_w() | CONCAT | None | None | No| No|| 52||arm_concatenation_s8_x() | CONCAT | None | None | No| No|| 53||arm_concatenation_s8_y() | CONCAT | None | None | No| No|| 54||arm_concatenation_s8_z() | CONCAT | None | None | No| No|| 55 56 57## Reference 58[1] Legacy CMSIS-NN and how to use it https://developer.arm.com/solutions/machine-learning-on-arm/developer-material/how-to-guides/converting-a-neural-network-for-arm-cortex-m-with-cmsis-nn/single-page 59
