1# TFLite for Microcontrollers Benchmarks
2
3These benchmarks are for measuring the performance of key models and workloads.
4They are meant to be used as part of the model optimization process for a given
5platform.
6
7## Table of contents
8
9-   [Keyword Benchmark](#keyword-benchmark)
10-   [Person Detection Benchmark](#person-detection-benchmark)
11-   [Run on x86](#run-on-x86)
12-   [Run on Xtensa XPG Simulator](#run-on-xtensa-xpg-simulator)
13-   [Run on Sparkfun Edge](#run-on-sparkfun-edge)
14-   [Run on FVP based on Arm Corstone-300 software](#run-on-fvp-based-on-arm-corstone-300-software)
15
16## Keyword benchmark
17
18The keyword benchmark contains a model for keyword detection with scrambled
19weights and biases.  This model is meant to test performance on a platform only.
20Since the weights are scrambled, the output is meaningless. In order to validate
21the accuracy of optimized kernels, please run the kernel tests.
22
23## Person detection benchmark
24
25The keyword benchmark provides a way to evaluate the performance of the 250KB
26visual wakewords model.
27
28## Run on x86
29
30To run the keyword benchmark on x86, run
31
32```
33make -f tensorflow/lite/micro/tools/make/Makefile run_keyword_benchmark
34```
35
36To run the person detection benchmark on x86, run
37
38```
39make -f tensorflow/lite/micro/tools/make/Makefile run_person_detection_benchmark
40```
41
42## Run on Xtensa XPG Simulator
43
44To run the keyword benchmark on the Xtensa XPG simulator, you will need a valid
45Xtensa toolchain and license.  With these set up, run:
46
47```
48make -f tensorflow/lite/micro/tools/make/Makefile TARGET=xtensa OPTIMIZED_KERNEL_DIR=xtensa TARGET_ARCH=<target architecture> XTENSA_CORE=<xtensa core> run_keyword_benchmark -j18
49```
50
51## Run on Sparkfun Edge
52The following instructions will help you build and deploy this benchmark on the
53[SparkFun Edge development board](https://sparkfun.com/products/15170).
54
55
56If you're new to using this board, we recommend walking through the
57[AI on a microcontroller with TensorFlow Lite and SparkFun Edge](https://codelabs.developers.google.com/codelabs/sparkfun-tensorflow)
58codelab to get an understanding of the workflow.
59
60Build binary using
61
62```
63make -f tensorflow/lite/micro/tools/make/Makefile TARGET=sparkfun_edge person_detection_benchmark_bin
64```
65
66Refer to flashing instructions in the [Person Detection Example](https://github.com/tensorflow/tflite-micro/blob/main/tensorflow/lite/micro/examples/person_detection/README.md#running-on-sparkfun-edge).
67
68## Run on FVP based on Arm Corstone-300 software
69
70For more info about the Corstone-300 software see:
71[tensorflow/lite/micro/cortex_m_corstone_300/README.md](../cortex_m_corstone_300/README.md).
72
73Disclaimer: Executing the benchmark test on the Corstone-300 software will
74provide a general metric of instructions executed. The estimates are not cycle
75accurate, however it aligns to instruction per cycle, and is a consistent
76environment. This means it can detect if code changes changed performance.
77
78The person detection benchmark can also run with Ethos-U enabled, as the
79downloaded model will be optimized for Ethos-U. For more info see:
80[tensorflow/lite/micro/kernels/ethos_u/README.md](../kernels/ethos_u/README.md).
81
82To run the keyword benchmark on FVP:
83
84```
85make -j -f tensorflow/lite/micro/tools/make/Makefile TARGET=cortex_m_corstone_300 TARGET_ARCH=cortex-m55 run_keyword_benchmark
86```
87
88To run the person detection benchmark on FVP:
89
90```
91make -j -f tensorflow/lite/micro/tools/make/Makefile TARGET=cortex_m_corstone_300 TARGET_ARCH=cortex-m55 run_person_detection_benchmark
92```
93
94To run the person detection benchmark on FVP with Ethos-U:
95
96```
97make -j -f tensorflow/lite/micro/tools/make/Makefile CO_PROCESSOR=ethos_u TARGET=cortex_m_corstone_300 TARGET_ARCH=cortex-m55 run_person_detection_benchmark
98```
99