1| Supported Targets | ESP32 | ESP32-S2 | ESP32-C3 |
2| ----------------- | ----- | -------- | -------- |
3# Example: Application Level Tracing - SystemView Tracing (sysview_tracing)
4
5This test code shows how to perform system-wide behavioral analysis of the program using [SEGGER SystemView tool](https://www.segger.com/products/development-tools/systemview/).
6
7For description of [SystemView tracing](https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/app_trace.html#system-behaviour-analysis-with-segger-systemview) please refer to **ESP32 Programming Guide**, section **Application Level Tracing library**. The following example provides practical implementation of this functionality.
8
9
10## Use Case
11
12To find the reason of program's misbehaviour it is often necessary to have the whole picture of interacting components (scheduler, IRQs, tasks, semaphores etc.) in the system. In such cases tools which allow to trace the behaviour of the system as a whole can be very useful.
13Consider the following situation. User program have a timer and a task. Upon every tick the timer sends an event to the task. When task receives event it prints a message. Timer should notify the task 10 times.
14There can be a problem which causes the task to lose some events.
15Below is the timer's ISR code:
16
17```
18static void example_timer_isr(void *arg)
19{
20    example_event_data_t *tim_arg = (example_event_data_t *)arg;
21
22    if (tim_arg->thnd != NULL) {
23        if (tim_arg->count++ < 10) {
24            BaseType_t xHigherPriorityTaskWoken = pdFALSE;
25            if (xTaskNotifyFromISR(tim_arg->thnd, tim_arg->count, eSetValueWithOverwrite, &xHigherPriorityTaskWoken) != pdPASS) {
26                ESP_EARLY_LOGE(TAG, "Failed to notify task %p", tim_arg->thnd);
27            } else {
28                if (xHigherPriorityTaskWoken == pdTRUE) {
29                    portYIELD_FROM_ISR();
30                }
31            }
32        }
33    }
34    // re-start timer
35    example_timer_rearm(tim_arg->group, tim_arg->timer);
36}
37```
38
39Below is the task's code:
40
41```
42static void example_task(void *p)
43{
44    example_event_data_t *arg = (example_event_data_t *) p;
45    timer_isr_handle_t inth;
46
47    ESP_LOGI(TAG, "%p: run task", xTaskGetCurrentTaskHandle());
48
49    esp_err_t res = timer_isr_register(arg->group, arg->timer, example_timer_isr, arg, 0, &inth);
50    if (res != ESP_OK) {
51        ESP_LOGE(TAG, "%p: failed to register timer ISR", xTaskGetCurrentTaskHandle());
52    }
53    else {
54        res = timer_start(arg->group, arg->timer);
55        if (res != ESP_OK) {
56            ESP_LOGE(TAG, "%p: failed to start timer", xTaskGetCurrentTaskHandle());
57        }
58    }
59
60    while (1) {
61        uint32_t event_val;
62        xTaskNotifyWait(0, 0, &event_val, portMAX_DELAY);
63        ESP_LOGI(TAG, "Task[%p]: received event %d", xTaskGetCurrentTaskHandle(), event_val);
64    }
65}
66```
67
68Potential problem can arise in such program because task and timer has no any mechanism to acknowledge the events transfer. Task needs some time to process an event before waiting for the next one. In this case call to `ESP_LOGI` will be the most time consuming part of event processing. Therefore when timer's ISR is called at some rate it can happen that `xTaskNotifyFromISR` gets called several times before task calls `xTaskNotifyWait`. In these conditions some events will be lost. Possible solution for this is to increase timer's tick period or to use some events acknowledgement mechanism.
69
70Check the full example code [sysview_tracing](main/sysview_tracing.c) that when compiled in dual core mode reproduces the described problem on both cores. Below is the output of example compiled in single core mode. It shows that task misses several events:
71
72```
73I (295) example: 0x3ffb6c10: run task
74I (297) example: Task[0x3ffb6c10]: received event 1
75I (300) example: Task[0x3ffb6c10]: received event 2
76I (306) example: Task[0x3ffb6c10]: received event 5
77I (311) example: Task[0x3ffb6c10]: received event 8
78I (317) example: Task[0x3ffb6c10]: received event 10
79```
80
81
82## Detecting The Reason Of The Problem
83
84Besides built-in functionality to trace FreeRTOS internal operations SystemView also provides user with means to define its own trace messages and transfer problem-specific data to the host. In this example we extend SystemView with user-defined tracing module in order to
85make the root cause of the problem more clear in the gathered trace.
86
87There are two ways to send user-defined info to SystemView:
881. Using built-in SystemView messages. This method uses `SEGGER_SYSVIEW_OnUserStart` and `SEGGER_SYSVIEW_OnUserStop` API to indicate that some user event started or stopped. Disadvantge of this way is that those API do not carry any other data except for user event ID. Advantage is that you do not need to write any additional code to use these functions. In SystemView messages sent by means of that API will be shown as `User Start` or `User Stop` with user event ID as only parameter: 0 - for timer and 1 - for task in this example.
892. Using custom SystemView messages. This is more flexible way. It implies extending SystemView with user module having its own set of events with any number of parameters (data) in each. For more details on extending see [code of this example](main/sysview_tracing.c) and [SystemView User Manual](https://www.segger.com/downloads/jlink/UM08027).
90
91To run the example and find out the reason of the problem:
92
931.  Connect JTAG interface to ESP32 board, power up both JTAG and ESP32. For details how to setup JTAG interface see [JTAG Debugging](https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/jtag-debugging/index.html).
94
952.  [Run OpenOCD](https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/jtag-debugging/index.html#run-openocd). If you are using the [binary distribution of OpenOCD](https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/jtag-debugging/index.html#jtag-debugging-setup-openocd) and one of versions of [ESP-WROVER-KIT](https://docs.espressif.com/projects/esp-idf/en/latest/hw-reference/modules-and-boards.html#esp-wrover-kit-v3), respective command line will look as follows:
96
97    ```
98    cd ~/esp/openocd-esp32
99    bin/openocd -s share/openocd/scripts -f board/esp32-wrover-kit-3.3v.cfg
100    ```
101NOTE: In order to run this example you need OpenOCD version `v0.10.0-esp32-20181105` or later.
102
1033.  Compile and load the example. Note to enable application tracing in menuconfig by going to `Component config > Application Level Tracing` and selecting `(X) Trace memory`. Also in order to generate SystemView trace data you need to enable this in `Component config > Application Level Tracing > FreeRTOS SystemView Tracing` by checking `(X) SystemView Tracing Enable`. Also ensure that all SystemView events are enabled there.
104
1054.  If you are going to use custom events enable them by checking `Example Configuration > Use custom SystemView event IDs`. By default SystemView shows only numeric values of IDs and parameters for custom messages in `Events` view. To make them pretty-looking you need to define for them parsing templates in `SYSVIEW_FreeRTOS.txt` which is resided in SystemView installation directory. For this example you can use `SYSVIEW_FreeRTOS.txt` from the project's root directory.
106
1075.  It is useful to use GDB to start and/or stop tracing automatically. To do this you need to prepare special `gdbinit` file:
108
109    ```
110    target remote :3333
111    mon reset halt
112    b app_main
113    commands
114    mon esp sysview start file:///tmp/sysview_example.svdat
115    # For dual-core mode uncomment the line below and comment the line above
116    # mon esp sysview start file:///tmp/sysview_example0.svdat file:///tmp/sysview_example1.svdat
117    c
118    end
119    c
120    ```
121
122    Using this file GDB will connect to the target, reset it, and start tracing when it hit breakpoint at `app_main`. Trace data will be saved to `/tmp/sysview_example.svdat`.
123
1246.  Run GDB using the following command from the project root directory:
125
126    ```
127    xtensa-esp32-elf-gdb -x gdbinit build/sysview_tracing.elf
128    ```
129
130    **Note:** Replace `xtensa-esp32-elf-gdb` with `xtensa-esp32s2-elf-gdb` if running the example on ESP32-S2.
131
1327.  When program prints the last message, interrupt its execution (e.g. by pressing `CTRL+C`) and type the following command in GDB console to stop tracing:
133
134    ```
135    mon esp sysview stop
136    ```
137
138    You can also use another breakpoint to stop tracing and add respective lines to `gdbinit`  at step 5.
139
1408.  Open trace data file in SystemView tool.
141
1429.  Right-click on any event in `Events` view and select:
143
144    - `Show User Events only` if you compiled example to use only built-in event.
145    - `Show APIs only` if you compiled example to use custom events.
146
14710.  Now you can navigate over user-defined messages and see when timer `TG1_T0_LEVEL` sends events (`SYSVIEW_EXAMPLE_SEND_EVENT_START/STOP` or `User Start/Stop(0)`) and when task `svtrace0` reads them (`SYSVIEW_EXAMPLE_WAIT_EVENT_START/STOP` or `User Start/Stop(1)`).
148    If you compiled example to use custom events you will also be able to see the values sent by timer and ones actually received by tasks.
149