1Runtime Instrumentation Testing - N1SDP
2=======================================
3
4For this test we used the N1 System Development Platform (`N1SDP`_), which
5contains an SoC consisting of two dual-core Arm N1 clusters.
6
7The following source trees and binaries were used:
8
9- TF-A [`v2.9-rc0-16-g666aec401`_]
10- TFTF [`v2.9-rc0`_]
11- SCP/MCP `Prebuilt Images`_
12
13Please see the Runtime Instrumentation :ref:`Testing Methodology
14<Runtime Instrumentation Methodology>` page for more details.
15
16Procedure
17---------
18
19#. Build TFTF with runtime instrumentation enabled:
20
21    .. code:: shell
22
23        make CROSS_COMPILE=aarch64-none-elf- PLAT=n1sdp \
24            TESTS=runtime-instrumentation all
25
26#. Build TF-A with the following build options:
27
28    .. code:: shell
29
30        make CROSS_COMPILE=aarch64-none-elf- PLAT=n1sdp \
31            ENABLE_RUNTIME_INSTRUMENTATION=1 fiptool all
32
33#. Fetch the SCP firmware images:
34
35    .. code:: shell
36
37        curl --fail --connect-timeout 5 --retry 5 \
38            -sLS -o build/n1sdp/release/scp_rom.bin \
39            https://downloads.trustedfirmware.org/tf-a/css_scp_2.12.0/n1sdp/release/n1sdp-bl1.bin
40        curl --fail --connect-timeout 5 \
41            --retry 5 -sLS -o build/n1sdp/release/scp_ram.bin \
42            https://downloads.trustedfirmware.org/tf-a/css_scp_2.12.0/n1sdp/release/n1sdp-bl2.bin
43
44#. Fetch the MCP firmware images:
45
46    .. code:: shell
47
48        curl --fail --connect-timeout 5 --retry 5 \
49            -sLS -o build/n1sdp/release/mcp_rom.bin \
50            https://downloads.trustedfirmware.org/tf-a/css_scp_2.12.0/n1sdp/release/n1sdp-mcp-bl1.bin
51        curl --fail --connect-timeout 5 --retry 5 \
52            -sLS -o build/n1sdp/release/mcp_ram.bin \
53            https://downloads.trustedfirmware.org/tf-a/css_scp_2.12.0/n1sdp/release/n1sdp-mcp-bl2.bin
54
55#. Using the fiptool, create a new FIP package and append the SCP ram image onto
56   it.
57
58    .. code:: shell
59
60        ./tools/fiptool/fiptool create --blob \
61                uuid=cfacc2c4-15e8-4668-82be-430a38fad705,file=build/n1sdp/release/bl1.bin \
62                --scp-fw build/n1sdp/release/scp_ram.bin build/n1sdp/release/scp_fw.bin
63
64#. Append the MCP image to the FIP.
65
66    .. code:: shell
67
68        ./tools/fiptool/fiptool create \
69            --blob uuid=54464222-a4cf-4bf8-b1b6-cee7dade539e,file=build/n1sdp/release/mcp_ram.bin \
70            build/n1sdp/release/mcp_fw.bin
71
72#. Then, add TFTF as the Non-Secure workload in the FIP image:
73
74    .. code:: shell
75
76        make CROSS_COMPILE=aarch64-none-elf- PLAT=n1sdp \
77            ENABLE_RUNTIME_INSTRUMENTATION=1 SCP_BL2=/dev/null \
78            BL33=<path/to/tftf.bin>  fip
79
80#. Load the following images onto the development board: ``fip.bin``,
81   ``scp_rom.bin``, ``scp_ram.bin``, ``mcp_rom.bin``, and ``mcp_ram.bin``.
82
83.. note::
84
85    These instructions presume you have a complete firmware stack. The N1SDP
86    `user guide`_ provides a detailed explanation on how to get setup from
87    scratch.
88
89Results
90-------
91
92``CPU_SUSPEND`` to deepest power level
93~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
94
95.. table:: ``CPU_SUSPEND`` latencies (µs) to deepest power level in
96        parallel (v2.9)
97
98    +---------+------+-----------+--------+-------------+
99    | Cluster | Core | Powerdown | Wakeup | Cache Flush |
100    +---------+------+-----------+--------+-------------+
101    |    0    |  0   |    2.80   | 10.08  |     0.80    |
102    +---------+------+-----------+--------+-------------+
103    |    0    |  0   |    4.14   | 15.92  |     0.16    |
104    +---------+------+-----------+--------+-------------+
105    |    1    |  0   |    3.68   | 12.96  |     0.16    |
106    +---------+------+-----------+--------+-------------+
107    |    1    |  0   |    3.36   | 18.58  |     0.18    |
108    +---------+------+-----------+--------+-------------+
109
110.. table:: ``CPU_SUSPEND`` latencies (µs) to deepest power level in
111        parallel (v2.10)
112
113    +---------+------+----------------+------------------+-----------------+
114    | Cluster | Core |   Powerdown    |      Wakeup      |   Cache Flush   |
115    +---------+------+----------------+------------------+-----------------+
116    |    0    |  0   |      2.12      | 23.94 (+137.50%) |  0.42 (-47.50%) |
117    +---------+------+----------------+------------------+-----------------+
118    |    0    |  0   |      3.52      | 42.08 (+164.32%) |  0.26 (+62.50%) |
119    +---------+------+----------------+------------------+-----------------+
120    |    1    |  0   | 2.76 (-25.00%) | 38.3 (+195.52%)  |  0.26 (+62.50%) |
121    +---------+------+----------------+------------------+-----------------+
122    |    1    |  0   |      2.64      | 44.56 (+139.83%) | 0.36 (+100.00%) |
123    +---------+------+----------------+------------------+-----------------+
124
125.. table:: ``CPU_SUSPEND`` latencies (µs) to deepest power level in
126        serial (v2.9)
127
128    +---------+------+-----------+--------+-------------+
129    | Cluster | Core | Powerdown | Wakeup | Cache Flush |
130    +---------+------+-----------+--------+-------------+
131    |    0    |  0   |    1.86   |  9.92  |     0.32    |
132    +---------+------+-----------+--------+-------------+
133    |    0    |  0   |    2.70   | 10.48  |     0.36    |
134    +---------+------+-----------+--------+-------------+
135    |    1    |  0   |    1.78   |  9.72  |     0.16    |
136    +---------+------+-----------+--------+-------------+
137    |    1    |  0   |    1.94   | 10.44  |     0.16    |
138    +---------+------+-----------+--------+-------------+
139
140.. table:: ``CPU_SUSPEND`` latencies (µs) to deepest power level in
141        serial (v2.10)
142
143    +---------+------+-----------+------------------+----------------+
144    | Cluster | Core | Powerdown |      Wakeup      |  Cache Flush   |
145    +---------+------+-----------+------------------+----------------+
146    |    0    |  0   |    1.74   | 23.7 (+138.91%)  |      0.3       |
147    +---------+------+-----------+------------------+----------------+
148    |    0    |  0   |    2.08   | 23.96 (+128.63%) | 0.26 (-27.78%) |
149    +---------+------+-----------+------------------+----------------+
150    |    1    |  0   |    1.9    | 23.62 (+143.00%) | 0.28 (+75.00%) |
151    +---------+------+-----------+------------------+----------------+
152    |    1    |  0   |    2.06   | 23.92 (+129.12%) | 0.26 (+62.50%) |
153    +---------+------+-----------+------------------+----------------+
154
155``CPU_SUSPEND`` to power level 0
156~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
157
158.. table:: ``CPU_SUSPEND`` latencies (µs) to power level 0 in
159        parallel (v2.9)
160
161    +---------------------------------------------------+
162    |          test_rt_instr_cpu_susp_parallel          |
163    +---------+------+-----------+--------+-------------+
164    | Cluster | Core | Powerdown | Wakeup | Cache Flush |
165    +---------+------+-----------+--------+-------------+
166    |    0    |  0   |    0.88   | 12.32  |     0.26    |
167    +---------+------+-----------+--------+-------------+
168    |    0    |  0   |    2.12   | 14.62  |     0.26    |
169    +---------+------+-----------+--------+-------------+
170    |    1    |  0   |    1.86   | 14.14  |     0.16    |
171    +---------+------+-----------+--------+-------------+
172    |    1    |  0   |    1.92   |  9.44  |     0.18    |
173    +---------+------+-----------+--------+-------------+
174
175.. table:: ``CPU_SUSPEND`` latencies (µs) to power level 0 in
176        parallel (v2.10)
177
178    +---------+------+---------------+------------------+----------------+
179    | Cluster | Core |   Powerdown   |      Wakeup      |  Cache Flush   |
180    +---------+------+---------------+------------------+----------------+
181    |    0    |  0   | 1.5 (+70.45%) | 35.02 (+184.25%) |      0.24      |
182    +---------+------+---------------+------------------+----------------+
183    |    0    |  0   |      1.92     | 38.12 (+160.74%) |      0.28      |
184    +---------+------+---------------+------------------+----------------+
185    |    1    |  0   |      1.88     | 38.1 (+169.45%)  | 0.26 (+62.50%) |
186    +---------+------+---------------+------------------+----------------+
187    |    1    |  0   |      2.04     | 23.1 (+144.70%)  |      0.24      |
188    +---------+------+---------------+------------------+----------------+
189
190.. table:: ``CPU_SUSPEND`` latencies (µs) to power level 0 in serial (v2.9)
191
192    +---------------------------------------------------+
193    |           test_rt_instr_cpu_susp_serial           |
194    +---------+------+-----------+--------+-------------+
195    | Cluster | Core | Powerdown | Wakeup | Cache Flush |
196    +---------+------+-----------+--------+-------------+
197    |    0    |  0   |    1.52   |  9.40  |     0.30    |
198    +---------+------+-----------+--------+-------------+
199    |    0    |  0   |    1.92   |  9.80  |     0.18    |
200    +---------+------+-----------+--------+-------------+
201    |    1    |  0   |    2.20   |  9.60  |     0.14    |
202    +---------+------+-----------+--------+-------------+
203    |    1    |  0   |    1.82   |  9.78  |     0.18    |
204    +---------+------+-----------+--------+-------------+
205
206.. table:: ``CPU_SUSPEND`` latencies (µs) to power level 0 in serial (v2.10)
207
208    +---------+------+-----------+------------------+-----------------+
209    | Cluster | Core | Powerdown |      Wakeup      |   Cache Flush   |
210    +---------+------+-----------+------------------+-----------------+
211    |    0    |  0   |    1.52   | 23.08 (+145.53%) |       0.3       |
212    +---------+------+-----------+------------------+-----------------+
213    |    0    |  0   |    1.98   | 23.68 (+141.63%) |  0.28 (+55.56%) |
214    +---------+------+-----------+------------------+-----------------+
215    |    1    |  0   |    1.84   | 23.86 (+148.54%) | 0.28 (+100.00%) |
216    +---------+------+-----------+------------------+-----------------+
217    |    1    |  0   |    1.98   | 23.68 (+142.13%) |  0.28 (+55.56%) |
218    +---------+------+-----------+------------------+-----------------+
219
220``CPU_OFF`` on all non-lead CPUs
221~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
222
223``CPU_OFF`` on all non-lead CPUs in sequence then, ``CPU_SUSPEND`` on the lead
224core to the deepest power level.
225
226.. table:: ``CPU_OFF`` latencies (µs) on all non-lead CPUs (v2.9)
227
228    +---------+------+-----------+--------+-------------+
229    | Cluster | Core | Powerdown | Wakeup | Cache Flush |
230    +---------+------+-----------+--------+-------------+
231    |    0    |  0   |    1.84   |  9.94  |     0.32    |
232    +---------+------+-----------+--------+-------------+
233    |    0    |  0   |   14.20   | 13.10  |     0.50    |
234    +---------+------+-----------+--------+-------------+
235    |    1    |  0   |   13.88   | 12.36  |     0.42    |
236    +---------+------+-----------+--------+-------------+
237    |    1    |  0   |   14.40   | 13.26  |     0.52    |
238    +---------+------+-----------+--------+-------------+
239
240.. table:: ``CPU_OFF`` latencies (µs) on all non-lead CPUs (v2.10)
241
242    +---------+------+-----------+------------------+----------------+
243    | Cluster | Core | Powerdown |      Wakeup      |  Cache Flush   |
244    +---------+------+-----------+------------------+----------------+
245    |    0    |  0   |    1.78   | 23.7 (+138.43%)  |      0.3       |
246    +---------+------+-----------+------------------+----------------+
247    |    0    |  0   |   13.96   | 31.16 (+137.86%) | 0.34 (-32.00%) |
248    +---------+------+-----------+------------------+----------------+
249    |    1    |  0   |   13.54   | 30.24 (+144.66%) | 0.26 (-38.10%) |
250    +---------+------+-----------+------------------+----------------+
251    |    1    |  0   |   14.46   | 31.12 (+134.69%) | 0.7 (+34.62%)  |
252    +---------+------+-----------+------------------+----------------+
253
254``CPU_VERSION`` in parallel
255~~~~~~~~~~~~~~~~~~~~~~~~~~~
256
257.. table:: ``CPU_VERSION`` latency (µs) in parallel on all cores (v2.9)
258
259    +------------------------------------+
260    | test_rt_instr_psci_version_parallel|
261    +-------------+--------+-------------+
262    |   Cluster   |  Core  |   Latency   |
263    +-------------+--------+-------------+
264    |      0      |   0    |     0.08    |
265    +-------------+--------+-------------+
266    |      0      |   0    |     0.26    |
267    +-------------+--------+-------------+
268    |      1      |   0    |     0.20    |
269    +-------------+--------+-------------+
270    |      1      |   0    |     0.26    |
271    +-------------+--------+-------------+
272
273.. table:: ``CPU_VERSION`` latency (µs) in parallel on all cores (v2.10)
274
275    +----------------------------------------------+
276    | test_rt_instr_psci_version_parallel (latest) |
277    +-------------+--------+-----------------------+
278    |   Cluster   |  Core  |        Latency        |
279    +-------------+--------+-----------------------+
280    |      0      |   0    |     0.14 (+75.00%)    |
281    +-------------+--------+-----------------------+
282    |      0      |   0    |          0.22         |
283    +-------------+--------+-----------------------+
284    |      1      |   0    |          0.2          |
285    +-------------+--------+-----------------------+
286    |      1      |   0    |          0.26         |
287    +-------------+--------+-----------------------+
288
289--------------
290
291*Copyright (c) 2023, Arm Limited. All rights reserved.*
292
293.. _v2.9-rc0-16-g666aec401: https://review.trustedfirmware.org/plugins/gitiles/TF-A/trusted-firmware-a/+/refs/heads/v2.9-rc0-16-g666aec401
294.. _v2.9-rc0: https://review.trustedfirmware.org/plugins/gitiles/TF-A/tf-a-tests/+/refs/tags/v2.9-rc0
295.. _user guide: https://gitlab.arm.com/arm-reference-solutions/arm-reference-solutions-docs/-/blob/master/docs/n1sdp/user-guide.rst
296.. _Prebuilt Images:  https://downloads.trustedfirmware.org/tf-a/css_scp_2.11.0/n1sdp/release/
297.. _N1SDP: https://developer.arm.com/documentation/101489/latest
298