1############################## 2Mailbox NS Agent Design Update 3############################## 4 5:Organization: Arm Limited 6:Contact: tf-m@lists.trustedfirmware.org 7 8********** 9Background 10********** 11The SPE component that maintains the non-secure clients' request is called 12'NS Agent' in TF-M. Besides the Trustzone-based isolation mechanism, there is 13one other isolation mechanism that implements individual PEs in physically 14isolated cores respectively. NSPE and SPE transfer non-secure client requests 15via inter-processor communication based on mailboxes. The component that 16handles inter-processor communication messages is called ``Mailbox NS Agent``. 17 18.. note:: 19 There may be hardware components and software solutions containing 'mailbox' 20 in their names. The concept ``mailbox`` in this document represent the 21 mechanism described above, which is not referring to the external concepts. 22 23When the first version ``Mailbox NS Agent`` was introduced, the generic FF-M 24interrupt handling was not ready. Hence a customized solution 25``Multiple Core`` is implemented. This customized implementation: 26 27- Perform customized operations on SPM internal data in a deferred interrupt 28 handler. 29- Process mailbox operations as special cases in SPM common logic. 30 31These behaviours couple SPM tightly with mailbox logic, which bring issues for 32maintenance. To address the issue, an updated design shall: 33 34- Make SPM manage other components in a unified way (For example, it is 35 simpler for SPM if all non-SPM components under the IPC model act as 36 ``processes``). 37- Can use FF-M compliant interrupt mechanism and APIs. 38 39Following the above guidelines makes the ``Mailbox NS Agent`` work like a 40``partition``. The agent has an endless loop and waits for signals, calls FF-M 41API based on the parsing result on the communication messages. But there are 42still issues after looking closer to the requirements of the agent: 43 44- SPM treats FF-M Client API caller's ID as the client ID. While the mailbox NS 45 agent may represent multiple non-secure clients. Hence it needs to tell 46 SPM which non-secure client it is representing, and the default FF-M Client 47 API does not have such capability. 48- FF-M Client API blocks caller before the call is replied; while the 49 mailbox NS Agent needs to respond to the non-secure interrupts in time. 50 Blocking while waiting for a reply may cause the non-secure communication 51 message not to be handled in time. 52 53Extra design items need to be added to address the issues. 54 55************* 56Design Update 57************* 58The below figure shows the overall design to cover various component types. 59NS Agents are the implementation-defined components that provide FF-M compliant 60Client API to the non-secure clients. Hence from the view of the non-secure 61clients, the FF-M client API behaviour follows the FF-M definition. And NS 62Agent needs customization in SPM since it has extra requirements compared to 63a generic secure partition. 64 65.. figure:: /design_docs/media/mailbox_ns_agent1.* 66 :align: center 67 :name: fig-mailbox1 68 :width: 70% 69 70 Component types and the callable API set 71 72.. note:: 73 3 non-SPM component types here: FF-M-compliant Secure Partition 74 (aka ``partition``), Trustzone-based NS Agent (aka ``Trustzone NS Agent``) 75 and mailbox-based NS Agent (aka ``Mailbox NS Agent``). 76 ``Trustzone NS Agent`` is mentioned here for the comparison purpose. The 77 implementation details for this NS agent type is not introduced here. 78 79To make the programming model close to the FF-M compliance, the 80``Mailbox NS Agent`` is designed as: 81 82- Working like a standard Secure Partition under the IPC model, has one 83 single thread, can call FF-M standard API. 84- Having a manifest file to describe the attributes and resources and a 85 positive value ``Partition ID`` in the manifest. 86 87Services rely on the ``client_id`` to apply policy-checking, hence SPM 88needs to know which ``client_id`` the mailbox NS Agent is representing when 89mailbox API is calling Client API. The standard API treats the caller as the 90client of the service, which means that a specific API is required to 91support identifying the represented non-secure client. SPM sets the non-secure 92``client_id`` into the message right at the moment the message is 93going to be sent. Before this point, SPM performs the call based on the 94agent's ID. 95 96This specifc ``Agent API`` is non-blocking, unlike the standard FF-M Client 97APIs. This can improve the communication efficiency between NS clients and 98mailbox NS agents. With this mechanism, extra signals and APIs for message 99acknowledges are also required. 100 101.. note:: 102 A standard Secure Partition gets errors when calling the ``Agent API``. 103 104Updated programming interfaces 105============================== 106These Client APIs are expanded from the standard Client APIs: 107 108- ``agent_psa_connect()`` is extended from ``psa_connect()``. 109- ``agent_psa_close()`` is extended from ``psa_close()``. 110- ``agent_psa_call()`` is extended from ``psa_call()``. 111 112And to cooperate with the changed behaviour of these APIs, extra defined 113signals and types are also involved. 114 115.. note:: 116 Namespace ``agent`` is involved for NS Agent callable API; namespace ``tfm`` 117 is involved for TF-M specific concepts. Even though ``agent`` is TF-M 118 specific at the current stage, it is proposed to be a common concept for 119 general FF-M compliant implementations, hence assigning ``agent`` for 120 proposed API and data structures. 121 122Input and output vectors 123======================== 124When non-secure clients call ``psa_call()``, a mailbox message containing 125``psa_call()`` parameters is delivered into ``Mailbox NS Agent`` and the agent 126needs to extract parameters from the message and then call ``agent_psa_call()``. 127Revisit the ``psa_call()`` prototype to see the parameters: 128 129.. code-block:: c 130 131 psa_status_t psa_call(psa_handle_t handle, 132 int32_t type, 133 const psa_invec *in_vec, 134 size_t in_len, 135 psa_outvec *out_vec, 136 size_t out_len); 137 138Interface ``agent_psa_call()`` has 4 arguments only, to avoid ABI complexity 139when more than 4 arguments get involved. Then input and output vectors must 140be squashed into a new type to achieve this squashing. There are several 141scenarios to be considered: 142 143- In the shared-memory-based mailbox scheme, the non-secure interface layer 144 puts ``psa_invec`` and ``psa_outvec`` instances and their members in the 145 shared memory. The pointers of these items are delivered to 146 ``Mailbox NS Agent`` for the agent's referencing. All vectors are 147 non-secure in this case. 148- Still, in the shared-memory-based mailbox scheme, but this time not all 149 parameters are prepared by non-secure clients. Some of the items are 150 allocated by the agent. Secure and non-secure vectors get mixed in this case. 151- In the scheme that a memory address can not be delivered (A serial 152 interface e.g.), ``Mailbox NS Agent`` allocates ``psa_invec`` and 153 ``psa_outvec`` in local memory to collect pointers and sizes of received 154 buffers. All the vectors are secure in this case. 155 156Based on these scenarios, and the case that an agent might access depending 157services with the agent itself's identifier and secure vectors, a straight 158conclusion is that the memory source information - secure or non-secure - for 159input and output vectors and their pointing memories need to be indicated by 160the agent so that SPM can perform a memory check towards given vectors. 161 162Most of the SPE platforms have the capability to identify if a given memory 163pointer is secure or not, which makes this indication look duplicated. But the 164indication is necessary for these scenarios: 165 166- The SPE platform identifies the memory source by the address mapped into 167 SPE's address space, but this mapping happens inside the agent instead of 168 SPM. It is the agent who receives the mailbox data and maps it, so it knows 169 which addresses need mapping and the others do not. 170- The SPE platform just does not have such capability. The addresses from the 171 mailbox can be treated as non-secure always but there are cases that the 172 agent itself needs to access services with its own memory instead of 173 representing the non-secure clients. 174 175To cover the above mentioned scenarios, guidelines are listed for input and 176output vector processing: 177 178- The agent needs to tell SPM where vectors and descriptors come from, to 179 assist SPM performs a proper memory checking. The source information is 180 encoded in the parameter ``control``. 181- When SPE platforms have the capability to identify the memory sources, 182 platforms can decide whether to skip the indication or not, in the HAL. 183 184A composition type is created for packing the vectors: 185 186.. code-block:: c 187 188 struct client_params_t { 189 int32_t ns_client_id_stateless; 190 const psa_invec *p_invecs; 191 psa_outvec *p_outvecs; 192 }; 193 194``ns_client_id_stateless`` indicates the non-secure client id when the client 195is accessing a stateless service. This member is ignored if the target service 196is a connection-based one. 197 198.. note:: 199 The vectors and non-secure client ID are recorded in the internal handle. 200 Hence it is safe to claim ``client_params_t`` instance as local variable. 201 202Agent-specific Client API 203========================= 204``agent_psa_connect()`` and ``agent_psa_close()`` are the APIs added to support 205agent forwarding NS requests. 206 207.. code-block:: c 208 209 psa_handle_t agent_psa_connect(uint32_t sid, uint32_t version, 210 int32_t ns_client_id, const void *client_data); 211 psa_status_t agent_psa_close(psa_handle_t handle, int32_t ns_client_id); 212 213One extra parameter ``ns_client_id`` added to tell SPM which NS client the 214agent is representing when API gets called. It is recorded in the handle 215association data in SPM and requires to be a negative value; ZERO or positive 216values are invalid non-secure client IDs, SPM does not use these invalid IDs 217in the message. Instead, it puts the agent's ID into the messaging in this 218case. This mechanism can provide chances for the agents calling APIs for their 219own service accessing and API works asynchronously. 220 221As mentioned, the standard FF-M Client service accessing API are blocked until 222the IPC message gets replied to. While this API returns immediately without 223waiting for acknowledgement. Unless an error occurred, these agent-specific 224API returns PSA_SUCCESS always. The replies for these access requests are 225always fetched initiative by the agent with a ``psa_get()``. 226 227.. code-block:: c 228 229 psa_status_t agent_psa_call(psa_handle_t handle, uint32_t control, 230 const struct client_param_t *params, 231 const void *client_data_stateless); 232 233Compared to the standard ``psa_call()``, this API: 234 235- Squashes the ``psa_invec``, ``psa_outvec``, and ``ns_client_id_stateless`` 236 into parameter ``params``. 237- One extra parameter ``client_data_stateless`` for ``agent_psa_call()`` stands 238 for the auxiliary data added. This member is ignored for connection-based 239 services because ``agent_psa_connect()`` already assigned one in the 240 connected handle. 241- Has a composite argument ``control``. 242 243The encoding scheme for ``control``: 244 245====== ====== ================================================================ 246Bit(s) Name Description 247====== ====== ================================================================ 24827 NSIV `1`: Input vectors in non-secure memory. `0`: Secure memory. 24924-26 IVNUM Number of input vectors. 25019 NSOV `1`: Output vectors in non-secure memory. `0`: Secure memory. 25116-18 OVNUM Number of output vectors. 2520-15 type signed 16-bit service `type`. 253====== ====== ================================================================ 254 255.. note:: 256 ``control`` is a 32-bit unsigned integer and bits not mentioned in the 257 table are reserved for future usage. 258 259Agent-specific signal 260===================== 261To cooperate with the agent-specific API, one extra acknowledgement signal is 262defined: 263 264.. code-block:: c 265 266 #define ASYNC_MSG_REPLY (0x00000004u) 267 268This signal can be sent to agent type component only. An agent can call 269``psa_get()`` with this signal to get one acknowledged message. This signal is 270cleared when all queued messages for the agent have been retrieved using 271``psa_get()``. SPM assembles the information into agent provided message object. 272For the stateless handle, the internal handle object is freed after this 273``psa_get()`` call. The agent can know what kind of message is acknowledged by 274the ``type`` member in the ``psa_msg_t``, and the ``client_data`` passed in is 275put in member ``rhandle``. If no 'ASYNC_MSG_REPLY' signals pending, calling 276``psa_get()`` gets ``panic``. 277 278Code Example 279============ 280 281.. code-block:: c 282 283 /* 284 * The actual implementation can change this __customized_t freely, or 285 * discard this type and apply some in-house mechanism - the example 286 * here is to introduce how an agent works only. 287 */ 288 struct __customized_t { 289 int32_t type; 290 int32_t client_id; 291 psa_handle_t handle; 292 psa_handle_t status; 293 }; 294 295 void mailbox_main(void) 296 { 297 psa_signal_t signals; 298 psa_status_t status; 299 psa_msg_t msg; 300 struct client_param_t client_param; 301 struct __customized_t ns_msg; 302 303 while (1) { 304 signals = psa_wait(ALL, BLOCK); 305 306 if (signals & MAILBOX_INTERRUPT_SIGNAL) { 307 /* NS memory check needs to be performed. */ 308 __customized_platform_get_mail(&ns_msg); 309 310 /* 311 * MACRO 'SID', 'VER', 'NSID', 'INVEC_LEN', 'OUTVEC_LEN', and 312 * 'VECTORS' represent necessary information extraction from 313 * 'ns_msg', put MACRO names here and leave the details to the 314 * implementation. 315 */ 316 if (ns_msg.type == PSA_IPC_CONNECT) { 317 status = agent_psa_connect(SID(ns_msg), VER(ns_msg), 318 NSID(ns_msg), &ns_msg); 319 } else if (ns_msg.type == PSA_IPC_CLOSE) { 320 status = agent_psa_close(ns_msg.handle, NSID(ns_msg)); 321 } else { 322 /* Other types as call type and let API check errors. */ 323 client_param.ns_client_id_stateless = NSID(ns_msg); 324 325 /* 326 * Use MACRO to demonstrate two cases: local vector 327 * descriptor and direct descriptor forwarding. 328 */ 329 330 /* Point to vector pointers in ns_msg. */ 331 PACK_VECTOR_POINTERS(client_param, ns_msg); 332 status = agent_psa_call(ns_msg.handle, 333 PARAM_PACK(ns_msg.type, 334 INVEC_LEN(ns_msg), 335 OUTVEC_LEN(ns_msg))| 336 NSIV | NSOV, 337 &client_param, 338 &ns_msg); 339 } 340 /* 341 * The service access reply is always fetched by a later 342 * `psa_get` hence here only errors need to be dispatched. 343 */ 344 error_dispatch(status); 345 346 } else if (signals & ASYNC_MSG_REPLY) { 347 /* The handle is freed for stateless service after 'psa_get'. */ 348 status = psa_get(ASYNC_MSG_REPLY, &msg); 349 ms_msg = msg.rhandle; 350 ns_msg.status = status; 351 __customized_platform__send_mail(&ns_msg); 352 } 353 } 354 } 355 356.. note:: 357 ``__customized*`` API are implementation-specific APIs to be implemented by 358 the mailbox Agent developer. 359 360Customized manifest attribute 361============================= 362Three extra customized manifest attributes are added: 363 364=============== ==================================================== 365Name Description 366=============== ==================================================== 367ns_agent Indicate if manifest owner is an Agent. 368--------------- ---------------------------------------------------- 369client_id_base The minimum client ID value (<0) 370--------------- ---------------------------------------------------- 371client_id_limit The maximum client ID value (<0) 372=============== ==================================================== 373 374``client_id_base`` and ``client_id_limit`` are negative numbers. This means that 375``client_id_base <= client_id_limit``, but 376``abs(client_id_base) >= abs(client_id_limit)``. SPM can detect ID overlap when 377initialising secure partitions 378 379The Non-secure callers are expected to provide a negative (<0) client ID when 380calling PSA API. A uniform mapping is implemented across all the NS agents, 381where the mapping is defined as the following: 382 383+---------------------------------------------+---------------------+ 384|NS client ID |Transformed client ID| 385+=============================================+=====================+ 386| -1 | client_id_limit | 387+---------------------------------------------+---------------------+ 388| -2 | client_id_limit - 1 | 389+---------------------------------------------+---------------------+ 390| ... | 391+---------------------------------------------+---------------------+ 392|-(abs(client_id_limit)-abs(client_id_base)+1)| client_id_base | 393+---------------------------------------------+---------------------+ 394 395Any other IDs provided by the NSPE will result in PSA_ERROR_INVALID_ARGUMENT. 396 397*********************** 398Manifest tooling update 399*********************** 400The manifest for agents involves specific keys ('ns_agent' e.g.), these keys 401give hints about how to achieve out-of-FF-M partitions which might be abused 402easily by developers, for example, claim partitions as agents. Some 403restrictions need to be applied in the manifest tool to limit the general 404secure service development referencing these keys. 405 406.. note:: 407 The limitations can mitigate the abuse but can't prevent it, as developers 408 own all the source code they are working with. 409 410One mechanism: adding a confirmation in the partition list file. 411 412.. parsed-literal:: 413 414 "description": "Non-Secure Mailbox Agent", 415 "manifest": "${CMAKE_SOURCE_DIR}/secure_fw/partitions/ns_agent_mailbox/ns_agent_mailbox.yaml", 416 "non_ffm_attributes": "ns_agent", "other_option", 417 418``non_ffm_attributes`` tells the manifest tool that ``ns_agent`` is valid 419in ns_agent_mailbox.yaml. Otherwise, the manifest tool reports an error when a 420non-agent service abuses ns_agent in its manifest. 421 422*********************************** 423Runtime programming characteristics 424*********************************** 425 426Mailbox agent shall not be blocked by Agent-specific APIs. It can be blocked when: 427 428- It is calling standard PSA Client APIs. 429- It is calling ``psa_wait()``. 430 431IDLE processing 432=============== 433Only ONE place is recommended to enter IDLE. The place is decided based on the 434system topologies: 435 436- If there is one Trustzone-based NSPE, this NSPE is the recommended place no 437 matter how many mailbox agents there are in the system. 438- If there are only mailbox-based NSPEs, entering IDLE can happen in 439 one of the mailbox agents. 440 441The solution is: 442 443- An IDLE entering API is provided in SPRTL. 444- A partition without specific flag can't call this API. 445- The manifest tooling counts the partitions with this specific flag, and 446 assert errors when multiple instances are found. 447 448-------------- 449 450*Copyright (c) 2022-2024, Arm Limited. All rights reserved.* 451*Copyright (c) 2023 Cypress Semiconductor Corporation (an Infineon company) 452or an affiliate of Cypress Semiconductor Corporation. All rights reserved.* 453
