1Testing strategy for `MBEDTLS_USE_PSA_CRYPTO` 2============================================= 3 4This document records the testing strategy used so far in implementing 5`MBEDTLS_USE_PSA_CRYPTO`. 6 7 8General considerations 9---------------------- 10 11There needs to be at least one build in `all.sh` that enables 12`MBEDTLS_USE_PSA_CRYPTO` and runs the full battery of tests; currently that's 13ensured by the fact that `scripts/config.py full` enables 14`MBEDTLS_USE_PSA_CRYPTO`. There needs to be at least one build with 15`MBEDTLS_USE_PSA_CRYPTO` disabled (as long as it's optional); currently that's 16ensured by the fact that it's disabled in the default config. 17 18Generally, code review is enough to ensure that PSA APIs are indeed used where 19they should be when `MBEDTLS_USE_PSA_CRYPTO` is enabled. 20 21However, when it comes to TLS, we also have the option of using debug messages 22to confirm which code path is taken. This is generally unnecessary, except when 23a decision is made at run-time about whether to use the PSA or legacy code 24path. (For example, for record protection, previously (until 3.1), some ciphers were supported 25via PSA while some others weren't, with a run-time fallback. In this case, it's 26good to have a debug message checked by the test case to confirm that the 27right decision was made at run-time, i. e. that we didn't use the fallback for 28ciphers that are supposed to be supported.) 29 30 31New APIs meant for application use 32---------------------------------- 33 34For example, `mbedtls_pk_setup_opaque()` is meant to be used by applications 35in order to create PK contexts that can then be passed to existing TLS and 36X.509 APIs (which remain unchanged). 37 38In that case, we want: 39 40- unit testing of the new API and directly-related APIs - for example: 41 - in `test_suite_pk` we have a new test function `pk_psa_utils` that exercises 42 `mbedtls_pk_setup_opaque()` and checks that various utility functions 43 (`mbedtls_pk_get_type()` etc.) work and the functions that are expected to 44 fail (`mbedtls_pk_verify()` etc) return the expected error. 45 - in `test_suite_pk` we modified the existing `pk_psa_sign` test function to 46 check that signature generation works as expected 47 - in `test_suite_pkwrite` we should have a new test function checking that 48 exporting (writing out) the public part of the key works as expected and 49 that exporting the private key fails as expected. 50- integration testing of the new API with each existing API which should 51 accepts a context created this way - for example: 52 - in `programs/ssl/ssl_client2` a new option `key_opaque` that causes the 53 new API to be used, and one or more tests in `ssl-opt.sh` using that. 54 (We should have the same server-side.) 55 - in `test_suite_x509write` we have a new test function 56 `x509_csr_check_opaque()` checking integration of the new API with the 57 existing `mbedtls_x509write_csr_set_key()`. (And also 58 `mbedtls_x509write_crt_set_issuer_key()` since #5710.) 59 60For some APIs, for example with `mbedtls_ssl_conf_psk_opaque()`, testing in 61`test_suite_ssl` was historically not possible, so we only have testing in 62`ssl-opt.sh`. 63 64New APIs meant for internal use 65------------------------------- 66 67For example, `mbedtls_cipher_setup_psa()` (no longer used, soon to be 68deprecated - #5261) was meant to be used by the TLS layer, but probably not 69directly by applications. 70 71In that case, we want: 72 73- unit testing of the new API and directly-related APIs - for example: 74 - in `test_suite_cipher`, the existing test functions `auth_crypt_tv` and 75 `test_vec_crypt` gained a new parameter `use_psa` and corresponding test 76 cases 77- integration testing: 78 - usually already covered by existing tests for higher-level modules: 79 - for example simple use of `mbedtls_cipher_setup_psa()` in TLS is already 80 covered by running the existing TLS tests in a build with 81 `MBEDTLS_USA_PSA_CRYPTO` enabled 82 - however if use of the new API in higher layers involves more logic that 83 use of the old API, specific integrations test may be required 84 - for example, the logic to fall back from `mbedtls_cipher_setup_psa()` to 85 `mbedtls_cipher_setup()` in TLS is tested by `run_test_psa` in 86 `ssl-opt.sh`. 87 88Internal changes 89---------------- 90 91For example, use of PSA to compute the TLS 1.2 PRF. 92 93Changes in this category rarely require specific testing, as everything should 94be already be covered by running the existing tests in a build with 95`MBEDTLS_USE_PSA_CRYPTO` enabled; however we need to make sure the existing 96test have sufficient coverage, and improve them if necessary. 97 98However, if additional logic is involved, or there are run-time decisions about 99whether to use the PSA or legacy code paths, specific tests might be in order. 100
