Lines Matching full:test
1 # TEST FRAMEWORK
36 **Now the test suite you want to run can be selected:**
40 Each time you want to change the test suite to run, you need to execute this function. No need to r…
42 Note that if the test suite is part of the half float tests, then you'll need to do instead:
46 **Building the test framework:**
73 For instance, to run the test on the virtual hardware for Corstone 300, if you have the Arm MDK ins…
75 …_SSE-300_Ethos-U55.exe -f configs\ARM_VHT_Corstone_300_config.txt Objects\test.Release+VHT-Corston…
85 Requirements for the test framework.
87 ### Test descriptions
93 #### R2 : The test description should support a hierarchy
97 #### R3 : A test shall be uniquely identified
98 We need a way to identify in an unique way each test to ensure traceability and enable to create
99 history of test results and benchmark.
104 #### R5 : The test description shall list the test patterns and input patterns required by the tests
113 For instance, if our test is dependent on a vector size, we may want to compute a linear regression
116 But, our test may also depend on another parameter B which is not interesting us in the regression.…
120 #### R8 : The concept of a test suite shall be supported.
121 A test suite is a set of tests packaged with some data.
123 ### Test execution
125 … device under tests (DUT) as the place where the function to test is executed. But the test itself…
129 A test should start (as far as possible) in a clean state. There should not be interferences betwee…
131 #### R10 : The test may be run on the DUT or on the host
139 #### R14 : Summary of test execution shall support several formats
142 #### R15 : One should not assume the test environment on the DUT has access to IOs.
149 ### Test description
151 A test description file is defined with a specific syntax to support R1 to R8.
157 group DSP Test {
166 For the test suite, the script is generating a partial implementation since a test suite is contain…
173 A test suite is more complex than a group since it contains the description of the tests and relate…
175 #### Test suite
184 Test arm_add_f32:test_add_f32
193 A test is requiring input patterns, reference patterns and outputs (to be compared to the reference…
194 Since the test must not know where is the data and how to get it, this information is provided in t…
196 So, the test suite would be:
208 Test arm_add_f32:test_add_f32
251 Once parameters have been described, we need a way to feed parameter values to a test.
253 …meter file. Problem of a parameter file when it has to be included in the test (C array) is that i…
275 Test A:testA -> PARAM3_ID
321 A disabled group/suite/test is not executed (and its code not generated for group/suite).
322 Using disabled for tests is allowing to disable a test without changing the test ID of following te…
328 The details of the APIs are in Test.h
333 According to R13 , the test may be controlled on the DUT or from an external host.
339 The runner is running a test and for benchmark measuring the cycles.
352 Patterns is the interface to patterns and output from the test point of view.
358 … be able to disable tests done on the DUT and dump the output so that the test itself can be done …
360 There are 3 modes, Test only, Dump only, Test and dump.
364 But it means that no test using patterns shoudl be used in the middle of the test or some part of i…
388 ### Generate the test patterns in Patterns folder
390 We have archived lot of test patterns on github. So this step is needed only if you write new test …
410 the parsed data structure. It is done because parsing a big test description file is quite slow.
412 So, it is needed to be done only once or if you modify the test description file.
414 Then, the tests can be processed to configure the test environment with
431 You can add a test ID to specify that you wan to run only a specific test in the suite:
437 The very first time you configure the test framework, you'll need to generate C files for all the t…
439 …nd will include some source files even if they are not needed for a given test suite. So those fil…
462 You can then run the executable on Virtual Hardware. For instance, to run the test on the virtual h…
466 Objects\test.Release+FVP_M55.axf > results.txt`
480 Some cycles are displayed with the test status (passed or failed). **Don't trust** those cycles for…
503 Each test is uniquely identified with the CATEGORY and test ID (ID in the suite).
539 of the tests to run and the test pattern samples to be used.
561 and of course, the test must contain a line to dump the outputs.
563 In DumpOnly mode, reference patterns are not loaded and the test assertions are "failing" but repor…
565 So, if a test is in the middle of useful code, some part of the code will not execute.
567 As consequence, if you intend to use the DumpOnly mode, you must ensure that all test assertions ar…
568 end of your test.
577 * Choose a test Runner (IORunner)
585 For a test suite MyClass, the scripts are generating an include file MyClass_decl.h
592 #include "Test.h"
603 // Definitions of the patterns you have in the test description file
604 // for this test suite
622 // Implementation of your test
657 // We can load different pattern or length according to the test ID
673 In tearDown we have to clean the test. No need to free the buffer since the memory manager will do …
756 * The corresponding float16_t tests are put in a different test file desc_f16.txt
757 * Code for those float16_t test is not built when ac5.cmake toolchain is used