1
2Utilities to manage the test suite
3----------------------------------
4
5
6Do utility
7
8This is the test suite command dispatcher. It allows the user to
9compile/install the AML tests, run them, and compare results of runs.
10The output log of tests and results of processing the test logs are
11saved in the aslts/tmp/RESULTS automatically generated directory.
12
13Run Do without parameters to print out its Usage.
14
15The simplest sequence of actions to deal with the test suite
16is shown in the example below.
17
18
19   1. Create the following environment variables:
20
21      ASL      - pathname of iASL compiler
22      acpiexec - pathname of AcpiExec utility
23      ASLTSDIR - pathname of root directory of aslts test suite
24
25   2. Add the following directory to the PATH variable:
26
27        aslts/bin
28
29      If necessary, convert all scripts in the aslts/bin directory to unix
30      line endings:
31
32        > d2u aslts/bin/*
33
34   3. To configure the required/desired set of test cases and the modes of
35      the execution runs, manually edit the settings file:
36
37        aslts/bin/settings
38
39   4. Usage message:
40
41        > cd aslts/bin
42        > Do
43
44        The utility prints out the usage message.
45
46   5. Compile and install all AML test cases:
47
48        > Do 0 aslts
49
50        This command will compile and install all of the AML test cases
51        under the tmp directory:
52
53            aslts/tmp/aml
54
55   6. Run all configured test cases in all modes:
56
57        > Do 1
58
59        This command will run all the specified test cases in the specified
60        modes. The possible modes are:
61
62            32-bit normal mode (AML interpreter slack mode disabled)
63            64-bit normal mode
64            32-bit slack mode (AML interpreter slack mode enabled)
65            64-bit slack mode
66
67   7. Examine the results:
68
69        a) See the summary file for test execution information:
70
71            aslts/tmp/RESULTS/<date.time.acpica_version>/Summary
72
73        b) Compare the obtained multi-result with the previous benchmark
74           multi-result for to catch regressions (the 'Do 2' command below).
75
76   8. Compare the results to previous test executions:
77
78        > Do 2
79
80        This command will compare the current test execution with the
81        previous one. Also used for comparing any two particular
82        multi-results.
83
84        > Do 2 <first multi-result directory> <second multi-result directory>
85
86        The <first multi-result directory> contains the results from one of
87        test suite executions. The <second multi-result directory> contains
88        the results of a previous test suite execution.
89
90        The 'Do 2' command will compare the results of both test suite
91        executions and report any differences/regressions.
92
93        The result directories are contained in aslts/tmp/RESULTS, and
94        have names of the form: <date.time.acpica_version>
95
96        Example:
97
98        > cd aslts
99        > Do 2 tmp/RESULTS/20090325.151601.20090320 tmp/RESULTS/20090325.150744.20090320
100
101
102Do 2 output:
103============
104
1051. Summary information related to a particular test case:
106
107   echo "|$2|$cnt|$pass|$fail|$skip|$start|$finish|$total|$outstand0|$blck|$memtotal|$max0|$max1|$max2|$max3|$max4|$max5|$out0|$out1|$out2|$out3|$out4|$out5|$outstand1|$LargeRefCount|$memtotalbytes|$exceptionsnum|" >> "$3"
108
109outstand0 - Outstanding allocations of the particular test case
110            Example from arithmetic test case log:
111
112                Outstanding: 0x11 allocations after execution
113
114outstand1 - Outstanding allocations of the particular test case
115            Example from reference test case log:
116
117                ACPI Error (uttrack-0719): 39(27) Outstanding allocations [20090320]
118
119memtotal  - Total memory in bytes consumed during execution of the
120            particular test case
121start     - Time when test case started
122finish    - Time when test case finished
123total     - Duration of test case execution
124
125
1262. Summary information related to all test cases that have been run for a
127   particular test mode (32-bit normal, etc..):
128
129   echo "|TOTAL|$outstand0|$memtotal|${MODES_TIMES[$2]}|$outstand1|$LargeRefCount|$memtotalbytes|"  >> "$path"
130
131totaloutstand00 - Summary of outstand0 of all test cases
132totalmem0       - Summary of memtotal of all test cases
133totaltime0      - All the test cases execution duration
134totaloutstand01 - Summary of outstand1 of all test cases
135