README
1
2Utilities to manage the test suite
3----------------------------------
4
5
6Do utility
7
8This is the test suite command dispatcher. It allows the user to
9compile/install the AML tests, run them, and compare results of runs.
10The output log of tests and results of processing the test logs are
11saved in the aslts/tmp/RESULTS automatically generated directory.
12
13Run Do without parameters to print out its Usage.
14
15The simplest sequence of actions to deal with the test suite
16is shown in the example below.
17
18
19 1. Create the following environment variables:
20
21 ASL - pathname of iASL compiler
22 acpiexec - pathname of AcpiExec utility
23 ASLTSDIR - pathname of root directory of aslts test suite
24
25 2. Add the following directory to the PATH variable:
26
27 aslts/bin
28
29 If necessary, convert all scripts in the aslts/bin directory to unix
30 line endings:
31
32 > d2u aslts/bin/*
33
34 3. To configure the required/desired set of test cases and the modes of
35 the execution runs, manually edit the settings file:
36
37 aslts/bin/settings
38
39 4. Usage message:
40
41 > cd aslts/bin
42 > Do
43
44 The utility prints out the usage message.
45
46 5. Compile and install all AML test cases:
47
48 > Do 0 aslts
49
50 This command will compile and install all of the AML test cases
51 under the tmp directory:
52
53 aslts/tmp/aml
54
55 6. Run all configured test cases in all modes:
56
57 > Do 1
58
59 This command will run all the specified test cases in the specified
60 modes. The possible modes are:
61
62 32-bit normal mode (AML interpreter slack mode disabled)
63 64-bit normal mode
64 32-bit slack mode (AML interpreter slack mode enabled)
65 64-bit slack mode
66
67 7. Examine the results:
68
69 a) See the summary file for test execution information:
70
71 aslts/tmp/RESULTS/<date.time.acpica_version>/Summary
72
73 b) Compare the obtained multi-result with the previous benchmark
74 multi-result for to catch regressions (the 'Do 2' command below).
75
76 8. Compare the results to previous test executions:
77
78 > Do 2
79
80 This command will compare the current test execution with the
81 previous one. Also used for comparing any two particular
82 multi-results.
83
84 > Do 2 <first multi-result directory> <second multi-result directory>
85
86 The <first multi-result directory> contains the results from one of
87 test suite executions. The <second multi-result directory> contains
88 the results of a previous test suite execution.
89
90 The 'Do 2' command will compare the results of both test suite
91 executions and report any differences/regressions.
92
93 The result directories are contained in aslts/tmp/RESULTS, and
94 have names of the form: <date.time.acpica_version>
95
96 Example:
97
98 > cd aslts
99 > Do 2 tmp/RESULTS/20090325.151601.20090320 tmp/RESULTS/20090325.150744.20090320
100
101
102Do 2 output:
103============
104
1051. Summary information related to a particular test case:
106
107 echo "|$2|$cnt|$pass|$fail|$skip|$start|$finish|$total|$outstand0|$blck|$memtotal|$max0|$max1|$max2|$max3|$max4|$max5|$out0|$out1|$out2|$out3|$out4|$out5|$outstand1|$LargeRefCount|$memtotalbytes|$exceptionsnum|" >> "$3"
108
109outstand0 - Outstanding allocations of the particular test case
110 Example from arithmetic test case log:
111
112 Outstanding: 0x11 allocations after execution
113
114outstand1 - Outstanding allocations of the particular test case
115 Example from reference test case log:
116
117 ACPI Error (uttrack-0719): 39(27) Outstanding allocations [20090320]
118
119memtotal - Total memory in bytes consumed during execution of the
120 particular test case
121start - Time when test case started
122finish - Time when test case finished
123total - Duration of test case execution
124
125
1262. Summary information related to all test cases that have been run for a
127 particular test mode (32-bit normal, etc..):
128
129 echo "|TOTAL|$outstand0|$memtotal|${MODES_TIMES[$2]}|$outstand1|$LargeRefCount|$memtotalbytes|" >> "$path"
130
131totaloutstand00 - Summary of outstand0 of all test cases
132totalmem0 - Summary of memtotal of all test cases
133totaltime0 - All the test cases execution duration
134totaloutstand01 - Summary of outstand1 of all test cases
135