Lines Matching +full:twister +full:- +full:out
3 Twister Status
6 What is a Twister Status?
9 Twister Status formulates the current state of
11 - ``Harness``
12 - ``TestCase``
13 - ``TestSuite``
14 - ``TestInstance``
16 in a comprehensive and easy-to understand way.
18 of Instances and Cases after the conclusion of their Twister runs.
28 ``Harness`` is a Python class inside Twister that allows us
29 to capture and analyse output from a program external to Twister.
41 One can modify Twister's behaviour on a per-Suite basis via ``testcase.yaml`` files.
43 for it to make sense to treat them all the same by Twister.
48 Twister typically reports its results for Instances,
56 Possible Twister Statuses
59 .. list-table:: Twister Statuses
61 :header-rows: 1
63 * - In-code
64 - In-text
65 - Description
66 - Suite
67 - Case
68 * - FILTER
69 - filtered
70 - A static or runtime filter has eliminated the test from the list of tests to use.
71 - ✓
72 - ✓
73 * - NOTRUN
74 - not run
75 - The test wasn't run because it was not runnable in current configuration.
77 - ✓
78 - ✓
79 * - BLOCK
80 - blocked
81 - The test was not run because of an error or crash in the test suite.
82 - ✕
83 - ✓
84 * - SKIP
85 - skipped
86 - Test was skipped by some other reason than previously delineated.
87 - ✓
88 - ✓
89 * - ERROR
90 - error
91 - The test produced an error in running the test itself.
92 - ✓
93 - ✓
94 * - FAIL
95 - failed
96 - The test produced results different than expected.
97 - ✓
98 - ✓
99 * - PASS
100 - passed
101 - The test produced results as expected.
102 - ✓
103 - ✓
105 **In-code** and **In-text** are the naming contexts of a given status:
106 the former is rather internal for Twister and appears in logs,
111 There are two more Statuses that are internal to Twister.
113 ``STARTED`` (Indicating an in-progress Case).
114 Those should not appear in the final Twister report.
115 Appearance of those non-terminal Statuses in one's report files indicates a problem with Twister.
121 .. list-table:: Case and Suite Status combinations
124 :header-rows: 1
125 :stub-columns: 1
127 * - ↓ Case\\Suite →
128 - FILTER
129 - ERROR
130 - FAIL
131 - PASS
132 - NOTRUN
133 - SKIP
134 * - FILTER
135 - ✓
136 - ✕
137 - ✕
138 - ✕
139 - ✕
140 - ✕
141 * - ERROR
142 - ✕
143 - ✓
144 - ✕
145 - ✕
146 - ✕
147 - ✕
148 * - BLOCK
149 - ✕
150 - ✓
151 - ✓
152 - ✕
153 - ✕
154 - ✕
155 * - FAIL
156 - ✕
157 - ✓
158 - ✓
159 - ✕
160 - ✕
161 - ✕
162 * - PASS
163 - ✕
164 - ✓
165 - ✓
166 - ✓
167 - ✕
168 - ✕
169 * - NOTRUN
170 - ✕
171 - ✕
172 - ✕
173 - ✕
174 - ✓
175 - ✕
176 * - SKIP
177 - ✕
178 - ✓
179 - ✓
180 - ✓
181 - ✕
182 - ✓
184 ✕ indicates that such a combination should not happen in a proper Twister run. In other words,
191 -------------------------------------------
195 out of a given Twister run. Thus, any Case within it should also have such a status.
212 As runnability is decided on a per-Suite basis, only ``NOTRUN`` is applicable for its Cases.