1              Thread-Metric RTOS Test Suite
2
3
41. Thread-Metric Test Suite
5
6The Thread-Metric test suite consists of 8 distinct RTOS
7tests that are designed to highlight commonly used aspects
8of an RTOS. The test measures the total number of RTOS events
9that can be processed during a specific timer interval. A 30
10second time interval is recommended.
11
121.1. Basic Processing Test
13
14This is the baseline test consisting of a single thread. This
15should execute the same on every operating system. Test values
16from testing with different RTOS products should be scaled
17relative to the difference between the values of this test.
18
191.2. Cooperative Scheduling Test
20
21This test consists of 5 threads created at the same priority that
22voluntarily release control to each other in a round-robin fashion.
23Each thread will increment its run counter and then relinquish to
24the next thread.  At the end of the test the counters will be verified
25to make sure they are valid (should all be within 1 of the same
26value). If valid, the numbers will be summed and presented as the
27result of the cooperative scheduling test.
28
291.3. Preemptive Scheduling Test
30
31This test consists of 5 threads that each have a unique priority.
32In this test, all threads except the lowest priority thread are
33left in a suspended state. The lowest priority thread will resume
34the next highest priority thread.  That thread will resume the
35next highest priority thread and so on until the highest priority
36thread executes. Each thread will increment its run count and then
37call thread suspend.  Eventually the processing will return to the
38lowest priority thread, which is still in the middle of the thread
39resume call. Once processing returns to the lowest priority thread,
40it will increment its run counter and once again resume the next
41highest priority thread - starting the whole process over once again.
42
431.4. Interrupt Processing Test
44
45This test consists of a single thread. The thread will cause an
46interrupt (typically implemented as a trap), which will result in
47a call to the interrupt handler. The interrupt handler will
48increment a counter and then post to a semaphore. After the
49interrupt handler completes, processing returns to the test
50thread that initiated the interrupt. The thread then retrieves
51the semaphore set by the interrupt handler, increments a counter
52and then generates another interrupt.
53
541.5. Interrupt Preemption Processing Test
55
56This test is similar to the previous interrupt test. The big
57difference is the interrupt handler in this test resumes a
58higher priority thread, which causes thread preemption.
59
601.6. Message Processing Test
61
62This test consists of a thread sending a 16 byte message to a
63queue and retrieving the same 16 byte message from the queue.
64After the send/receive sequence is complete, the thread will
65increment its run counter.
66
671.4. Synchronization Processing Test
68
69This test consists of a thread getting a semaphore and then
70immediately releasing it. After the get/put cycle completes,
71the thread will increment its run counter.
72
731.5. RTOS Memory allocation
74
75This test consists of a thread allocating a 128-byte block and
76releasing the same block. After the block is released, the thread
77will increment its run counter.
78
79
802. Thread-Metric Source Code
81
82The Thread-Metric source code may be freely used, providing its use
83complies with the stated copyright banner in each source file. For
84simplicity, each test is defined in its own file. Furthermore, each
85test is written in a generic C fashion. Providing the RTOS vendor
86completes the porting layer file with calls to its services, the
87tests are ready to run on any RTOS.
88
89The source code to the Thread-Metric test suite is organized into
90the following files:
91
92            File                                    Meaning
93
94tm_api.h                                    API and constants for test suite
95tm_basic_processing_test.c                  Basic test for determining board
96                                              processing capabilities
97tm_cooperative_scheduling_test.c            Cooperative scheduling test
98tm_preemptive_scheduling_test.c             Preemptive scheduling test
99tm_interrupt_processing_test.c              No-preemption interrupt processing
100                                              test
101tm_interrupt_preemption_processing_test.c   Interrupt preemption processing
102                                              test
103tm_message_processing_test.c                Message exchange processing test
104tm_synchronization_processing_test.c        Semaphore get/put processing test
105tm_memory_allocation_test.c                 Basic memory allocation test
106tm_porting_layer.h                          Port specific information, including
107                                              in-line assembly instruction to
108                                              cause an interrupt for the
109                                              interrupt processing tests
110tm_porting_layer_template.c                 Generic template for RTOS porting
111                                              layer
112tm_porting_layer_threadx.c                  Specific porting layer source
113                                              code for ThreadX
114
1152.1 Porting Layer
116
117As for the porting layer defined in tm_porting_layer_template.c, this file contain
118shell services of the generic RTOS services used by the actual tests. The
119shell services provide the mapping between the tests and the underlying RTOS.
120The following generic API's are required to map any RTOS to the performance
121measurement tests:
122
123
124    void  tm_initialize(void (*test_initialization_function)(void));
125
126    This function is typically called by the application from its
127    main() function. It is responsible for providing all the RTOS
128    initialization, calling the test initialization function as
129    specified, and then starting the RTOS.
130
131    int  tm_thread_create(int thread_id, int priority, void (*entry_function)(void));
132
133    This function creates a thread of the specified priority where 1 is
134    the highest and 16 is the lowest.  If successful, TM_SUCCESS
135    returned. If an error occurs, TM_ERROR is returned. The created thread
136    is not started.
137
138    int  tm_thread_resume(int thread_id);
139
140    This function resumes the previously created thread specified by
141    thread_id. If successful, a TM_SUCCESS is returned.
142
143    int  tm_thread_suspend(int thread_id);
144
145    This function suspend the previously created thread specified by
146    thread_id. If successful, a TM_SUCCESS is returned.
147
148    void  tm_thread_relinquish(void);
149
150    This function lets all other threads of same priority execute
151    before the calling thread runs again.
152
153    void  tm_thread_sleep(int seconds);
154
155    This function suspends the calling thread for the specified
156    number of seconds.
157
158    int  tm_queue_create(int queue_id);
159
160    This function creates a queue with a capacity to hold at least
161    one 16-byte message. If successful, a TM_SUCCESS is returned.
162
163    int  tm_queue_send(int queue_id, unsigned long *message_ptr);
164
165    This function sends a message to the previously created queue.
166    If successful, a TM_SUCCESS is returned.
167
168    int  tm_queue_receive(int queue_id, unsigned long *message_ptr);
169
170    This function receives a message from the previously created
171    queue. If successful, a TM_SUCCESS is returned.
172
173    int  tm_semaphore_create(int semaphore_id);
174
175    This function creates a binary semaphore. If successful, a
176    TM_SUCCESS is returned.
177
178    int  tm_semaphore_get(int semaphore_id);
179
180    This function gets the previously created binary semaphore.
181    If successful, a TM_SUCCESS is returned.
182
183    int  tm_semaphore_put(int semaphore_id);
184
185    This function puts the previously created binary semaphore.
186    If successful, a TM_SUCCESS is returned.
187
188    int  tm_memory_pool_create(int pool_id);
189
190    This function creates a memory pool able to satisfy at least one
191    128-byte block of memory. If successful, a TM_SUCCESS is returned.
192
193    int  tm_memory_pool_allocate(int pool_id, unsigned char **memory_ptr);
194
195    This function allocates a 128-byte block of memory from the
196    previously created memory pool. If successful, a TM_SUCCESS
197    is returned along with the pointer to the allocated memory
198    in the "memory_ptr" variable.
199
200    int  tm_memory_pool_deallocate(int pool_id, unsigned char *memory_ptr);
201
202    This function releases the previously allocated 128-byte block
203    of memory. If successful, a TM_SUCCESS is returned.
204
205
2062.2 Porting Requirements
207
208The following requirements are made in order to ensure fair benchmarks
209are achieved on each RTOS performing the test:
210
211    1. Time period should be 30 seconds. This will ensure the printf
212       processing in the reporting thread is insignificant.
213
214    2. The porting layer services are implemented inside of
215       tm_porting_layer_[RTOS].c and NOT as macros.
216
217    3. The tm_thread_sleep service is implemented by a 10ms RTOS
218       periodic interrupt source.
219
220    4. Locking regions of the tests and/or the RTOS in cache is
221       not allowed.
222
223    5. The Interrupt Processing and Interrupt Preemption Processing tests
224       require an instruction that generates an interrupt. Please refer
225       to tm_porting_layer.h for an example implementation.
226
227
228