1\page mipi_syst_catgen_page Automated Catalog Generation
2
3[TOC]
4
5Catalog Example {#mipi_syst_catgen_sample}
6==============================================================================
7The distribution includes an example application that uses catalog
8messages generated by the  [MIPI_SYST_CATPRINTF](@ref PrintfApi),
9[MIPI_SYST_CATALOG64](@ref CatAPI64) or  [MIPI_SYST_CATALOG32](@ref CatAPI32)
10families of instrumentation API calls. The example also includes the necessary
11configuration to support the automated decode collateral generation tool. This
12tool scans the source code and extracts the catalog decode information into
13collateral needed by SyS-T decoders.
14
15The example is stored under `examples/client` in the source code distribution.
16
17Decode Collateral Creation Tool {#mipi_syst_catgen_tool}
18==============================================================================
19The project includes a PERL based collateral generator that automates the
20creation and updating of SyS-T decode collateral in XML format. This
21generator parses the catalog instrumentation points by scanning the
22client source code. The tool can be embedded into a software build process to
23keep software builds and decode collateral updated at the same time. The tool
24is stored in the ```collateral/generator``` directory of the project with the
25name ```syst_cgen.pl```.
26
27Dependencies {#mipi_syst_catgen_deps}
28------------------------------------------------------------------------------
29The generator is written in PERL and requires a PERL installation with the
30following optional modules installed.
31 * String::Escape
32 * Xml::Simple
33
34Refer to the documentation for your PERL installation on module installation.
35On Linux, the following commands can be used:
36
37```
38$ perl -MCPAN -e 'install XML::Simple'
39$ perl -MCPAN -e 'install String::Escape'
40```
41
42Collateral Generation Process {#mipi_syst_catgen_process}
43------------------------------------------------------------------------------
44The collateral generator takes a SyS-T collateral template and
45a configuration file as input. The configuration file defines the locations
46and file extensions of the source files to be scanned and how the catalog
47message calls
48inside the source code are named. The tool can then detect the catalog
49calls, and extract the format strings, source locations, and
50catalog IDs to update the collateral template file. The result is a
51new collateral file that matches the actual state of the source code.
52
53Catalog Generation Example {#mipi_syst_catgen_example_generation}
54------------------------------------------------------------------------------
55The client application in ```example/client``` uses various catalog calls.
56It  provides a configuration file for ```syst_cgen.pl``` to detect
57the SyS-T catalog message calls, and a collateral template file that is
58updated by the generator. It is executed in the following way:
59
60```
61$ perl ../../collateral/generator/syst_cgen.pl -config collateral_config.xml
62syst_cgen.pl: Parsing: ./othersource.c
63syst_cgen.pl: Add ./othersource.c with file id 1 to file catalog
64syst_cgen.pl: Parsing finished: ./othersource.c, found 127 call(s)
65syst_cgen.pl: Parsing: ./systclient.c
66syst_cgen.pl: Add ./systclient.c with file id 2 to file catalog
67syst_cgen.pl: Parsing finished: ./systclient.c, found 4 call(s)
68syst_cgen.pl: Generating XML structure
69syst_cgen.pl: Loaded template collateral file template.xml
70syst_cgen.pl: Generating XML structure finished
71syst_cgen.pl: Writing XML file: generated_catalog.xml
72syst_cgen.pl: Writing XML file finished
73```
74Catalog Generator Configuration Example {#mipi_syst_catgen_example_config}
75------------------------------------------------------------------------------
76The following XML file shows the example configuration file from the client
77example. The file lists the SyS-T catalog instrumentation call patterns and
78can be reused for other clients. Only the options at the beginning may require
79adaptations.
80
81```{.xml}
82<CatalogGenerator>
83    <CatalogConfigs>
84        <CatalogConfig option="catalog" value="generated_catalog.xml" />
85        <CatalogConfig option="template" value="template.xml" />
86        <CatalogConfig option="indentation" value="4" />
87        <CatalogConfig option="nolocation"  value="false"/>
88        <CatalogConfig option="src" value="." />
89    </CatalogConfigs>
90    <SrcFilePatterns>
91        <SrcFilePattern Pattern="*.{cpp,c,h}" />
92    </SrcFilePatterns>
93    <CatalogCalls>
94        <Catalog32>
95            <CatalogCall Name="MIPI_SYST_HASH" Algorithm="hash65599" IdParamIdx="2" StringParamIdx="1" />
96
97            <CatalogCall Name="MIPI_SYST_CATPRINTF32" IdParamIdx="3" StringParamIdx="4"    />
98            <CatalogCall Name="MIPI_SYST_CATPRINTF32_0" IdParamIdx="3" StringParamIdx="4"  />
99            <CatalogCall Name="MIPI_SYST_CATPRINTF32_1" IdParamIdx="3" StringParamIdx="4"  />
100            <CatalogCall Name="MIPI_SYST_CATPRINTF32_2" IdParamIdx="3" StringParamIdx="4"  />
101            <CatalogCall Name="MIPI_SYST_CATPRINTF32_3" IdParamIdx="3" StringParamIdx="4"  />
102            <CatalogCall Name="MIPI_SYST_CATPRINTF32_4" IdParamIdx="3" StringParamIdx="4"  />
103            <CatalogCall Name="MIPI_SYST_CATPRINTF32_5" IdParamIdx="3" StringParamIdx="4"  />
104            <CatalogCall Name="MIPI_SYST_CATPRINTF32_6" IdParamIdx="3" StringParamIdx="4"  />
105        </Catalog32>
106
107        <Catalog64>
108            <CatalogCall Name="MIPI_SYST_CATPRINTF64" IdParamIdx="3" StringParamIdx="4" />
109            <CatalogCall Name="MIPI_SYST_CATPRINTF64_0" IdParamIdx="3" StringParamIdx="4"  />
110            <CatalogCall Name="MIPI_SYST_CATPRINTF64_1" IdParamIdx="3" StringParamIdx="4"  />
111            <CatalogCall Name="MIPI_SYST_CATPRINTF64_2" IdParamIdx="3" StringParamIdx="4"  />
112            <CatalogCall Name="MIPI_SYST_CATPRINTF64_3" IdParamIdx="3" StringParamIdx="4"  />
113            <CatalogCall Name="MIPI_SYST_CATPRINTF64_4" IdParamIdx="3" StringParamIdx="4"  />
114            <CatalogCall Name="MIPI_SYST_CATPRINTF64_5" IdParamIdx="3" StringParamIdx="4"  />
115            <CatalogCall Name="MIPI_SYST_CATPRINTF64_6" IdParamIdx="3" StringParamIdx="4"  />
116        </Catalog64>
117    </CatalogCalls>
118</CatalogGenerator>
119```
120
121Catalog Generator Template File {#mipi_syst_catgen_example_template}
122------------------------------------------------------------------------------
123The following XML file shows the template collateral file that is updated by
124the generator tool with the source code information. Note the empty XML
125elements called ``<syst:SourceFiles/>``, ``<syst:Catalog32/>``, and
126``<syst:Catalog64/>``. The generator replaces these empty elements with the
127collected information. The remaining contents are kept unchanged.
128```{.xml}
129<?xml version="1.0" encoding="utf-8"?>
130<syst:Collateral xmlns:syst="http://www.mipi.org/1.0/sys-t"
131    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
132    xsi:schemaLocation="http://www.mipi.org/1.0/sys-t
133                        https://www.mipi.org/schema/sys-t/sys-t_1-0.xsd">
134
135  <!--
136  A SyS-T collateral file starts with one syst:Collateral element
137  and contains at least one syst:Client child node.
138 -->
139  <syst:Client Name="example">
140
141    <!--
142    List of message GUIDs for this client:
143    A GUID is a 128bit numeric UUID following RFC 4122.
144    Clients that are identified by transport layer properties
145    can use pseudo GUID values to identify their messages. A
146    pseudo GUID is defined as a GUID with bit 71 set to zero.
147    The following encoding might be used for clients identified
148    by MIPI System Trace Protocol (STP) master/channel pairs:
149
150     {00000000-<MasterID>-<ChannelID>-<Origin>00-000000000000}
151
152    A Mask attribute can be added to define which bits of the GUID are
153    used for comparison. Without a mask, all bits need to match.
154    The example below makes the catalog valid for all channels from the
155    STP master value 0x644, or messages with the "494E..." GUID in their
156    payload.
157    -->
158    <syst:Guids>
159      <syst:Guid ID=  "{00000000-0644-0000-0000-000000000000}"
160                 Mask="{00000000-FFFF-0000-8000-000000000000}"><![CDATA[kernel]]></syst:Guid>
161      <syst:Guid ID="{494E5443-6F64-43D1-93F0-F3D49D92670C}"><![CDATA[user]]></syst:Guid>
162    </syst:Guids>
163
164    <!--
165    Define for which client build versions this collateral is valid.
166    Builds are expressed as values up to 64 bits in size, and can include a
167    mask that defines which bits need to match with a client build version
168    message.
169    -->
170    <syst:Builds>
171      <syst:Build ID="0x00010000" Mask="0xFFFF0000" ><![CDATA[SW Build 1.x]]></syst:Build>
172      <syst:Build ID="0x00020000" Mask="0xFFFF0000" ><![CDATA[SW Build 2.x]]></syst:Build>
173    </syst:Builds>
174
175     <!--
176     Settings to guide decoding tools what to do in cases
177     where several options are possible. The example below tells
178     a decoder to interprete SyS-T message timstamps only as an
179     informal field, but not the message creation time.
180     -->
181    <syst:Options>
182      <syst:Option Name="syst:timestamp">displayAsField</syst:Option>
183    </syst:Options>
184
185    <!--
186    Provide ID to name mapping for Origin header field values.
187    -->
188    <syst:Modules>
189      <syst:Module ID="0x1"><![CDATA[kernel]]></syst:Module>
190      <syst:Module ID="0x2"><![CDATA[user]]></syst:Module>
191  </syst:Modules>
192
193    <!--
194    Define the mapping of the numeric File-ID to source files.
195    This table is optional and only needed if source references are used
196    by the Catalog32 or Catalog64 elements.
197    -->
198    <syst:SourceFiles>
199      <syst:File ID="0x00000001"><![CDATA[example/src/main.c]]></syst:File>
200      <syst:File ID="0x00000002"><![CDATA[example/h/example.h]]></syst:File>
201    </syst:SourceFiles>
202
203    <!--
204    Define the mapping of catalog message ID to human-readable strings.
205    This table is optional and only needed if catalog messages are used.
206    A message can have optional "File" and "Line" attributes to specify the
207    source position of the instrumentation call. A decoder uses the
208    information from these attributes if the message payload doesn't provide
209    explicit location information.
210    -->
211    <syst:Catalog32>
212      <syst:Format ID="0x00000001" File="0x00000001" Line="23"><![CDATA[This is example %d.%d]]></syst:Format>
213      <syst:Format ID="0x00000002" File="0x00000002" Line="72"><![CDATA[task switch %x->%x]]></syst:Format>
214    </syst:Catalog32>
215
216    <syst:Catalog64>
217      <syst:Format ID="0x0000000000000001"><![CDATA[error %d]]></syst:Format>
218    </syst:Catalog64>
219
220    <!--
221    Define short message mapping from 28-Bit ID to user visible string.
222    -->
223    <syst:Short32>
224      <syst:Format ID="0x0000000" Mask="0xFFFFFF0"><![CDATA[Fatal Error code]]></syst:Format>
225      <syst:Format ID="0x0000013"><![CDATA[state ready]]></syst:Format>
226      <syst:Format ID="0x000001A"><![CDATA[state halt]]></syst:Format>
227    </syst:Short32>
228
229    <!--
230    Define short message mapping from 60-Bit ID to user visible string.
231    -->
232    <syst:Short64>
233      <syst:Format ID="0x123456700000000" Mask="0xFFFFFFF00000000"><![CDATA[warning state]]></syst:Format>
234
235    </syst:Short64>
236
237    <!--
238    Specify raw message protocol id assignment
239    -->
240    <syst:Write>
241      <syst:Protocol ID="0x01"><![CDATA[console]]></syst:Protocol>
242      <syst:Protocol ID="0x02"><![CDATA[console]]></syst:Protocol>
243      <syst:Protocol ID="0x03"><![CDATA[ftrace]]></syst:Protocol>
244      <syst:Protocol ID="0x20" Mask="0x30"><![CDATA[reserved]]></syst:Protocol>
245    </syst:Write>
246
247</syst:Client>
248
249</syst:Collateral>
250```