1# LwM2M Interoperability tests using Leshan demo server
2
3This directory contains list of testcases that use
4the Twister's Pytest integration to run testcases against Leshan demo server.
5These tests use emulated hardware (native_sim).
6
7These tests require setup that is not done in Twister run, so follow this documentation to set
8up the test environment.
9
10## Network setup
11
12As with typical network samples, host machine uses IP address `192.0.2.2` and the emulated device
13running Zephyr is using address `192.0.2.1`.
14
15Follow [Networking with the host system](https://docs.zephyrproject.org/latest/connectivity/networking/networking_with_host.html#networking-with-the-host-system)
16from Zephyr's documentation how to set it up, or follow [Create NAT and routing for Zephyr native network on Linux](https://github.com/zephyrproject-rtos/net-tools/blob/master/README%20NAT.md).
17
18
19### Run Lehan server from net-tools Docker container
20
21Zephyr's net-tools Docker container already contains Leshan, so if you don't want to set the environment up manually,
22use the pre-made docker.
23
24First, build the docker container. You only need to do this one, or when you update the container.
25```
26cd tools/net-tools/docker
27docker build -t net-tools .
28```
29
30Start the docker networking
31```
32cd tools/net-tools/
33sudo ./net-setup.sh --config docker.conf start
34```
35
36Start the docker container and run leshan
37```
38docker run --hostname=net-tools --name=net-tools --ip=192.0.2.2 --ip6=2001:db8::2 -p 8080:8080 -p 8081:8081 -p 5683:5683/udp --rm -dit --network=net-tools0 net-tools
39docker container exec net-tools /net-tools/start-leshan.sh
40```
41
42### Stop Leshan, docker and networking
43
44```
45cd tools/net-tools/
46docker kill net-tools
47sudo ./net-setup.sh --config docker.conf stop
48docker network rm net-tools0
49```
50### Leshan server setup (manual)
51
52* Leshan server must be reachable from the device using IP address `192.0.2.2`.
53  Configure the port forwarding, if you use Docker to run Leshan.
54* Leshan demo server REST API must be reachable from localhost.
55* tcp/8080 Leshan web interface and REST API
56* tcp/8081 Leshan bootstrap server REST API
57* udp/5683 Leshan non-secure CoAP
58* udp/5684 Leshan DTLS CoAP
59* udp/5783 non-secure Bootstrap CoAP
60* udp/5684 DTLS Bootstrap CoAP
61* Download Leshan JAR file from https://ci.eclipse.org/leshan/job/leshan/lastSuccessfulBuild/artifact/leshan-server-demo.jar
62* Download Leshan Bootstrap server JAR file from https://ci.eclipse.org/leshan/job/leshan/lastSuccessfulBuild/artifact/leshan-bsserver-demo.jar
63
64Both server can be started like this:
65```
66java -jar ./leshan-server-demo.jar -wp 8080 -vv
67java -jar ./leshan-bsserver-demo.jar -lp=5783 -slp=5784 -wp 8081
68```
69
70Or create a helper script that does everything, including download:
71```
72#!/bin/sh -eu
73
74# Download Leshan if needed
75if [ ! -f leshan-server-demo.jar ]; then
76        wget https://ci.eclipse.org/leshan/job/leshan/lastSuccessfulBuild/artifact/leshan-server-demo.jar
77fi
78
79if [ ! -f leshan-bsserver-demo.jar ]; then
80	wget 'https://ci.eclipse.org/leshan/job/leshan/lastSuccessfulBuild/artifact/leshan-bsserver-demo.jar'
81fi
82
83mkdir -p log
84
85start-stop-daemon --make-pidfile --pidfile log/leshan.pid --chdir $(pwd) --background --start \
86        --startas /bin/bash -- -c "exec java -jar ./leshan-server-demo.jar -wp 8080 -vv --models-folder objects >log/leshan.log 2>&1"
87
88start-stop-daemon --make-pidfile --pidfile log/leshan_bs.pid --chdir $(pwd) --background --start \
89        --startas /bin/bash -- -c "exec java -jar ./leshan-bsserver-demo.jar -lp=5783 -slp=5784 -wp 8081 -vv >log/leshan_bs.log 2>&1"
90```
91
92Then stopping would require similar script:
93```
94#!/bin/sh -eu
95
96start-stop-daemon --remove-pidfile --pidfile log/leshan.pid --stop
97start-stop-daemon --remove-pidfile --pidfile log/leshan_bs.pid --stop
98```
99
100## Python package requirements
101
102These tests require extra package that is not installed when you follow the Zephyr's setup.
103Install with `pip install CoAPthon3`
104
105## Running tests
106
107```
108twister -p native_sim -vv --enable-slow -T tests/net/lib/lwm2m/interop
109```
110
111Or use the Docker based testing
112
113```
114./scripts/net/run-sample-tests.sh tests/net/lib/lwm2m/interop
115```
116
117## Test specification
118
119Tests are written from test spec;
120[OMA Enabler Test Specification (Interoperability) for Lightweight M2M](https://www.openmobilealliance.org/release/LightweightM2M/ETS/OMA-ETS-LightweightM2M-V1_1-20190912-D.pdf)
121
122## Current status
123
124|Test case|Status|Notes|
125|---------|------|-----|
126|LightweightM2M-1.1-int-0 - Client Initiated Bootstrap                  |:white_check_mark:| |
127|LightweightM2M-1.1-int-1 - Client Initiated Bootstrap Full (PSK)       |:white_check_mark:| |
128|LightweightM2M-1.1-int-2 - Client Initiated Bootstrap Full (Cert)      | |testcase not implemented |
129|LightweightM2M-1.1-int-3 – Simple Bootstrap from Smartcard             |:large_orange_diamond:|We don't have any smartcard support.|
130|LightweightM2M-1.1-int-4 - Bootstrap Delete                            |:white_check_mark:| |
131|LightweightM2M-1.1-int-5 - Server Initiated Bootstrap                  |:white_check_mark:| |
132|LightweightM2M-1.1-int-6 - Bootstrap Sequence                          |:white_check_mark:| |
133|LightweightM2M-1.1-int-7 - Fallback to bootstrap                       |:white_check_mark:| |
134|LightweightM2M-1.1-int-8 - Bootstrap Read | |Test cannot be implemented from client side.|
135|LightweightM2M-1.1-int-9 - Bootstrap and Configuration Consistency     | |testcase not implemented |
136|LightweightM2M-1.1-int-101 - Initial Registration                      |:white_check_mark:| |
137|LightweightM2M-1.1-int-102 - Registration Update                       |:white_check_mark:| |
138|LightweightM2M-1.1-int-103 - Deregistration                            |:white_check_mark:| |
139|LightweightM2M-1.1-int-104 - Registration Update Trigge                |:white_check_mark:| |
140|LightweightM2M-1.1-int-105 - Discarded Register Update                 |:white_check_mark:| |
141|LightweightM2M-1.1-int-107 - Extending the lifetime of a registration  |:white_check_mark:| |
142|LightweightM2M-1.1-int-108 - Turn on Queue Mode                        |:white_check_mark:| |
143|LightweightM2M-1.1-int-109 - Behavior in Queue Mode                    |:white_check_mark:| |
144|LightweightM2M-1.1-int-201 - Querying basic information in Plain Text  |:white_check_mark:| |
145|LightweightM2M-1.1-int-203 - Querying basic information in TLV format  |:white_check_mark:| |
146|LightweightM2M-1.1-int-204 - Querying basic information in JSON format |:white_check_mark:| |
147|LightweightM2M-1.1-int-205 - Setting basic information in Plain Text   |:white_check_mark:| |
148|LightweightM2M-1.1-int-211 - Querying basic information in CBOR format |:white_check_mark:| |
149|LightweightM2M-1.1-int-212 - Setting basic information in CBOR format  |:white_check_mark:| |
150|LightweightM2M-1.1-int-215 - Setting basic information in TLV format   |:white_check_mark:| |
151|LightweightM2M-1.1-int-220 - Setting basic information in JSON format  |:white_check_mark:| |
152|LightweightM2M-1.1-int-221 - Attempt to perform operations on Security |:white_check_mark:| |
153|LightweightM2M-1.1-int-222 - Read on Object                            |:white_check_mark:| |
154|LightweightM2M-1.1-int-223 - Read on Object Instance                   |:white_check_mark:| |
155|LightweightM2M-1.1-int-224 - Read on Resource                          |:white_check_mark:| |
156|LightweightM2M-1.1-int-225 - Read on Resource Instance                 |:white_check_mark:| |
157|LightweightM2M-1.1-int-226 - Write (Partial Update) on Object Instance |:white_check_mark:| |
158|LightweightM2M-1.1-int-222 - Read on Object                            |:white_check_mark:| |
159|LightweightM2M-1.1-int-223 - Read on Object Instance                   |:white_check_mark:| |
160|LightweightM2M-1.1-int-224 - Read on Resource                          |:white_check_mark:| |
161|LightweightM2M-1.1-int-225 - Read on Resource Instance                 |:white_check_mark:| |
162|LightweightM2M-1.1-int-226 - Write (Partial Update) on Object Instance |:white_check_mark:| |
163|LightweightM2M-1.1-int-227 - Write (replace) on Resource               |:white_check_mark:| |
164|LightweightM2M-1.1-int-228 - Write on Resource Instance                |:white_check_mark:|[~~#64011~~](https://github.com/zephyrproject-rtos/zephyr/issues/64011) |
165|LightweightM2M-1.1-int-229 - Read-Composite Operation                  |:white_check_mark:|[~~#64012~~](https://github.com/zephyrproject-rtos/zephyr/issues/64012) [~~#64189~~](https://github.com/zephyrproject-rtos/zephyr/issues/64189) |
166|LightweightM2M-1.1-int-230 - Write-Composite Operation                 |:white_check_mark:| |
167|LightweightM2M-1.1-int-231 - Querying basic information in SenML JSON format|:white_check_mark:| |
168|LightweightM2M-1.1-int-232 - Querying basic information in SenML CBOR format|:white_check_mark:| |
169|LightweightM2M-1.1-int-233 - Setting basic information in SenML CBOR format|:white_check_mark:| |
170|LightweightM2M-1.1-int-234 - Setting basic information in SenML JSON format|:white_check_mark:| |
171|LightweightM2M-1.1-int-235 - Read-Composite Operation on root path     |:white_check_mark:| |
172|LightweightM2M-1.1-int-236 - Read-Composite - Partial Presence|:white_check_mark:| |
173|LightweightM2M-1.1-int-237 - Read on Object without specifying Content-Type|:white_check_mark:| |
174|LightweightM2M-1.1-int-241 - Executable Resource: Rebooting the device |:white_check_mark:| |
175|LightweightM2M-1.1-int-256 - Write Operation Failure                   |:white_check_mark:| |
176|LightweightM2M-1.1-int-257 - Write-Composite Operation                 |:white_check_mark:| |
177|LightweightM2M-1.1-int-260 - Discover Command                          |:white_check_mark:| |
178|LightweightM2M-1.1-int-261 - Write-Attribute Operation on a multiple resource|:white_check_mark:| |
179|LightweightM2M-1.1-int-280 - Successful Read-Composite Operation       |:white_check_mark:| |
180|LightweightM2M-1.1-int-281 - Partially Successful Read-Composite Operation|:white_check_mark:| |
181|LightweightM2M-1.1-int-301 - Observation and Notification of parameter values|:white_check_mark:| |
182|LightweightM2M-1.1-int-302 - Cancel Observations using Reset Operation |:white_check_mark:| |
183|LightweightM2M-1.1-int-303 - Cancel observations using Observe with Cancel parameter|:white_check_mark:||
184|LightweightM2M-1.1-int-304 - Observe-Composite Operation               |:white_check_mark:| |
185|LightweightM2M-1.1-int-305 - Cancel Observation-Composite Operation    |:white_check_mark:| |
186|LightweightM2M-1.1-int-306 – Send Operation                            |:white_check_mark:|[~~#64290~~](https://github.com/zephyrproject-rtos/zephyr/issues/64290)|
187|LightweightM2M-1.1-int-307 – Muting Send                               |:white_check_mark:| |
188|LightweightM2M-1.1-int-308 - Observe-Composite and Creating Object Instance|:white_check_mark:|[~~#64634~~](https://github.com/zephyrproject-rtos/zephyr/issues/64634)|
189|LightweightM2M-1.1-int-309 - Observe-Composite and Deleting Object Instance|:white_check_mark:|[~~#64634~~](https://github.com/zephyrproject-rtos/zephyr/issues/64634)|
190|LightweightM2M-1.1-int-310 - Observe-Composite and modification of parameter values|:white_check_mark:| |
191|LightweightM2M-1.1-int-311 - Send command                              |:white_check_mark:| |
192|LightweightM2M-1.1-int-401 - UDP Channel Security - PSK Mode           |:white_check_mark:| |
193|LightweightM2M-1.1-int-1630 - Create Portfolio Object Instance         |:white_check_mark:| |
194|LightweightM2M-1.1-int-1635 - Delete Portfolio Object Instance         |:white_check_mark:| |
195
196* :white_check_mark: Working OK.
197* :large_orange_diamond: Feature or operation not implemented.
198* :red_circle: Broken
199