1Cipher suite (CCMP, TKIP, GCMP, ..) and key management testing 2============================================================== 3 4wpa_supplicant and hostapd include number of extensions that allow 5special test builds to be used for testing functionality related to 6correct implementation of IEEE 802.11. These extensions allow behavior 7to be modified and invalid operations to be performed to verify behavior 8of other devices in unexpected situations. While most of the testing 9extensions are focused on the fully automated testing framework with 10mac80211_hwsim (see tests/hwsim subdirectory), many of these can be used 11for over-the-air testing of the protocol as well. 12 13Since some of the testing extensions can result in exposing key 14information or allowing non-compliant behavior, these changes are 15disabled in default wpa_supplicant and hostapd builds for production 16purposes. Testing functionality can be enabled by adding 17CONFIG_TESTING_OPTIONS=y into build configuration (hostapd/.config and 18wpa_supplicant/.config). 19 20 21Testing setup 22------------- 23 24These tests can be run as black-box testing without having to modify the 25tested device at all or without knowing details of its 26functionality. The test commands in wpa_supplicant/hostapd control 27interfaces are used to perform unexpected operations and normal data 28traffic is used to verify reaction of the tested device to such 29operations. 30 31In theory, the test functionality is available with most drivers 32supported by wpa_supplicant/hostapd, but the most reliable results are 33likely available through ath9k-based devices. If you are using something 34else, it is strongly recommended that you'll run the first tests with 35sniffer captures and verify that the test tools are behaving correctly. 36 37wpa_supplicant is used to control a test device in station mode to test 38an AP and hostapd is similarly used to control a test device in AP mode 39to test a station. 40 41Various data traffic generators could be used to test the behavior, but 42this document focuses on using ping to test unicast traffic and arping 43to test broadcast traffic. To keep things simple and to reduce 44interference from unrelated traffic, the steps here assume static IPv4 45addresses are used and IPv6 is disabled. 46 47The tests here use WPA2-Personal for simplicity. WPA2-Enterprise and 48other cipher suites can also be tested for more complete coverage. 49 50Example hostapd.conf for the test tool in AP mode: 51 52driver=nl80211 53hw_mode=g 54channel=1 55ieee80211n=1 56interface=wlan0 57ctrl_interface=/var/run/hostapd 58ctrl_interface_group=adm 59ssid=test-psk 60wpa=2 61wpa_key_mgmt=WPA-PSK 62wpa_pairwise=CCMP 63wpa_passphrase=12345678 64 65Example wpa_supplicant.conf for the test tool in station mode: 66 67ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=adm 68 69network={ 70 ssid="test-psk" 71 key_mgmt=WPA-PSK 72 psk="12345678" 73} 74 75The examples in this document assume following IPv4 address 76configuration: 77 78Test tool (either AP or station mode): 192.168.1.1/24 79Device under test: 192.168.1.2/24 80 81 82Data traffic tests 83------------------ 84 85ping is used to test whether unicast frames go through on the data 86link. It should be noted that ping may need to use broadcast ARP at the 87beginning if the other device is not yet in the ARP table, so working 88broadcast and unicast connectivity may be needed to get this started. 89 90Example: 91 92$ ping -n -c 5 192.168.1.2 93PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. 9464 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=43.7 ms 9564 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=67.9 ms 9664 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=900 ms 9764 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=5.81 ms 9864 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=135 ms 99 100--- 192.168.1.2 ping statistics --- 1015 packets transmitted, 5 received, 0% packet loss, time 4004ms 102rtt min/avg/max/mdev = 5.811/230.605/900.223/337.451 ms 103 104This shows working unicast data connectivity. 105 106$ ping -n -c 5 192.168.1.2 107PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. 108 109--- 192.168.1.2 ping statistics --- 1105 packets transmitted, 0 received, 100% packet loss, time 4033ms 111 112This shows not working unicast data connectivity. 113 114 115arping is used to test broadcast connectivity. 116 117Example: 118 119$ arping -b -I wlan0 192.168.1.2 -c 5 120ARPING 192.168.1.2 from 192.168.1.1 wlan0 121Unicast reply from 192.168.1.2 [<DUT MAC address>] 119.695ms 122Unicast reply from 192.168.1.2 [<DUT MAC address>] 144.496ms 123Unicast reply from 192.168.1.2 [<DUT MAC address>] 166.788ms 124Unicast reply from 192.168.1.2 [<DUT MAC address>] 2.283ms 125Unicast reply from 192.168.1.2 [<DUT MAC address>] 2.234ms 126Sent 5 probes (5 broadcast(s)) 127Received 5 response(s) 128 129This shows working broadcast data connectivity. 130 131$ arping -b -I wlan0 192.168.1.2 -c 5 132ARPING 192.168.1.2 from 192.168.1.1 wlan0 133Sent 5 probes (5 broadcast(s)) 134Received 0 response(s) 135 136This shows not working broadcast data connectivity. 137 138If testing results do not look consistent, the testing state can be 139cleared by disconnection and reconnecting the station (the test tool or 140the DUT) to the network. 141 142 143Sniffer and wlantest 144-------------------- 145 146It is useful to get a wireless sniffer capture from the operating 147channel of the AP to be able to confirm DUT behavior if any of the data 148tests indicate reason to believe something is not working as expected. 149 150wlantest (from the wlantest directory of hostap.git) can be used to 151decrypt and analyze a sniffer capture. For example: 152 153wlantest -r wlan0.pcap -n decrypted.pcap -p 12345678 154 155The debug prints and comments in the generated file indicate where 156unexpected behavior has been detected, e.g., when the test tool ends up 157clearing its packet number to test replay protection. That can help in 158checking whether the DUT actually replies to a frame that it was 159supposed to drop due replay. 160 161 162Testing replay protection on a station device 163--------------------------------------------- 164 165Start hostapd and use hostapd_cli on the test device to control testing 166operations. Connect the DUT to the network. 167 168<3>AP-STA-CONNECTED <DUT MAC address> 169 170This indicates that the connection was completed successfully. 171 172Verify that broadcast and unicast traffic works correctly (if not, 173something is wrong in the test setup and that needs to be resolved 174before being able to run any tests). 175 176Verify that unicast traffic works and issue the following command in 177hostapd_cli: 178 179> raw RESET_PN <DUT MAC address> 180OK 181 182Verify that unicast traffic does not work anymore. If it does, the DUT 183does not implement replay protection correctly for unicast frames. Note 184that unicast traffic can recover once the packet number from the test 185device increases beyond the value used prior to that RESET_PN command. 186 187 188Verify that broadcast traffic works and issue the following command in 189hostapd_cli: 190 191> raw RESET_PN ff:ff:ff:ff:ff:ff 192OK 193 194Verify that broadcast traffic does not work anymore. If it does, the DUT 195does not implement replay protection correctly for broadcast 196frames. Note that broadcast traffic can recover once the packet number 197from the test device increases beyond the value used prior to that 198RESET_PN command. 199 200 201Testing replay protection on an AP device 202----------------------------------------- 203 204Start the AP (DUT) and start wpa_supplicant on the test device to 205connect to the network. Use wpa_cli to control the test device. 206 207<3>SME: Trying to authenticate with <DUT MAC address> (SSID='test-psk' freq=5240 MHz) 208<3>CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD 209<3>Trying to associate with <DUT MAC address> (SSID='test-psk' freq=5240 MHz) 210<3>Associated with <DUT MAC address> 211<3>WPA: Key negotiation completed with <DUT MAC address> [PTK=CCMP GTK=CCMP] 212<3>CTRL-EVENT-CONNECTED - Connection to <DUT MAC address> completed [id=0 id_str=] 213 214Verify that unicast traffic works and issue the following command in 215wpa_cli: 216 217> raw RESET_PN 218OK 219 220Verify that unicast traffic does not work anymore. If it does, the DUT 221does not implement replay protection correctly. Note that unicast 222traffic can recover once the packet number from the test device 223increases beyond the value used prior to that RESET_PN command. 224 225IEEE 802.11 protocol uses unicast frames in station-to-AP direction, so 226there is no need to test AP replay protection behavior separately with 227the broadcast IPv4 traffic (which would be converted to unicast frames 228on the link layer). 229 230 231Testing GTK reinstallation protection on a station device (group handshake) 232--------------------------------------------------------------------------- 233 234Use the procedure describe above for testing replay protection, but with 235the following hostapd_cli commands: 236 237Test broadcast connectivity; should work 238 239> raw RESEND_GROUP_M1 <DUT MAC address> 240OK 241> raw RESET_PN ff:ff:ff:ff:ff:ff 242OK 243 244Test broadcast connectivity; should not work; if it does, the device 245does not implement protection for delayed retransmission of Group Key 246Message 1/2. 247 248 249Testing GTK reinstallation protection on a station device (4-way handshake) 250--------------------------------------------------------------------------- 251 252Use the procedure described above for testing replay protection for 253broadcast traffic, but with the following hostapd_cli commands: 254 255Test broadcast connectivity; should work 256 257> raw RESEND_M3 <DUT MAC address> 258OK 259> raw RESET_PN ff:ff:ff:ff:ff:ff 260OK 261 262Test broadcast connectivity; should not work; if it does, the device 263does not implement protection for delayed retransmission of 4-way 264handshake EAPOL-Key Message 3/4. 265 266Variant 1: Include extra Message 1/4 267 268Otherwise same as above, but replace RESEND_M3 command with: 269 270> raw RESEND_M1 <DUT MAC address> 271OK 272> raw RESEND_M3 <DUT MAC address> 273OK 274 275Variant 2: Include two extra Message 1/4 276 277Otherwise same as above, but replace RESEND_M3 command with: 278 279> raw RESEND_M1 <DUT MAC address> change-anonce 280OK 281> raw RESEND_M1 <DUT MAC address> 282OK 283> raw RESEND_M3 <DUT MAC address> 284OK 285 286 287Testing TK reinstallation protection on a station device (4-way handshake) 288-------------------------------------------------------------------------- 289 290Use the procedure described above for testing replay protection for 291unicast traffic, but with the following hostapd_cli commands: 292 293Test unicast connectivity; should work 294 295> raw RESEND_M3 <DUT MAC address> 296OK 297> raw RESET_PN <DUT MAC address> 298OK 299 300Test unicast connectivity; should not work; if it does, the device 301does not implement protection for delayed retransmission of 4-way 302handshake EAPOL-Key Message 3/4. 303 304Variant 1: Include extra Message 1/4 305 306Otherwise same as above, but replace RESEND_M3 command with: 307 308> raw RESEND_M1 <DUT MAC address> 309OK 310> raw RESEND_M3 <DUT MAC address> 311OK 312 313Variant 2: Include two extra Message 1/4 314 315Otherwise same as above, but replace RESEND_M3 command with: 316 317> raw RESEND_M1 <DUT MAC address> change-anonce 318OK 319> raw RESEND_M1 <DUT MAC address> 320OK 321> raw RESEND_M3 <DUT MAC address> 322OK 323 324 325Testing ANonce generation on an AP device 326----------------------------------------- 327 328Start the AP (DUT) and start wpa_supplicant on the test device to 329connect to the network. Use wpa_cli to control the test device. 330 331<3>SME: Trying to authenticate with <DUT MAC address> (SSID='test-psk' freq=5240 MHz) 332<3>CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD 333<3>Trying to associate with <DUT MAC address> (SSID='test-psk' freq=5240 MHz) 334<3>Associated with <DUT MAC address> 335<3>WPA: Key negotiation completed with <DUT MAC address> [PTK=CCMP GTK=CCMP] 336<3>CTRL-EVENT-CONNECTED - Connection to <DUT MAC address> completed [id=0 id_str=] 337 338Show the ANonce from the first 4-way handshake, request PTK rekeying, 339and show the ANonce from the second 4-way handshake: 340 341> GET anonce 342df8c61d1f1f7aca9f1739dd888199547f4af2b8b07f8bf15b45ea271da0072b2 343> raw KEY_REQUEST 0 1 344OK 345> GET anonce 346d8ddcb716f28abfdf1352a05d51e7a70f58802122e99d13c730c3c0f09594aac 347 348If the ANonce values are same, the AP did not update the ANonce for 349rekeying (it should have as shown in the example above). 350 351 352Testing FT Reassociation Request frame retransmission on an AP device 353--------------------------------------------------------------------- 354 355This test case requires a sniffer to be used and manually analyzed. 356 357Enable FT on the DUT AP (likely two AP devices needed), connect test 358tool to the AP using FT protocol (e.g., connect to another AP first and 359then use the "ROAM <BSSID>" command), and do the following steps: 360 361- verify unicast traffic from the AP to test station (either ping from 362 the AP or from a device behind the AP); this needs to work 363- wpa_cli "raw RESEND_ASSOC" 364- verify unicast traffic from the AP to test station (either ping from 365 the AP or from a device behind the AP); this is likely to fail, but 366 the real analysis is done based on the sniffer capture 367 368In the sniffer capture, find the last Reassociation Request frame from 369the test station (this is more or less identical to the previous one and 370the only one that should not have Authentication frame exchange before 371it). Look at the last used PN in a unicast Data frame from the AP to the 372test station before the last Reassociation Request frame and the PN in 373the following unicast Data frame after the last Reassociation Request 374frame. If the PN goes down (e.g., is reset to 1), this would be a sign 375of a likely security vulnerability. The AP's TK configuration should be 376verified (i.e., whether it is configuring the same TK again and then 377allowing it to be used with reused PN values). 378
