Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ce-oem] Add wifi AP mode test (New) #1606

Merged
merged 34 commits into from
Dec 26, 2024
Merged

[ce-oem] Add wifi AP mode test (New) #1606

merged 34 commits into from
Dec 26, 2024

Conversation

rickwu666666
Copy link
Contributor

@rickwu666666 rickwu666666 commented Nov 19, 2024 •
edited
Loading

The wifi ap mode test are intend to test if DUT is able to create wifi AP and allow a HOST to connect with. And also the HOST will perform ping test to make sure the connection works fine.

Description

This test require a HOST machine to connect to DUT's AP. And support only if wireless interface managed by Network-Manager for both DUT and HOST, since networkd does not support handle wireless interface working in AP mode[1].
Also, even the Network-Manager is able to handle wireless interface working in AP mode but it support only on band a and bg[2].

[ref 1] https://netplan.readthedocs.io/en/stable/netplan-yaml/#properties-for-device-type-wifis
[ref 2] https://networkmanager.dev/docs/api/latest/settings-802-11-wireless.html

Resolved issues

Documentation

Tests

Passed result:

ceqa@ubuntu:~$ sudo checkbox-baytown.checkbox-cli 
Using sideloaded provider: checkbox-provider-ce-oem, version 0.1 from /var/tmp/checkbox-providers/checkbox-provider-ce-oem
Preparing...
WARNING:plainbox.secure.providers.v1:Skipped file: /var/tmp/checkbox-providers/checkbox-provider-ce-oem/bin/__pycache__/check_gpio.cpython-310.pyc
WARNING:plainbox.secure.providers.v1:Skipped file: /var/tmp/checkbox-providers/checkbox-provider-ce-oem/bin/gst_utils.py
WARNING:plainbox.secure.providers.v1:Skipped file: /snap/checkbox-baytown/x1/providers/plainbox-provider-checkbox/units/stress/suspend_cycles_reboot.md
============[ Bootstrap com.canonical.certification::device (1/4) ]=============
======[ Bootstrap com.canonical.certification::wifi_interface_mode (2/4) ]======
=======[ Bootstrap com.canonical.certification::net_if_management (3/4) ]=======
=========[ Bootstrap com.canonical.contrib::ce_oem_wifi_ap_mode (4/4) ]=========
Reports will be saved to: /root/snap/checkbox-baytown/x1/.local/share/checkbox-ng
==============[ Running job 1 / 2. Estimated time left: 0:04:00 ]===============
------[ Create open 802.11bg Wi-Fi AP on wlP1p1s0f0 and check connection ]------
ID: com.canonical.contrib::ce-oem-wireless/ap_open_bg_ch5_none_wlP1p1s0f0_automated
Category: com.canonical.certification::wifi_ap
... 8< -------------------------------------------------------------------------
2024-11-19 03:17:47,116 - INFO - Initializing connection
2024-11-19 03:17:53,865 - INFO - Initialized connection successful!
2024-11-19 03:17:54,275 - INFO - IP address is 10.42.0.1
2024-11-19 03:17:54,277 - INFO - Ping target Host first ...
2024-11-19 03:17:57,311 - INFO - Ping target Host 10.102.89.253 successful...
2024-11-19 03:17:57,316 - INFO - Attempting to connect DUT AP qa-test-ssid...
2024-11-19 03:17:57,317 - INFO - Attempting to connect DUT AP qa-test-ssid 1 time...
2024-11-19 03:17:57,858 - ERROR - Command failed: Error: No network with SSID 'qa-test-ssid' found.

2024-11-19 03:17:57,859 - WARNING - Not able to found SSID qa-test-ssid
2024-11-19 03:18:07,869 - INFO - Attempting to connect DUT AP qa-test-ssid 2 time...
2024-11-19 03:18:10,308 - INFO - Connect successful!
2024-11-19 03:18:10,309 - INFO - Attempting to ping DUT...
2024-11-19 03:18:14,174 - INFO - Packet loss: 0 %
2024-11-19 03:18:14,175 - INFO - Ping DUT pass
2024-11-19 03:18:14,684 - INFO - Deleted host connection successfully.
2024-11-19 03:18:14,686 - INFO - Exiting context and cleaning up connection

------------------------------------------------------------------------- >8 ---
Outcome: job passed

==============[ Running job 1 / 1. Estimated time left: 0:02:00 ]===============
-------[ Create wpa 802.11a Wi-Fi AP on wlP1p1s0f0 and check connection ]-------
ID: com.canonical.contrib::ce-oem-wireless/ap_wpa_wpa-psk_a_ch44_ccmp_wlP1p1s0f0_automated
Category: com.canonical.certification::wifi_ap
... 8< -------------------------------------------------------------------------
2024-11-19 03:19:48,719 - INFO - Initializing connection
2024-11-19 03:19:55,530 - INFO - Initialized connection successful!
2024-11-19 03:19:55,955 - INFO - IP address is 10.42.0.1
2024-11-19 03:19:55,955 - INFO - Ping target Host first ...
2024-11-19 03:19:59,006 - INFO - Ping target Host 10.102.89.253 successful...
2024-11-19 03:19:59,009 - INFO - Attempting to connect DUT AP qa-test-ssid...
2024-11-19 03:19:59,011 - INFO - Attempting to connect DUT AP qa-test-ssid 1 time...
2024-11-19 03:20:05,883 - INFO - Connect successful!
2024-11-19 03:20:05,883 - INFO - Attempting to ping DUT...
2024-11-19 03:20:09,759 - INFO - Packet loss: 0 %
2024-11-19 03:20:09,760 - INFO - Ping DUT pass
2024-11-19 03:20:10,255 - INFO - Deleted host connection successfully.
2024-11-19 03:20:10,255 - INFO - Exiting context and cleaning up connection

------------------------------------------------------------------------- >8 ---
Outcome: job passed

Failed result:

==============[ Running job 2 / 2. Estimated time left: 0:02:00 ]===============
-------[ Create wpa 802.11a Wi-Fi AP on wlP1p1s0f0 and check connection ]-------
ID: com.canonical.contrib::ce-oem-wireless/ap_wpa_wpa-psk_a_ch44_ccmp_wlP1p1s0f0_automated
Category: com.canonical.certification::wifi_ap
... 8< -------------------------------------------------------------------------
2024-11-19 03:18:16,983 - INFO - Initializing connection
2024-11-19 03:18:22,581 - INFO - Initialized connection successful!
2024-11-19 03:18:23,018 - INFO - IP address is 10.42.0.1
2024-11-19 03:18:23,019 - INFO - Ping target Host first ...
2024-11-19 03:18:26,079 - INFO - Ping target Host 10.102.89.253 successful...
2024-11-19 03:18:26,081 - INFO - Attempting to connect DUT AP qa-test-ssid...
2024-11-19 03:18:26,082 - INFO - Attempting to connect DUT AP qa-test-ssid 1 time...
2024-11-19 03:18:51,948 - INFO - Connect successful!
2024-11-19 03:18:51,948 - INFO - Attempting to ping DUT...
2024-11-19 03:19:05,899 - ERROR - Command failed: PING 10.42.0.1 (10.42.0.1) 56(84) bytes of data.

--- 10.42.0.1 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3091ms


2024-11-19 03:19:05,899 - ERROR - An error occurred during ping_test: Command 'sshpass -p Rick@canonical ssh -o StrictHostKeyChecking=no [email protected] ping 10.42.0.1 -c 4' returned non-zero exit status 1.
2024-11-19 03:19:06,428 - INFO - Deleted host connection successfully.
2024-11-19 03:19:06,429 - INFO - Exiting context and cleaning up connection
------------------------------------------------------------------------- >8 ---
Outcome: job failed

@rickwu666666 rickwu666666 added the ce-oem The PR only contains to the ce-oem provider under the contrib area label Nov 19, 2024
@rickwu666666 rickwu666666 changed the title Add wifi AP mode test (New) [ce-oem] Add wifi AP mode test (New) Nov 19, 2024
stanley31huang
stanley31huang requested changes Nov 29, 2024
View reviewed changes
Copy link
Collaborator

@stanley31huang stanley31huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left several inline comments, please take a look.

@rickwu666666 rickwu666666 force-pushed the add-wifi-ap-mode-test branch from cb0d340 to a41429a Compare December 4, 2024 07:56
rickwu666666 and others added 11 commits December 11, 2024 08:41
@rickwu666666
Fix argparse issue
812a99e
@rickwu666666
Modify test and add a defualt set to test
bd7c1de 
This default set will be get the first available wireless interface
@rickwu666666 rickwu666666 marked this pull request as ready for review December 12, 2024 06:42
@rickwu666666 rickwu666666 marked this pull request as draft December 19, 2024 01:59
rickwu666666 and others added 3 commits December 23, 2024 08:17
@rickwu666666 @stanley31huang
Update contrib/checkbox-ce-oem/checkbox-provider-ce-oem/bin/wifi_test.py
c7944a7 
Co-authored-by: stanley31huang <[email protected]>
@rickwu666666
Change the logic of connect DUT from HOST and ping test
0175bd8 
Implemented as a context manager to better handle the setup and teardown of tests.
@rickwu666666
Fix black format
cce3047
@rickwu666666 rickwu666666 marked this pull request as ready for review December 23, 2024 06:36
@rickwu666666 rickwu666666 marked this pull request as draft December 24, 2024 03:05
@rickwu666666
change error message to handle by raise SystemError
66cd32e
@rickwu666666
Refactor the logic of creating and bringing up the connection from th…
fee3486 
…e HOST.

We created two functions: one for creating the connection and another for bringing up the connection to make the code more straightforward.
@rickwu666666
Add arg --host-interface to the job, since we add this args in the sc…
7f2c307 
…ript for make the test environment more consistent
@rickwu666666
Fix black format
656bcf4
@rickwu666666
Copy link
Contributor Author

rickwu666666 commented Dec 24, 2024 •
edited
Loading

Test result of the refactor logic, for open ap

rickwu@rickwu:~/dev/checkbox-providers/checkbox/contrib/checkbox-ce-oem/checkbox-provider-ce-oem/bin$ ./wifi_test.py --interface wlp0s20f3 --host-interface wlp1s0 --host-ip 10.102.89.99 --host-user ubuntu --host-pwd insecure wifi --mode ap  --band a --channel 36
2024-12-24 12:26:58,429 - INFO - Initializing connection
2024-12-24 12:26:58,429 - INFO - Run command: nmcli c add type wifi ifname wlp0s20f3 con-name qa-test-ap autoconnect no wifi.ssid TRpwtRyJxm wifi.mode ap ipv4.method shared
2024-12-24 12:26:58,524 - INFO - Run command: nmcli c modify qa-test-ap wifi.band a wifi.channel 36 
2024-12-24 12:26:58,644 - INFO - Run command: nmcli c up qa-test-ap
2024-12-24 12:27:01,063 - INFO - Initialized connection successful!
2024-12-24 12:27:01,063 - INFO - Run command: nmcli -g IP4.ADDRESS device show wlp0s20f3
2024-12-24 12:27:01,153 - INFO - IP address is 10.42.0.1
2024-12-24 12:27:01,154 - INFO - Pinging target host first...
2024-12-24 12:27:01,154 - INFO - Run command: ping 10.102.89.99 -c 4
2024-12-24 12:27:04,263 - INFO - Ping to target host 10.102.89.99 successful.
2024-12-24 12:27:04,264 - INFO - Attempting to create the connection on HOST (1/10)...
2024-12-24 12:27:04,264 - INFO - Run command: sshpass -p insecure ssh -o StrictHostKeyChecking=no [email protected] nmcli con add type wifi ifname wlp1s0 con-name qa-test-ap ssid TRpwtRyJxm
2024-12-24 12:27:07,761 - INFO - Create connection successful!
2024-12-24 12:27:07,761 - INFO - Attempting to bring up the connection on HOST (1/10)...
2024-12-24 12:27:07,762 - INFO - Run command: sshpass -p insecure ssh -o StrictHostKeyChecking=no [email protected] nmcli c up qa-test-ap
2024-12-24 12:27:14,289 - INFO - Bring up connection successful!
2024-12-24 12:27:14,289 - INFO - Run command: nmcli -g IP4.ADDRESS device show wlp0s20f3
2024-12-24 12:27:14,388 - INFO - Attempting to ping DUT...
2024-12-24 12:27:14,388 - INFO - Run command: sshpass -p insecure ssh -o StrictHostKeyChecking=no [email protected] ping 10.42.0.1 -c 4
2024-12-24 12:27:17,833 - INFO - Packet loss: 0 %
2024-12-24 12:27:17,833 - INFO - Ping DUT passed.
2024-12-24 12:27:17,833 - INFO - Run command: sshpass -p insecure ssh -o StrictHostKeyChecking=no [email protected] nmcli c delete qa-test-ap
2024-12-24 12:27:19,774 - INFO - Deleted host connection successfully.
2024-12-24 12:27:19,775 - INFO - Exiting context and cleaning up connection
2024-12-24 12:27:19,775 - INFO - Run command: nmcli c delete qa-test-ap

for wpa-psk

rickwu@rickwu:~/dev/checkbox-providers/checkbox/contrib/checkbox-ce-oem/checkbox-provider-ce-oem$ ./bin/wifi_test.py --interface wlp0s20f3 --host-interface wlp1s0 --host-ip 10.102.89.99 --host-user ubuntu --host-pwd insecure wifi --mode ap  --band a --channel 36 --keymgmt wpa-psk --group ccmp
2024-12-24 13:54:13,045 - INFO - Initializing connection
2024-12-24 13:54:13,045 - INFO - Run command: nmcli c add type wifi ifname wlp0s20f3 con-name qa-test-ap autoconnect no wifi.ssid XJluQltlxA wifi.mode ap ipv4.method shared
2024-12-24 13:54:13,151 - INFO - Run command: nmcli c modify qa-test-ap wifi.band a wifi.channel 36 
2024-12-24 13:54:13,275 - INFO - Run command: nmcli c modify qa-test-ap wifi-sec.key-mgmt wpa-psk wifi-sec.psk insecure wifi-sec.group ccmp
2024-12-24 13:54:13,375 - INFO - Run command: nmcli c up qa-test-ap
2024-12-24 13:54:15,880 - INFO - Initialized connection successful!
2024-12-24 13:54:15,880 - INFO - Run command: nmcli -g IP4.ADDRESS device show wlp0s20f3
2024-12-24 13:54:15,965 - INFO - IP address is 10.42.0.1
2024-12-24 13:54:15,965 - INFO - Pinging target host first...
2024-12-24 13:54:15,965 - INFO - Run command: ping 10.102.89.99 -c 4
2024-12-24 13:54:19,079 - INFO - Ping to target host 10.102.89.99 successful.
2024-12-24 13:54:19,079 - INFO - Attempting to create the connection on HOST (1/10)...
2024-12-24 13:54:19,080 - INFO - Run command: sshpass -p insecure ssh -o StrictHostKeyChecking=no [email protected] nmcli con add type wifi ifname wlp1s0 con-name qa-test-ap ssid XJluQltlxA wifi-sec.key-mgmt wpa-psk wifi-sec.psk insecure
2024-12-24 13:54:22,613 - INFO - Create connection successful!
2024-12-24 13:54:22,613 - INFO - Attempting to bring up the connection on HOST (1/10)...
2024-12-24 13:54:22,613 - INFO - Run command: sshpass -p insecure ssh -o StrictHostKeyChecking=no [email protected] nmcli c up qa-test-ap
2024-12-24 13:54:38,697 - INFO - Bring up connection successful!
2024-12-24 13:54:38,697 - INFO - Run command: nmcli -g IP4.ADDRESS device show wlp0s20f3
2024-12-24 13:54:38,785 - INFO - Attempting to ping DUT...
2024-12-24 13:54:38,785 - INFO - Run command: sshpass -p insecure ssh -o StrictHostKeyChecking=no [email protected] ping 10.42.0.1 -c 4
2024-12-24 13:54:42,244 - INFO - Packet loss: 0 %
2024-12-24 13:54:42,244 - INFO - Ping DUT passed.
2024-12-24 13:54:42,244 - INFO - Run command: sshpass -p insecure ssh -o StrictHostKeyChecking=no [email protected] nmcli c delete qa-test-ap
2024-12-24 13:54:44,096 - INFO - Deleted host connection successfully.
2024-12-24 13:54:44,096 - INFO - Exiting context and cleaning up connection
2024-12-24 13:54:44,096 - INFO - Run command: nmcli c delete qa-test-ap

@rickwu666666 rickwu666666 marked this pull request as ready for review December 24, 2024 06:10
@rickwu666666
Reduce the fail retry to 3 time in 30s
57fba1e 
This change is try to prevent to cover any potential issue on connection bring up delay
@rickwu666666
Fix black
0abbda1
stanley31huang
stanley31huang previously approved these changes Dec 25, 2024
View reviewed changes
Copy link
Collaborator

@stanley31huang stanley31huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

stanley31huang
stanley31huang approved these changes Dec 26, 2024
View reviewed changes
Copy link
Collaborator

@stanley31huang stanley31huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rickwu666666 rickwu666666 merged commit 47ffbbd into main Dec 26, 2024
10 checks passed
@rickwu666666 rickwu666666 deleted the add-wifi-ap-mode-test branch December 26, 2024 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stanley31huang stanley31huang stanley31huang approved these changes

@baconYao baconYao Awaiting requested review from baconYao

@weizhenwu weizhenwu Awaiting requested review from weizhenwu

@LiaoU3 LiaoU3 Awaiting requested review from LiaoU3

Assignees
No one assigned
Labels
ce-oem The PR only contains to the ce-oem provider under the contrib area
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@rickwu666666 @stanley31huang