Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.github/workflows/test.yml: Improve test workflow #200

Merged
merged 49 commits into from
Dec 17, 2024
Merged
Changes from 3 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
d46ef63
.github/workflows/test.yml: Move IPXE server to script in osfv
EduKav1813 Dec 3, 2024
20ec568
.github/workflows/test.yml: Matrixify test execution
EduKav1813 Dec 5, 2024
576e468
.github/workflows/test.yml: change indentation from 4 to 2 spaces
EduKav1813 Dec 5, 2024
b56e343
.github/workflows/test.yml: Update ipxe-run.sh script usage
EduKav1813 Dec 5, 2024
051caa1
.github/workflows/test.yml: Move test preparation steps to the run-te…
EduKav1813 Dec 5, 2024
0a63259
./github/workflows/test.yml: replace matrix.config with matrix.include
EduKav1813 Dec 6, 2024
2f8993c
./github/workflows/test.yml: add step to create logs directory
EduKav1813 Dec 6, 2024
20c52a1
Merge branch 'develop' into improve-meta-dts-test-workflow
EduKav1813 Dec 6, 2024
d900bd3
./github/workflows/test.yml: make ipxe server work as a process
EduKav1813 Dec 6, 2024
0fb47d0
./github/workflows/test.yml: fix creating qemu-data if it already exists
EduKav1813 Dec 6, 2024
8ae4197
./github/workflows/test.yml: fix exporting variable in create_log_dir…
EduKav1813 Dec 6, 2024
447c744
./github/workflows/test.yml: fix creating many virtual envs
EduKav1813 Dec 6, 2024
44075c2
./github/workflows/test.yml: fix checking out osfv many times
EduKav1813 Dec 6, 2024
4e9b46d
./github/workflows/test.yml: update prerequisite jobs for run-tests
EduKav1813 Dec 6, 2024
8796158
./github/workflows/test.yml: fix runner to dts-builder
EduKav1813 Dec 6, 2024
60cee9b
./github/workflows/test.yml: fix using the LOG_DIR env variable
EduKav1813 Dec 6, 2024
a9dc9a3
Merge branch 'develop' into improve-meta-dts-test-workflow
EduKav1813 Dec 10, 2024
0a6c784
workflow: pre-commit fix
m-iwanicki Dec 10, 2024
afc0ca7
workflows/test.yml: Fix matrix.include.property -> matrix.property
EduKav1813 Dec 11, 2024
b75d736
fixup! workflows/test.yml: Fix matrix.include.property -> matrix.prop…
EduKav1813 Dec 11, 2024
4cdccc1
.github/workflows/test.yml: fix passing arguments to run.sh
EduKav1813 Dec 11, 2024
5cc50b4
.github/workflows/test.yml: replace osfv/run.sh with robot
EduKav1813 Dec 11, 2024
aab1af2
.github/workflows/test.yml: disable fail-fast (now all tests run even…
EduKav1813 Dec 11, 2024
5481c27
.github/workflows/test.yml: fix typo dst-test-ci -> dts-test-ci
EduKav1813 Dec 11, 2024
6cfb290
.github/workflows/test.yml: temporarily desable build stage
EduKav1813 Dec 12, 2024
4d9a3fa
.github/workflows/test.yml: replace robot with run.sh calling dts tes…
EduKav1813 Dec 12, 2024
4f48ae0
.github/workflows/test.yml: disable run-tests's dependency on build-dts
EduKav1813 Dec 12, 2024
7bd75d2
.github/workflows/test.yml: fix isolating robot arguments for run.sh
EduKav1813 Dec 12, 2024
6164d8a
.github/workflows/test.yml: fix result log file path
EduKav1813 Dec 12, 2024
0cb6e8b
Revert ".github/workflows/test.yml: fix result log file path"
EduKav1813 Dec 12, 2024
13de755
.github/workflows/test.yml: rework passing env variables to run.sh
EduKav1813 Dec 12, 2024
4ca9bd5
fixup! .github/workflows/test.yml: rework passing env variables to ru…
EduKav1813 Dec 12, 2024
544fb90
.github/workflows/test.yml: fix spaces before and after matrix variab…
EduKav1813 Dec 12, 2024
cf7a9ee
.github/workflows/test.yml: move testcase name from robot arguments t…
EduKav1813 Dec 12, 2024
9f39e5f
.github/workflows/test.yml: move snipeit variable closer to robot
EduKav1813 Dec 12, 2024
4b6cf0b
.github/workflows/test.yml: attempt to fix run.sh arguments
EduKav1813 Dec 12, 2024
5be1b20
workflows/test.yml: fix moving the test log with find
EduKav1813 Dec 13, 2024
e4bd401
workflows/test.yml: fix current directory for log transfer
EduKav1813 Dec 13, 2024
4d13f32
workflows/test.yml: TEMPORARY change ipxe script filepaths
EduKav1813 Dec 13, 2024
f49dbb6
fixup! workflows/test.yml: TEMPORARY change ipxe script filepaths
EduKav1813 Dec 13, 2024
a31ff4f
workflows/test.yml: Add step with testing secrets mask
EduKav1813 Dec 13, 2024
5522652
test
m-iwanicki Dec 13, 2024
1074e41
workflows/test.yml: fix dpp_logs_key variable passing
EduKav1813 Dec 13, 2024
f886cf7
workflows/test.yml: split run.sh command to multiple lines
EduKav1813 Dec 13, 2024
eaf54a2
workflow: concurrency - allow only one workflow to run
m-iwanicki Dec 13, 2024
6303eb3
workflows/test.yml: Change dpp_logs_key for pcengines_seabios to
EduKav1813 Dec 17, 2024
85eb55e
workflows/test.yml: fix matrix configuration
EduKav1813 Dec 17, 2024
db49103
workflows/test.yml: cleanup temporary changes
EduKav1813 Dec 17, 2024
8de5a6c
fixup! workflows/test.yml: cleanup temporary changes
EduKav1813 Dec 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 56 additions & 60 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,15 @@ name: Run DTS tests
on:
pull_request:
branches:
- 'main'
- "main"
jobs:
build-dts:
uses: ./.github/workflows/build.yml
with:
cacheless: false
run-tests:
name: Run DTS tests

prepare-for-tests:
m-iwanicki marked this conversation as resolved.
Show resolved Hide resolved
name: Prepare for DTS tests
if: ${{ github.head_ref == 'develop' && contains(join(needs.*.result, ','), 'success') }}
needs: build-dts
runs-on:
Expand All @@ -19,23 +20,13 @@ jobs:
- name: Checkout OSFV repo
uses: actions/checkout@v4
with:
repository: 'Dasharo/open-source-firmware-validation'
path: 'open-source-firmware-validation'
submodules: 'recursive'
ref: 'develop'
repository: "Dasharo/open-source-firmware-validation"
path: "open-source-firmware-validation"
submodules: "recursive"
ref: "develop"
- name: Run IPXE server
shell: bash
run: |
mkdir ipxe
cp build/tmp/deploy/images/genericx86-64/dts-base-image-genericx86-64.cpio.gz ipxe
cp build/tmp/deploy/images/genericx86-64/bzImage ipxe
echo -e "\n
#!ipxe\n
imgfetch --name file_kernel bzImage\n
imgfetch --name file_initrd dts-base-image-genericx86-64.cpio.gz\n
kernel file_kernel root=/dev/nfs initrd=file_initrd\n
boot" > ipxe/dts.ipxe
cd ipxe && python3 -m http.server 4321 &
run: ./open-source-firmware-validation/scripts/ci/ipxe-run.sh
- name: Install requirements
shell: bash
run: |
Expand All @@ -51,15 +42,48 @@ jobs:
mkdir qemu-data
touch qemu-data/hdd.qcow
./qemu-run.sh nographic os &
- name: Create directory for logs
shell: bash
id: log_dirs
run: |
timestamp=$(date -u +%Y-%m-%dT%H:%M:%S%Z)
directory="/tmp/dts-test-ci-${timestamp}"
mkdir $directory
echo "directory=$directory" >> "$GITHUB_OUTPUT"
- name: Run tests
m-iwanicki marked this conversation as resolved.
Show resolved Hide resolved

run-tests:
name: Run DTS tests
if: ${{ github.head_ref == 'develop' && contains(join(needs.*.result, ','), 'success') }}
needs: build-dts
runs-on:
labels: dts-builder
strategy:
matrix:
include:
- test: msi
dpp_download_key: $MSI_DOWNLOAD
dpp_logs_key: $MSI_LOGS
log_file: output_msi.log

- test: msi_heads
dpp_download_key: $MSI_HEADS_DOWNLOAD
dpp_logs_key: $MSI_HEADS_LOGS
log_file: output_msi_heads.log

- test: optiplex
dpp_download_key: $OPTIPLEX_DOWNLOAD
dpp_logs_key: $OPTIPLEX_LOGS
log_file: output_nc_heads.log

- test: novacustom_heads
dpp_download_key: $OPTIPLEX_DOWNLOAD
dpp_logs_key: $OPTIPLEX_LOGS
log_file: output_nc_heads.log

- test: pcengines
dpp_download_key: $PCENGINES_DOWNLOAD
dpp_logs_key: $PCENGINES_LOGS
log_file: output_pcengines.log

- test: pcengines_seabios
m-iwanicki marked this conversation as resolved.
Show resolved Hide resolved
dpp_download_key: $PCENGINES_SEABIOS_DOWNLOAD
dpp_logs_key: $PCENGINES_SEABIOS_LOGS
log_file: output_pcengines_seabios.log

steps:
- name: Run ${{ matrix.config.test }}
shell: bash
env:
LOG_DIR: ${{ steps.log_dirs.outputs.directory }}
Expand All @@ -76,41 +100,13 @@ jobs:
source ~/.secrets/dpp-keys
ip_addr=$(ip -o -4 addr list eno2 | awk '{print $4}' | cut -d/ -f1)

robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no \
-v dpp_password:$DPP_PASSWORD -v dpp_download_key:$MSI_DOWNLOAD \
-v dpp_logs_key:$MSI_LOGS -v netboot_utilities_support:True \
-v dts_ipxe_link:http://${ip_addr}:4321/dts.ipxe \
-i "msi" dts/dts-e2e.robot 2>&1 | tee $LOG_DIR/output_msi.log | grep "| PASS |\|| FAIL |"

robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no \
-v dpp_password:$DPP_PASSWORD -v dpp_download_key:$MSI_HEADS_DOWNLOAD \
-v dpp_logs_key:$MSI_HEADS_LOGS -v netboot_utilities_support:True \
./scripts/run.sh -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no \
-v dpp_password:$DPP_PASSWORD -v dpp_download_key:${{ matrix.config.dpp_download_key }} \
-v dpp_logs_key:${{ matrix.config.dpp_logs_key }} -v netboot_utilities_support:True \
m-iwanicki marked this conversation as resolved.
Show resolved Hide resolved
-v dts_ipxe_link:http://${ip_addr}:4321/dts.ipxe \
-i "msi_heads" dts/dts-e2e.robot 2>&1 | tee $LOG_DIR/output_msi_heads.log | grep "| PASS |\|| FAIL |"
-i ${{ matrix.config.test }} dts/dts-e2e.robot 2>&1 \
| tee $LOG_DIR/${{ matrix.config.log_file }} | grep "| PASS |\|| FAIL |"

robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no \
-v dpp_password:$DPP_PASSWORD -v dpp_download_key:$OPTIPLEX_DOWNLOAD \
-v dpp_logs_key:$OPTIPLEX_LOGS -v netboot_utilities_support:True \
-v dts_ipxe_link:http://${ip_addr}:4321/dts.ipxe \
-i "optiplex" dts/dts-e2e.robot 2>&1 | tee $LOG_DIR/output_optiplex.log | grep "| PASS |\|| FAIL |"

robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no \
-v dpp_password:$DPP_PASSWORD -v dpp_download_key:$NOVACUSTOM_HEADS_DOWNLOAD \
-v dpp_logs_key:$NOVACUSTOM_HEADS_LOGS -v netboot_utilities_support:True \
-v dts_ipxe_link:http://${ip_addr}:4321/dts.ipxe \
-i "novacustom_heads" dts/dts-e2e.robot 2>&1 | tee $LOG_DIR/output_nc_heads.log | grep "| PASS |\|| FAIL |"

robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no \
-v dpp_password:$DPP_PASSWORD -v dpp_download_key:$PCENGINES_DOWNLOAD \
-v dpp_logs_key:$PCENGINES_LOGS -v netboot_utilities_support:True \
-v dts_ipxe_link:http://${ip_addr}:4321/dts.ipxe \
-i "pcengines" dts/dts-e2e.robot 2>&1 | tee $LOG_DIR/output_pcengines.log | grep "| PASS |\|| FAIL |"

robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no \
-v dpp_password:$DPP_PASSWORD -v dpp_download_key:$PCENGINES_SEABIOS_DOWNLOAD \
-v dpp_logs_key:$PCENGINES_SEABIOS_LOGS -v netboot_utilities_support:True \
-v dts_ipxe_link:http://${ip_addr}:4321/dts.ipxe \
-i "pcengines_seabios" dts/dts-e2e.robot 2>&1 | tee $LOG_DIR/output_pcengines_seabios.log | grep "| PASS |\|| FAIL |"
- name: Copy log
shell: bash
env:
Expand Down