Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Compile AVR examples on Windows CI #1121

Merged
merged 4 commits into from
Jan 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 126 additions & 0 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
name: Run tests on Windows

on: [pull_request]

jobs:
windows_testing:
runs-on: windows-2022
env:
PYTHONIOENCODING: "utf-8"

steps:

# Disabling snake-oil for performance reasons
- name: Disable Windows Defender
run: Set-MpPreference -DisableRealtimeMonitoring $true

# This doesn't work due to files getting overwritten from inside the zip
# [System.IO.Compression.ZipFile]::ExtractToDirectory("gcc-arm-none-eabi-win64.zip", "C:\")
# And this manual expansion code
# function Unzip($zipfile, $outdir)
# {
# Add-Type -AssemblyName System.IO.Compression.FileSystem
# $archive = [System.IO.Compression.ZipFile]::OpenRead($zipfile)
# foreach ($entry in $archive.Entries)
# {
# $entryTargetFilePath = [System.IO.Path]::Combine($outdir, $entry.FullName)
# $entryDir = [System.IO.Path]::GetDirectoryName($entryTargetFilePath)
# if(!(Test-Path $entryDir )){
# New-Item -ItemType Directory -Path $entryDir | Out-Null
# }
# if (!$entryTargetFilePath.EndsWith("\") -And !$entryTargetFilePath.EndsWith("/")) {
# [System.IO.Compression.ZipFileExtensions]::ExtractToFile($entry, $entryTargetFilePath, $true);
# }
# }
# }
# Unzip -zipfile "gcc-arm-none-eabi-win64.zip" -outdir "C:\"
# is not faster than
# Expand-Archive -Path gcc-arm-none-eabi-win64.zip -DestinationPath C:\ -Force
- name: Download and Unzip GCCs
shell: powershell
run: |
$ProgressPreference = 'SilentlyContinue'
Start-Job {
Set-Location $using:PWD
Add-Type -Assembly "System.IO.Compression.Filesystem"
Invoke-WebRequest -OutFile gcc-win64.zip https://github.com/brechtsanders/winlibs_mingw/releases/download/12.2.0-15.0.6-10.0.0-msvcrt-r3/winlibs-x86_64-posix-seh-gcc-12.2.0-mingw-w64msvcrt-10.0.0-r3.zip
[System.IO.Compression.ZipFile]::ExtractToDirectory("gcc-win64.zip", "C:\")
}
Start-Job {
Set-Location $using:PWD
Invoke-WebRequest -OutFile gcc-arm-none-eabi-win64.zip https://developer.arm.com/-/media/Files/downloads/gnu/12.2.rel1/binrel/arm-gnu-toolchain-12.2.rel1-mingw-w64-i686-arm-none-eabi.zip
Expand-Archive -Path gcc-arm-none-eabi-win64.zip -DestinationPath C:\ -Force
}
Start-Job {
Set-Location $using:PWD
Add-Type -Assembly "System.IO.Compression.Filesystem"
Invoke-WebRequest -OutFile gcc-avr-win64.zip https://github.com/ZakKemble/avr-gcc-build/releases/download/v12.1.0-1/avr-gcc-12.1.0-x64-windows.zip
[System.IO.Compression.ZipFile]::ExtractToDirectory("gcc-avr-win64.zip", "C:\")
}
Get-Job | Wait-Job

- name: Install GCCs
if: always()
shell: powershell
run: |
dir C:\
dir C:\mingw64
dir C:\arm-gnu-toolchain-12.2.rel1-mingw-w64-i686-arm-none-eabi
dir C:\avr-gcc-12.1.0-x64-windows
echo "C:\mingw64\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "C:\arm-gnu-toolchain-12.2.rel1-mingw-w64-i686-arm-none-eabi\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "C:\avr-gcc-12.1.0-x64-windows\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
rm gcc-arm-none-eabi-win64.zip

- name: Install Python
if: always()
uses: actions/setup-python@v4
with:
python-version: "3.11"

- name: Install Python packages
if: always()
run: |
pip install --upgrade --upgrade-strategy=eager modm scons future

- name: Show Version Information
if: always()
run: |
gcc --version
g++ --version
make --version
arm-none-eabi-g++ --version
avr-g++ --version
lbuild --version
python -c "import os; print(os.cpu_count())"

- name: Check out repository
if: always()
uses: actions/checkout@v3
with:
submodules: 'recursive'

- name: Hosted Examples
if: always()
shell: bash
run: |
(cd examples && python ../tools/scripts/examples_compile.py linux/assert linux/block_device/ram linux/build_info linux/git linux/logger linux/printf linux/etl linux/fiber)

- name: Hosted Unittests
if: always()
shell: bash
run: |
(cd test && make run-hosted-windows)

- name: Compile STM32 Examples
if: always()
shell: bash
run: |
(cd examples && python ../tools/scripts/examples_compile.py nucleo_f031k6 nucleo_f103rb nucleo_f303re nucleo_f411re nucleo_f746zg)
(cd examples && python ../tools/scripts/examples_compile.py nucleo_g071rb nucleo_l031k6 nucleo_l152re nucleo_l476rg nucleo_g474re)

- name: Compile AVR Examples
if: always()
shell: bash
run: |
(cd examples && ../tools/scripts/examples_compile.py avr arduino_nano arduino_uno srxe)
64 changes: 0 additions & 64 deletions .github/workflows/windows_armcortexm.yml

This file was deleted.

66 changes: 0 additions & 66 deletions .github/workflows/windows_hosted.yml

This file was deleted.

6 changes: 3 additions & 3 deletions docs/src/guide/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ files in the next steps.

#### ARM Cortex-M

Install the [pre-built ARM toolchain via the 64-bit installer][toolchain-arm-xpack]
Install the [pre-built ARM toolchain via the 64-bit installer](https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads)
and make sure you select **Add path to environment variable** at the end!
Open a new command prompt to test the compiler:

Expand Down Expand Up @@ -416,12 +416,12 @@ picocom --baud 115200 --imap lfcrlf --echo /dev/ttyACM0
[openocd-install]: https://github.com/rleh/openocd-build
[udev-rules-openocd]: https://github.com/openocd-org/openocd/blob/master/contrib/60-openocd.rules#L84-L99
[usbipd]: https://github.com/dorssel/usbipd-win
[winavr]: https://blog.zakkemble.net/avr-gcc-builds/
[winavr]: https://github.com/ZakKemble/avr-gcc-build/releases/tag/v12.1.0-1
[windows-store-ubuntu-22-04-1-lts]: https://www.microsoft.com/store/productId/9PN20MSR04DW
[wingit]: https://git-scm.com/download/win
[winterm]: https://github.com/Microsoft/Terminal
[winwsl]: https://docs.microsoft.com/en-us/windows/wsl/about
[wsl2]: https://docs.microsoft.com/en-us/windows/wsl/about#what-is-wsl-2
[wsl2]: https://docs.microsoft.com/en-us/windows/wsl/about#what-is-wsl-2
[wsl-connect-usb]: https://docs.microsoft.com/en-us/windows/wsl/connect-usb
[wsl-install]: https://docs.microsoft.com/en-us/windows/wsl/install
[wsl-vscode]: https://docs.microsoft.com/en-us/windows/wsl/tutorials/wsl-vscode
Expand Down
1 change: 1 addition & 0 deletions examples/avr/xpcc/receiver/project.xml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<library>
<!-- CI: enable "Windows" not in platform.platform() -->
<options>
<option name="modm:target">atmega644-20au</option>
<option name="modm:platform:core:f_cpu">14.7456M</option>
Expand Down
1 change: 1 addition & 0 deletions examples/avr/xpcc/sender/project.xml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<library>
<!-- CI: enable "Windows" not in platform.platform() -->
<options>
<option name="modm:target">atmega644-20au</option>
<option name="modm:platform:core:f_cpu">14.7456M</option>
Expand Down
1 change: 0 additions & 1 deletion src/modm/platform/core/avr/module.lb
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ def build(env):
env.outbasepath = "modm/link"
env.copy("linkerscript.ld")
env.collect(":build:linkflags", "-L{project_source_dir}",
"-Wl,--no-warn-rwx-segment",
"-T{}".format(env.relcwdoutpath("modm/link/linkerscript.ld")))

env.substitutions = {
Expand Down
14 changes: 13 additions & 1 deletion tools/scripts/examples_compile.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,16 @@ def run_command(where, command, all_output=False):
output += result.stderr.decode("utf-8", errors="ignore").strip(" \n")
return (result.returncode, output)

def enable(projects):
filtered_projects = []
for project in projects:
if (query := re.search(r"<!-- CI: enable (.*?) -->", project.read_text())) is not None and not eval(query[1]):
print(f"Filtering out {project}: {query[1]}")
continue
filtered_projects.append(project)
return filtered_projects


def generate(project):
path = project.parent
output = ["=" * 90, "Generating: {}".format(path)]
Expand All @@ -54,7 +64,7 @@ def build(project):
commands.append( ("scons build --cache-show --random", "SCons") )
if ":build:make" in project_cfg and not is_running_on_windows:
commands.append( ("make build", "Make") )
elif ":build:cmake" in project_cfg:
elif ":build:cmake" in project_cfg and not is_running_on_windows:
build_dir = re.search(r'name=".+?:build:build.path">(.*?)</option>', project_cfg)[1]
cmd = "cmake -E make_directory {}/cmake-build-release; ".format(build_dir)
cmd += '(cd {}/cmake-build-release && cmake -DCMAKE_BUILD_TYPE=Release -G "Unix Makefiles" {}); '.format(build_dir, path.absolute())
Expand Down Expand Up @@ -107,6 +117,8 @@ def compile_examples(paths, jobs, split, part):
if split > 1:
chunk_size = math.ceil(len(projects) / args.split)
projects = projects[chunk_size*args.part:min(chunk_size*(args.part+1), len(projects))]
# Filter projects
projects = enable(projects)

# first generate all projects
with ThreadPool(jobs) as pool:
Expand Down
Loading