Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Split the Build Job arm-05 into multiple smaller jobs #83

Closed
wants to merge 1 commit into from

Conversation

lupyuen
Copy link

@lupyuen lupyuen commented Oct 9, 2024

Summary

CI Build Job arm-05 (runtime 2 hours) has become the Performance Bottleneck for CI Workflow. That's because arm-05 builds too many targets for nRF, RP2040, SAM 3, SAM A and SAM D. This PR splits arm-05 into multiple smaller jobs, to reduce the CI Build Duration.

Before the PR: arm-05 is overloaded, build requires 2 hours

  • arm-05 (2 hours): nRF, RP2040, SAM 3, SAM A, SAM D
  • arm-06 (56 mins): STM32 [a-m]*

After the PR: arm-05 is offloaded (to arm-06 and arm-07), completes within 1 hour

  • arm-05 (47 mins): nRF
  • arm-06 (1 hour): Reserve for RP2040 exclusively
  • arm-07 (1 hour 15 mins): SAM 3, SAM A, SAM D, STM32 [a-m]*

Build Jobs are sorted by Target Name. So we cascade the changes and rename the Build Jobs: arm-07 becomes arm-08, arm-08 becomes arm-09 etc. Then arm-13 becomes a new job arm-14. (Which we added to build.yml)

Performance of arm-05 is discussed in apache#13775 and apache#12773

Impact

With this PR, Arm32 Build Jobs will finish earlier:

Before the PR: Arm32 Build Jobs take between 29 mins (arm-10) to 2 hours (arm-05) to complete.

After the PR: Arm32 Build Jobs will take between 28 mins (arm-11) to 1 hour 15 mins (arm-07) to complete.

The Updated CI Workflow shall be synced to nuttx-apps repo in the next PR.

Testing

We verified that Arm32 Build Jobs are executed successfully with the Updated CI Workflow: https://github.com/lupyuen5/label-nuttx/actions/runs/11240191619

Timings for the Arm32 Build Jobs:

Before the PR

  • arm-01: 58 mins
  • arm-02: 1 hour 7 mins
  • arm-03: 34 mins
  • arm-04: 35 mins
  • arm-05: 1 hour 59 mins
  • arm-06: 56 mins
  • arm-07: 42 mins
  • arm-08: 43 mins
  • arm-09: 34 mins
  • arm-10: 29 mins
  • arm-11: 35 mins
  • arm-12: 1 hour
  • arm-13: 48 mins

After the PR

  • arm-01: 57 mins
  • arm-02: 1 hour 10 mins
  • arm-03: 34 mins
  • arm-04: 37 mins
  • arm-05: 47 mins
  • arm-06: 1 hour 1 min
  • arm-07: 1 hour 15 mins
  • arm-08: 43 mins
  • arm-09: 44 mins
  • arm-10: 35 mins
  • arm-11: 28 mins
  • arm-12: 34 mins
  • arm-13: 1 hour
  • arm-14: 49 mins

@lupyuen lupyuen changed the title Add arm-14 CI: Split the Build Job arm-05 into multiple smaller jobs Oct 9, 2024
CI Build Job `arm-05` (runtime 2 hours) has become the Performance Bottleneck for CI Workflow. That's because `arm-05` builds too many targets for nRF, RP2040, SAM 3, SAM A and SAM D. This PR splits `arm-05` into multiple smaller jobs, to reduce the CI Build Duration.

Before the PR: `arm-05` is overloaded, build requires 2 hours
- `arm-05` (2 hours): nRF, RP2040, SAM 3, SAM A, SAM D
- `arm-06` (56 mins): STM32 [a-m]*

After the PR: `arm-05` is offloaded (to `arm-06` and `arm-07`), completes within 1 hour
- `arm-05` (47 mins): nRF
- `arm-06` (1 hour): Reserve for RP2040 exclusively
- `arm-07` (1 hour 15 mins): SAM 3, SAM A, SAM D, STM32 [a-m]*

Build Jobs are sorted by Target Name. So we cascade the changes and rename the Build Jobs: `arm-07` becomes `arm-08`, `arm-08` becomes `arm-09` etc. Then `arm-13` becomes a new job `arm-14`. (Which we added to `build.yml`)

Performance of `arm-05` is discussed in apache#13775 and apache#12773
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant