Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Release automation #282

Merged
merged 15 commits into from
Sep 24, 2024
Merged

ci: Release automation #282

merged 15 commits into from
Sep 24, 2024

Conversation

ko3n1g
Copy link
Collaborator

@ko3n1g ko3n1g commented Sep 6, 2024

What does this PR do ?

Implements a manual workflow_dispatch that allows us to build, test, and publish a PyPi wheel as well as create GH release (tag).

It is secured by a GH environment that runs on main (protected) only by a PyPi token that is resolved in this environment only. Put different: If one tries to use this token on a branch other than main it will be empty. So to make use of this token, they need to be able to merge their script into main, which of course only works by review.

The release page will be populated by the CHANGELOG.md file (awesome that you already have that!!). I extract the section from (incl) to (excl) ## ... blocks as the description of the release page.

Changelog

  • Please update the CHANGELOG.md under next version with high level changes in this PR.

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this 

Before your PR is "Ready for review"

Pre checks:

Checklist when contributing a new algorithm

  • Does the trainer resume and restore model state all states?
  • Does the trainer support all parallelism techniques(PP, TP, DP)?
  • Does the trainer support max_steps=-1 and validation?
  • Does the trainer only call APIs defined in alignable_interface.py?
  • Does the trainer have proper logging?

Additional Information

  • Related to # (issue)

@ko3n1g ko3n1g requested review from terrykong and gshennvm September 6, 2024 01:34
@github-actions github-actions bot added the CI label Sep 6, 2024
@ko3n1g ko3n1g marked this pull request as ready for review September 6, 2024 01:48
.github/workflows/release.yaml Show resolved Hide resolved
.github/workflows/release.yaml Show resolved Hide resolved
.github/workflows/release.yaml Outdated Show resolved Hide resolved
.github/workflows/release.yaml Outdated Show resolved Hide resolved
@ko3n1g ko3n1g requested a review from terrykong September 6, 2024 15:28
@ko3n1g
Copy link
Collaborator Author

ko3n1g commented Sep 6, 2024

@terrykong could you have a second look?

.github/workflows/release.yaml Show resolved Hide resolved
.github/workflows/release.yaml Outdated Show resolved Hide resolved
.github/workflows/release.yaml Show resolved Hide resolved
@ko3n1g ko3n1g requested a review from terrykong September 6, 2024 21:40
@ko3n1g ko3n1g force-pushed the ko3n1g/ci/release-automation branch from dfedb3a to c35aea0 Compare September 19, 2024 05:26
@ko3n1g ko3n1g added Run CICD Set + un-set to retrigger and removed Run CICD Set + un-set to retrigger labels Sep 19, 2024
@ko3n1g ko3n1g force-pushed the ko3n1g/ci/release-automation branch from 1c4ffda to b7d0704 Compare September 19, 2024 05:47
@ko3n1g ko3n1g added Run CICD Set + un-set to retrigger and removed Run CICD Set + un-set to retrigger labels Sep 19, 2024
@ko3n1g ko3n1g force-pushed the ko3n1g/ci/release-automation branch from 3afd376 to e23faa4 Compare September 20, 2024 17:00
@ko3n1g
Copy link
Collaborator Author

ko3n1g commented Sep 20, 2024

Create a protected environment to hold our PyPi secrets: https://github.com/NVIDIA/NeMo-Aligner/settings/environments

Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
f
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
Signed-off-by: Oliver Koenig <[email protected]>
f
Signed-off-by: Oliver Koenig <[email protected]>
@ko3n1g ko3n1g force-pushed the ko3n1g/ci/release-automation branch from dc97088 to 93c1cae Compare September 24, 2024 07:59
@ko3n1g
Copy link
Collaborator Author

ko3n1g commented Sep 24, 2024

Merging this now since I incorporated @terrykong 's feedback and also tested the shell commands locally during the last release. It ultimately can only be tested "live" anyway, so there's no point in holding this PR back

@ko3n1g ko3n1g merged commit b2165bd into main Sep 24, 2024
4 checks passed
@ko3n1g ko3n1g deleted the ko3n1g/ci/release-automation branch September 24, 2024 08:09
abukharin3 pushed a commit to abukharin3/NeMo-Aligner that referenced this pull request Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Run CICD Set + un-set to retrigger
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants