Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wheel builds: standardizations for 25.02 #136

Closed
16 tasks done
jameslamb opened this issue Jan 21, 2025 · 1 comment
Closed
16 tasks done

wheel builds: standardizations for 25.02 #136

jameslamb opened this issue Jan 21, 2025 · 1 comment
Assignees

Comments

@jameslamb
Copy link
Member

jameslamb commented Jan 21, 2025

Description

As more C++ wheels PRs (e.g. for #33) have gone through review, we've picked up a list of new preferred practices for RAPIDS wheels builds in CI. This tracks the work of going back through RAPIDS repos and applying all those suggestions.

Benefits of this work

  • faster CI (via more parallelism)
  • lower CI resource usage (via running fewer jobs)
  • better reliability of nightly CI jobs (via running fewer jobs)

Acceptance Criteria

  • for every RAPIDS repo producing wheels, all of those practices mentioned in the description are applied

Approach

Batch these changes into 1 PR per repo.

update-version.sh can be tested like this:

./ci/release/update-verison.sh '25.04.00'
git grep -E '25\.2'
git grep -E '25\.02'
git grep -E '24\.12'

These can be done in any order.

Updates

Preview Give feedback
@jameslamb jameslamb self-assigned this Jan 21, 2025
@jameslamb jameslamb changed the title C++ wheel builds: standardizations for 25.02 wheel builds: standardizations for 25.02 Jan 22, 2025
rapids-bot bot pushed a commit to rapidsai/cugraph-gnn that referenced this issue Jan 23, 2025
#111)

Part of rapidsai/build-planning#136, which tracks some building/packaging simplifications and conventions we'd like to standardize across RAPIDS.

This proposes the following:

* using `cmake-format` to autoformat CMake code
* using `cmake-lint` to enforce style preferences for CMake code
* removing unnecessary use of `-DDETECT_CONDA_ENV` for wheel builds
* explicitly passing package type to GitHub Actions / `gha-tools` things handling wheels

## Notes for Reviewers

The `cmake-format` / `cmake-lint` approach was copied directly from RAFT:

* https://github.com/rapidsai/raft/blob/596d4b7338e62a92652503cd76feaeaa187ad740/.pre-commit-config.yaml#L52
* https://github.com/rapidsai/raft/blob/596d4b7338e62a92652503cd76feaeaa187ad740/cpp/cmake/config.json
* https://github.com/rapidsai/raft/blob/596d4b7338e62a92652503cd76feaeaa187ad740/cpp/scripts/run-cmake-format.sh

Other RAPIDS projects ([like cuDF](https://github.com/rapidsai/cudf/blob/1f0f51f96b79edd820e81343ca521c684b1f4918/.pre-commit-config.yaml#L97)) do this the same way.

All formatting-only changes to CMake in this PR were made automatically by `cmake-foramt`.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Gil Forsyth (https://github.com/gforsyth)
  - https://github.com/linhu-nv

URL: #111
rapids-bot bot pushed a commit to rapidsai/ucx-py that referenced this issue Jan 23, 2025
Contributes to rapidsai/build-planning#136

This project's `update-version.sh` misses some RAPIDS dependencies in `pyproject.toml`. This fixes that.

Also updates to the latest `rapids-dependency-file-generator`.

## Notes for Reviewers

### How I tested this

```shell
git fetch upstream --tags
./ci/release/update-version.sh '0.43.00'
git grep -E '25\.2'
git grep -E '25\.02'
git grep -E '0\.42'
```

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)

URL: #1106
@jameslamb
Copy link
Member Author

enforcing rapids-cmake style preferences via cmake-format (rapidsai/raft#2531 (comment))

Based on some review conversations (rapidsai/cugraph#4889 (comment)), I'm cutting the CMake-formatting part out of this issue.

Opened a separate discussion for that: rapidsai/pre-commit-hooks#62

rapids-bot bot pushed a commit to rapidsai/cudf that referenced this issue Jan 27, 2025
Contributes to rapidsai/build-planning#136

For nightly builds, some `wheel-build-{project}` jobs currently wait to start until some other `wheel-publish-{dependency}` jobs complete. This is unnecessary... `wheel-build-{dependency}` jobs will upload packages to S3, which is where `wheel-build-{project}` jobs will download them from.

This proposes changing that such that all nightly `wheel-build-*` jobs depend only other `wheel-build-*` jobs. This should decrease the end-to-end time it takes for all wheels to be built and published on nightly / branch builds.

Also updates `pre-commit` config to the latest `rapids-dependency-file-generator` version.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

URL: #17792
rapids-bot bot pushed a commit to rapidsai/cuspatial that referenced this issue Jan 28, 2025
Contributes to rapidsai/build-planning#136

Proposes the following:

* having `wheel-build-cuspatial` depend on `wheel-build-libcuspatial` in nightly CI, not `wheel-publish-libcuspatial`
   - *`wheel-build-cuspatial` just needs `libcuspatial` wheels from S3, not nightly PyPI*
   - *doing this reduces the end-to-end time for nightly builds (via increasing parallelism)*

## Notes for Reviewers

#

Authors:
  - James Lamb (https://github.com/jameslamb)
  - Matthew Roeschke (https://github.com/mroeschke)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1516
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant