[FEA]: Introduce Python module with CCCL headers

[gevtushenko]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this request and that I agree to the Code of Conduct

Area

General CCCL

Is your feature request related to a problem? Please describe.

As of today, CCCL provides two Python modules: cuda.cooperative and cuda.parallel. Both modules have to package all C++ headers. This creates duplication.

Describe the solution you'd like

We have to extract CCCL headers into a standalone Python module for further reuse in cuda.cooperative and cuda.parallel. While doing so, we have to make sure that cuda.parallel and cuda.cooperative packages are version locked with cccl headers package. This will likely involve fixing CI script https://github.com/NVIDIA/cccl/blob/main/ci/test_pycuda.sh to install cccl headers before cuda.cooperative and also updating the https://github.com/NVIDIA/cccl/blob/main/ci/update_version.sh to implement version locking.

Describe alternatives you've considered

No response

Additional context

No response

[leofang]

Since we already have an official way to copy headers with CMake and it is utilized in building the cccl package on conda-forge, we should do the same for the standalone CCCL wheel. Using scikit-build-core will be able for us to reuse the same CMake infrastructure.

[rwgk]

@shwina for awareness.

I just created Draft PR #3201. I tried to keep this really simple, but I'm running into two problems. Copy-pasting the current PR description:

• pip install --editable does not work (anymore).

• pip install ./cuda_cccl needs to be run manually before running pip install ./cuda_parallel or ./cuda_cooperative

  • "cuda-cccl @ file://../cuda_cccl" in setup.py does not work (in a CCCL Dev Container): ValueError: non-local file URIs are not supported on this platform: 'file://../cuda_cccl'

I'll try to solve the pip install --editable problem first, because it's really important for development agility.

I think the best solution is to make cuda.cccl_headers (or some other name) importable as a Python module, and to move the get_paths() function there.