How to ship a package for Tegra/Jetson?

[leofang]

@kvoronin raised this question offline. Starting v0.6.2 cuSPARSELt supports sm87:
https://docs.nvidia.com/cuda/cusparselt/release_notes.html#cusparselt-v0-6-2

[kvoronin]

Per offline discussion, currently the arm_variant needs to be extended to support this and since there are no actual requests for getting cusparselt for jetson via conda, for now the jetson package will not be distributed over conda.

[traversaro]

Hello @kvoronin @leofang, sorry for commenting on a closed issue. I work at the Italian Institute of Technology (https://www.iit.it), and indeed we tried to use conda-forge CUDA packages on Jetson Orin, and we hit the problem of cublas and cudnn not having sm_87 architecture (see conda-forge/pytorch-cpu-feedstock#303). We can work on adding the sm_87 architecture on open source packages (either directly in the recipe or building our own forks), but clearly we can't easily create cudnn and cublas sm_87-enabled packages.

I guess it would not be sufficient a single "actual requests" from us to convince NVIDIA to allocate work to add sm_87 architectures to conda-forge's cuda linux-aarch64, however I want to ask: what is the correct channel to "actual request" for sm_87-support in conda-forge's CUDA packages? Thanks in advance!

fyi @conda-forge/cuda

[traversaro]

but clearly we can't easily create cudnn and cublas sm_87-enabled packages.

Actually I am not sure this is true, as the redist packages available at https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/ used for cudnn and cublas packaging have two variants, linux-sbsa and linux-aarch64 . In the linux-aarch64 packages the sm87 build is indeed contained, see:

traversaro@IITBMP014LW012:~/cudaws$ cuobjdump -lelf ./cudnn-linux-aarch64-9.3.0.75_cuda12-archive/lib/libcudnn_engines_runtime_compiled.so | head -n 12
ELF file    1: tmpxft_0000043f_00000000-0.sm_53.cubin
ELF file    2: tmpxft_0000043f_00000000-1.sm_61.cubin
ELF file    3: tmpxft_0000043f_00000000-2.sm_62.cubin
ELF file    4: tmpxft_0000043f_00000000-3.sm_70.cubin
ELF file    5: tmpxft_0000043f_00000000-4.sm_72.cubin
ELF file    6: tmpxft_0000043f_00000000-5.sm_75.cubin
ELF file    7: tmpxft_0000043f_00000000-6.sm_80.cubin
ELF file    8: tmpxft_0000043f_00000000-7.sm_86.cubin
ELF file    9: tmpxft_0000043f_00000000-8.sm_87.cubin
ELF file   10: tmpxft_0000043f_00000000-9.sm_90.cubin
ELF file   11: tmpxft_0000043f_00000000-10.sm_80.cubin
ELF file   12: tmpxft_0000043f_00000000-11.sm_86.cubin
traversaro@IITBMP014LW012:~/cudaws$ cuobjdump -lelf ./cudnn-linux-sbsa-9.3.0.75_cuda12-archive/lib/libcudnn_engines_runtime_compiled.so | head -n 12
ELF file    1: tmpxft_00000443_00000000-0.sm_50.cubin
ELF file    2: tmpxft_00000443_00000000-1.sm_60.cubin
ELF file    3: tmpxft_00000443_00000000-2.sm_61.cubin
ELF file    4: tmpxft_00000443_00000000-3.sm_70.cubin
ELF file    5: tmpxft_00000443_00000000-4.sm_75.cubin
ELF file    6: tmpxft_00000443_00000000-5.sm_80.cubin
ELF file    7: tmpxft_00000443_00000000-6.sm_86.cubin
ELF file    8: tmpxft_00000443_00000000-7.sm_90.cubin
ELF file    9: tmpxft_00000443_00000000-8.sm_80.cubin
ELF file   10: tmpxft_00000443_00000000-9.sm_86.cubin
ELF file   11: tmpxft_00000443_00000000-10.sm_90.cubin
ELF file   12: tmpxft_00000443_00000000-11.sm_80.cubin

so we use this own jetson-compatible cudnn and cublas packages. So I guess the problem is to understand if this is packages can be packaged by conda, but for that I think it is sufficient to open an issue in the feedstock, sorry for the noise!

fyi @giotherobot

[kvoronin]

• @jakirkham for awareness/discussion. In Enable support for packages targeting Tegra devices on the channel cuda-feedstock#57 I see references to the drafts for attempts to add a tegra arm variant, will this work (I assume, not, which is the reason we didn't do it for cusparselt and other packages)?

[jakirkham]

Thanks Kirill! 🙏

Discussed this with Silvio in this thread: conda-forge/libcublas-feedstock#32 (comment)

After which Silvio opened the cuda-feedstock issue that you linked

We can discuss what will be needed to move that forward in our next meeting