Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate upload/sync of dependencies to pytorch s3 index #6097

Open
atalman opened this issue Dec 20, 2024 · 1 comment
Open

Automate upload/sync of dependencies to pytorch s3 index #6097

atalman opened this issue Dec 20, 2024 · 1 comment

Comments

@atalman
Copy link
Contributor

atalman commented Dec 20, 2024

We would like to solve the issue of adding and synchronising new dependencies to download pytoch.org

We do have following script: https://github.com/pytorch/test-infra/blob/main/s3_management/update_dependencies.py that run manually, which upload latest available binaries to pytorch nightly and test index.

I would like to create workflow that would automatically update required dependencies on push changes to this script, so that we can easily add new Cuda or Xpu dependencies.

Rather then syncing latest we should have {package, version} tuple that we want to sync.

We should perhaps do this in stages:

  1. Enable certain packages to be synced with pinned version on push changes
  2. Enable automatic sync of latest packages to the index on daily basis

cc @malfet @ptrblck @tinglvv @nWEIdia

@huydhn
Copy link
Contributor

huydhn commented Jan 7, 2025

To clarify, this is to automatically update new version of some dependencies like numpy, or latest nvidia libraries

@huydhn huydhn moved this to Cold Storage in PyTorch OSS Dev Infra Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Cold Storage
Development

No branches or pull requests

2 participants