Attempt to remove AWS S3 flaky cache for sccache #2953
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…n rustc_wrapper is present
mfuntowicz
commented
Jan 25, 2025
Comment on lines
+97
to
+100
| RUN export CMAKE_C_COMPILER_LAUNCHER=sccache && \ | ||
| export CMAKE_CXX_COMPILER_LAUNCHER=sccache && \ | ||
| export CMAKE_CUDA_COMPILER_LAUNCHER=sccache && \ | ||
| mkdir $TGI_INSTALL_PREFIX && mkdir "$TGI_INSTALL_PREFIX/include" && mkdir "$TGI_INSTALL_PREFIX/lib" && \ |
There was a problem hiding this comment.
We don't care about these environment variables not being persisted
mfuntowicz
commented
Jan 25, 2025
Comment on lines
-3
to
-17
| if (NOT DEFINED CMAKE_CXX_COMPILER_LAUNCHER) | ||
| find_program(CCACHE_EXECUTABLE "ccache") | ||
| if (CCACHE_EXECUTABLE) | ||
| message(STATUS "Using ccache") | ||
| set(CMAKE_C_COMPILER_LAUNCHER "${CCACHE_EXECUTABLE}") | ||
| set(CMAKE_CXX_COMPILER_LAUNCHER "${CCACHE_EXECUTABLE}") | ||
| set(CMAKE_CUDA_COMPILER_LAUNCHER "${CCACHE_EXECUTABLE}") | ||
| endif () | ||
| else () | ||
| message(STATUS "Using user specified cmake cxx compiler launcher: ${CMAKE_CXX_COMPILER_LAUNCHER}") | ||
| set(CMAKE_C_COMPILER_LAUNCHER "${CMAKE_CXX_COMPILER_LAUNCHER}") | ||
| set(CMAKE_CXX_COMPILER_LAUNCHER "${CMAKE_CXX_COMPILER_LAUNCHER}") | ||
| set(CMAKE_CUDA_COMPILER_LAUNCHER "${CMAKE_CXX_COMPILER_LAUNCHER}") | ||
| endif () | ||
|
|
There was a problem hiding this comment.
This is a purely manual thing to specify a compiler launcher, remove any way to look smart here :D
mfuntowicz
commented
Jan 25, 2025
| @@ -14,7 +14,7 @@ const TENSORRT_ROOT_DIR: Option<&str> = option_env!("TENSORRT_ROOT_DIR"); | |||
| const NCCL_ROOT_DIR: Option<&str> = option_env!("NCCL_ROOT_DIR"); | |||
|
|
|||
| const IS_GHA_BUILD: LazyLock<bool> = LazyLock::new(|| { | |||
| option_env!("IS_GHA_BUILD").map_or(false, |value| match value.to_lowercase().as_str() { | |||
| option_env!("SCCACHE_GHA_ENABLED").map_or(false, |value| match value.to_lowercase().as_str() { | |||
There was a problem hiding this comment.
We are setting this variable to ON to make sccache outputs to GHA caching layer.
So it should be true in GHA contexts and false otherwise
mfuntowicz
commented
Jan 25, 2025
Comment on lines
+44
to
+50
| - name: Inject required variables for sccache to interact with Github Actions Cache | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || ''); | ||
| core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); | ||
|
|
There was a problem hiding this comment.
This new step is required to expose the cache parameters inside the job so we can use in building step for TRTLLM forwarding to sccache
Narsil
approved these changes
Jan 27, 2025
| cache-from: type=s3,region=us-east-1,bucket=ci-docker-buildx-cache,name=text-generation-inference-cache${{ env.LABEL }},mode=min,access_key_id=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_ACCESS_KEY_ID }},secret_access_key=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_SECRET_ACCESS_KEY }},mode=min | ||
| cache-to: type=s3,region=us-east-1,bucket=ci-docker-buildx-cache,name=text-generation-inference-cache${{ env.LABEL }},mode=min,access_key_id=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_ACCESS_KEY_ID }},secret_access_key=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_SECRET_ACCESS_KEY }},mode=min | ||
| cache-from: type=s3,region=us-east-1,bucket=ci-docker-buildx-cache,name=text-generation-inference-cache${{ env.LABEL }},mode=min,access_key_id=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_ACCESS_KEY_ID }},secret_access_key=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_SECRET_ACCESS_KEY }},mode=max | ||
| cache-to: type=s3,region=us-east-1,bucket=ci-docker-buildx-cache,name=text-generation-inference-cache${{ env.LABEL }},mode=min,access_key_id=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_ACCESS_KEY_ID }},secret_access_key=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_SECRET_ACCESS_KEY }},mode=max |
There was a problem hiding this comment.
Letś try max. I think we reverted to min because it was uploading way too many layers, leading to actual degradation in runtime.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Leverage GHA cache rather than S3