Switch presubmit CI workflows to use pinned IREE versions. #774
Conversation
requirements-iree-pinned.txt
Outdated
|
|
||
| # TODO(#760): include iree-turbine in this requirements file too? | ||
| # iree-turbine==3.1.0rc20241205 | ||
| iree-turbine==3.1.0rc20241205 |
There was a problem hiding this comment.
All Llama sharktank/shortfin pre-submits pass on latest IREE release, so shouldn't be an issue updating to latest release. (Not sure about sdxl)
There was a problem hiding this comment.
They pass on the latest IREE release today. They might not in the future.
We'll catch issues as part of version pin update PRs, like #773.
There was a problem hiding this comment.
To be clear with this PR: this isn't a temporary stabilizing measure, this is rolling out a methodical way of managing versions. At the moment we only have manually created PRs to update the versions. I'm looking into tools like dependabot to automate that.
There was a problem hiding this comment.
Via, conversation in previous PR, this version should be fine!
There was a problem hiding this comment.
Or maybe not actually... shark-ai is currently passing in main, but seems to be failing at the compilation step in this PR
There was a problem hiding this comment.
Oh, good catch. I can update the pin in this PR too then 🤔
|
Oh this is weird... shortfin tests are hanging here now too, just like on #773, despite keeping the runtime version pin fixed. Sample logs: https://github.com/nod-ai/shark-ai/actions/runs/12658739223/job/35276342245?pr=774 |
It looks like ci-libshortfin.yml uses requirements-iree-pinned.txt, which was updated to the same version as #773 (20250107) in this PR. Functionally, maybe the lower-level error isn't properly handled, causing the main thread to hang? |
Found a few issues and sent fixes: |
| iree-base-compiler==3.1.0rc20250107 | ||
| iree-base-runtime==3.1.0rc20250107 | ||
| iree-turbine==3.1.0rc20250107 |
There was a problem hiding this comment.
Updated the pin to our release candidate (last night's IREE build). I would have made that change incrementally, but there were compilation errors at https://github.com/nod-ai/shark-ai/actions/runs/12656984889/job/35270801217. I can't tell from the logs what the issue was though.
Progress on #760. The idea here is that we will test with only pinned versions in all workflows that run on `pull_request` and `push` triggers, then we will create pull requests (ideally via automation like dependabot) that attempt to update the pinned versions. This will give us confidence that test regressions are _only_ due to the code changes in the pull request and not due to a dependency changing. Workflows will also be more reproducible as the versions they fetch will come from source code and not an external, time-dependent source.
Progress on #760.
The idea here is that we will test with only pinned versions in all workflows that run on
pull_requestandpushtriggers, then we will create pull requests (ideally via automation like dependabot) that attempt to update the pinned versions. This will give us confidence that test regressions are only due to the code changes in the pull request and not due to a dependency changing. Workflows will also be more reproducible as the versions they fetch will come from source code and not an external, time-dependent source.