BUG: corrupted build string in direct violation of its specification #5571
Labels
type::bug
describes erroneous operation, use severity::* to classify the type
Comments
h-vetinari
added
the
type::bug
This was referenced Dec 19, 2024
Merged
|
Attaching CI logs from the offending PR in conda-forge/blas-feedstock#130 @ 2b61a42. |
|
I'm poking around and tried to undo the Maybe the equals-separated form is bypassing those early checks, unintentionally? |
|
If I undo all the equals-separated pin-exact=True replacements: |
|
You can see why the equals-separated string doesn't run into this: def ensure_matching_hashes(output_metadata):
envs = "build", "host", "run"
problemos = []
for _, m in output_metadata.values():
for _, om in output_metadata.values():
if m != om:
run_exports = om.get_value("build/run_exports", [])
if hasattr(run_exports, "keys"):
run_exports_list = []
for export_type in utils.RUN_EXPORTS_TYPES:
run_exports_list = run_exports_list + run_exports.get(
export_type, []
)
run_exports = run_exports_list
deps = _get_all_dependencies(om, envs) + run_exports
for dep in deps:
if (
dep.startswith(m.name() + " ")
and len(dep.split(" ")) == 3 # <---- See this check!
and dep.split(" ")[-1] != m.build_id()
and _variants_equal(m, om)
):
problemos.append((m.name(), m.build_id(), dep, om.name()))
if problemos:
error = ""
for prob in problemos:
error += "Mismatching package: {} (id {}); dep: {}; consumer package: {}\n".format(
*prob
)
raise RecipeError(
"Mismatching hashes in recipe. Exact pins in dependencies "
"that contribute to the hash often cause this. Can you "
"change one or more exact pins to version bound constraints?\n"
"Involved packages were:\n" + error
)It only considers the three-field variant of MatchSpec (instead of parsing it). If this is a false positive, some part of that |
|
So what's the suggestion...? Remove the Also, I was wondering if the bit isn't something that should have been solved by the glob-merge logic (conda/conda@6db7b91). Or has this somehow not arrived in conda-build yet? |
Sidenote: I had been completely unaware about the the divergences when using It's incomprehensible to me that this divergence would serve some purpose other than being a major footgun. PS. Thanks a lot for digging into this! 🙏 |
Not exactly. I was just mentioning that it looks like conda-build had some logic to detect this magic hash contamination early, but that the equals bypassed it because they pre-parse fields assuming space separators. There should not be any difference but in practice we should lint against
Only 2 and 3 are equivalent. The first is an exact match, so |
I think that check is not considering |
|
The function was introduced in #2065, then modified in #3237 (big PR, better see this blame) Also, #4603, but that's just a cleanup I think?. In these PRs we could still see the original comments that explain why this is needed to begin with: # The loop above gives us enough info to determine the build order. After that,
# we can "finalize" the metadata (fill in version pins) for the build metadata.
# This does not, however, account for circular dependencies
# where a runtime exact dependency pin on a downstream build
# subpackage in an upstream subpackage changes the upstream's
# hash, which then changes the downstream package's hash, which
# then recurses infinitely
# In general, given upstream a and downstream b, b can depend on a
# exactly, but a can only have version constraints on b at run or
# run_exports, not exact=True
# 3 passes here:
# 1. fill in fully-resolved build-time dependencies
# 2. fill in fully-resolved run-time dependencies. Note that circular deps
# are allowed, but you can't have exact=True for circular run-time deps
# 3. finally, everything should be filled in and done.
conda_packages = finalize_outputs_pass(self, conda_packages, pass_no=0,
permit_unsatisfiable_variants=permit_unsatisfiable_variants)
# Sanity check: if any exact pins of any subpackages, make sure that they match
ensure_matching_hashes(conda_packages)However I'm having trouble parsing that text 😬 I'll need my morning brain I think 😂 |
|
I patched the I searched this hash locally and it does appear in the pkgs/ cache: {
"info": {
"subdir": "osx-64"
},
"packages": {},
"packages.conda": {
"blas-2.127-blish150452e.conda": {
"build": "blish150452e",
"build_number": 27,
"depends": [
"blas-devel 3.9.0 27_*_blis"
],
"license": "BSD-3-Clause",
"md5": "731ebd99f30c1b3fd2dc55e84e5436a8",
"name": "blas",
"sha256": "73b9c4abacad3cb8524556fa7b231b8f1b034731acb10afb6eee12af47010278",
"size": 16824,
"subdir": "osx-64",
"timestamp": 1737474981292,
"version": "2.127"
},
"blas-devel-3.9.0-27_he1c578d_blis.conda": {
"build": "27_he1c578d_blis",
"build_number": 27,
"depends": [
"blis 0.9.0.*",
"libblas 3.9.0 27_*_blis",
"libcblas 3.9.0 27_*_blis",
"liblapack 3.9.0 *_netlib",
"liblapacke 3.9.0 *_netlib"
],
"license": "BSD-3-Clause",
"md5": "c8befb15db1b0cf076f6682d9197982a",
"name": "blas-devel",
"sha256": "7b7f01e25d9938186e02335a5406515b9baaddc8ac080bcd0b18faa91b4cdc46",
"size": 16398,
"subdir": "osx-64",
"timestamp": 1737474966930,
"version": "3.9.0"
},
"libblas-3.9.0-27_h09e100e_blis.conda": {
"build": "27_h09e100e_blis",
"build_number": 27,
"constrains": [
"libcblas 3.9.0 27_*_blis",
"blas 2.127 blis"
],
"depends": [
"blis >=0.9.0,<0.9.1.0a0"
],
"license": "BSD-3-Clause",
"md5": "4e7c08108d69a415c317583edf563886",
"name": "libblas",
"sha256": "81b547d9f02d75ce274595cdc464aa786644046217b1f384991ad3911bd67b30",
"size": 16712,
"subdir": "osx-64",
"timestamp": 1737474957462,
"track_features": "blas_blis",
"version": "3.9.0"
},
"libcblas-3.9.0-27_hb1fc1ef_blis.conda": {
"build": "27_hb1fc1ef_blis",
"build_number": 27,
"constrains": [
"blas 2.127 blis"
],
"depends": [
"libblas 3.9.0 27_*_blis"
],
"license": "BSD-3-Clause",
"md5": "9e5e5b4dee396c1f33d984b9d4ea694e",
"name": "libcblas",
"sha256": "22eed66e2a4c857d4e119c388bd92c5fb2884e376e68167de04b33f68c1a0e50",
"size": 16691,
"subdir": "osx-64",
"timestamp": 1737474961943,
"track_features": "blas_blis",
"version": "3.9.0"
}
},
"removed": [],
"repodata_version": 1
}So we know it was generated by conda-build as such, and we are not hallucinating it in the solver side of things. The package itself contains this {
"arch": "x86_64",
"build": "blish150452e",
"build_number": 27,
"depends": [
"blas-devel 3.9.0 27_*_blis"
],
"license": "BSD-3-Clause",
"name": "blas",
"platform": "osx",
"subdir": "osx-64",
"timestamp": 1737478329382,
"version": "2.127"
}So once again, this is conda-build artifact generation pipeline, not a bug in conda-index. Patched bit: for dep in deps:
if (
dep.startswith(m.name() + " ")
and len(dep.split(" ")) == 3
and not GlobLowerStrMatch(
dep.split(" ")[-1]
).match(GlobLowerStrMatch(m.build_id()))
and _variants_equal(m, om)
):
problemos.append((m.name(), m.build_id(), dep, om.name())) |
|
Interestingly, the |
|
I can verify that in conda-build/conda_build/metadata.py Line 1735 in cf35214 This is because conda-build/conda_build/metadata.py Line 1739 in cf35214 liblapacke, not blas! And that output does export PKG_HASH so it crosses results somehow. I'll investigate why extract_package_and_build_text() is buggy.
|
|
Ok, it comes from here conda-build/conda_build/metadata.py Lines 2224 to 2227 in cf35214 Because of the Jinja conda-build/conda_build/metadata.py Lines 2247 to 2248 in cf35214 For now, I think this should work if Edit: Yea, confirmed: |
This was referenced Jan 25, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Checklist
What happened?
Our blas metapackage is surely not trivial and ends up pushing conda-build into unusual corners (there are several bugs in addition to this one; perhaps I get around to raising more issues).
This issue is pretty blatant though. As of h-vetinari/blas-feedstock@4a63a00, the recipe contains:
And yet, when the blis-variant gets built, what we see is (added linebreaks for readability)
That's despite the fully rendered recipe (in the logs) looking like this:
Later this obviously falls over when it doesn't match expectations anymore:
Conda Info
++ conda info active environment : base active env location : /opt/conda shell level : 1 user config file : /home/conda/.condarc populated config files : /opt/conda/.condarc /home/conda/.condarc conda version : 24.11.1 conda-build version : 24.11.2 python version : 3.12.8.final.0 solver : libmamba (default) virtual packages : __archspec=1=broadwell __conda=24.11.1=0 __glibc=2.34=0 __linux=6.5.0=0 __unix=0=0 base environment : /opt/conda (writable) conda av data dir : /opt/conda/etc/conda conda av metadata url : None channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch package cache : /home/conda/feedstock_root/build_artifacts/pkg_cache /opt/conda/pkgs envs directories : /opt/conda/envs /home/conda/.conda/envs platform : linux-64 user-agent : conda/24.11.1 requests/2.32.3 CPython/3.12.8 Linux/6.5.0-1025-azure almalinux/9.5 glibc/2.34 solver/libmamba conda-libmamba-solver/24.11.1 libmambapy/2.0.5 UID:GID : 1001:1001 netrc file : None offline mode : FalseConda Config
Conda list
The text was updated successfully, but these errors were encountered: