Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot artifacts have weird numbering scheme making them hard to depend on #134

Open
FelixGV opened this issue Jan 23, 2025 · 0 comments

Comments

@FelixGV
Copy link

FelixGV commented Jan 23, 2025

Hi,

The DuckDB snapshot artifacts have some ergonomic issues that make them difficult to leverage.

The snapshots can be depended on by adding the following Maven repo to the build: https://oss.sonatype.org/content/repositories/snapshots/

The artifacts for the current SNAPSHOT release (1.2.0-SNAPSHOT) are listed here: https://oss.sonatype.org/content/repositories/snapshots/org/duckdb/duckdb_jdbc/1.2.0-SNAPSHOT/

You can see that for a given date, there are three versions published, a few seconds apart. First the compiled code is published (e.g., *-124.jar), then the sources (e.g., *-125-sources.jar) and finally the JavaDoc (e.g., *-126-javadoc.jar).

While this may seem merely cosmetic, the issue is that when depending on a SNAPSHOT dependency, the build system automatically resolves what the latest version is in that directory, and then tries to get the compiled code for it. In the above example, if the build had run on Jan 18th, it would have determined that the latest version was 126, but then the build would fail because -126.jar is not available. At least, that is the behavior observed in Gradle. Perhaps other build systems are smarter, somehow, and look for the most recent version that has compiled code available, but in any case, it is broken for Gradle.

There is a workaround which is to fully specify the version ID, rather than just SNAPSHOT, e.g., 1.2.0-20250118.011454-124, and then the build system will find it, rather than automatically inferring what the latest is. This is actually good in one sense, because it provides a reproducible build (whereas auto-determining the latest means the dependencies can change from build to build), but it does cause a maintenance burden because the snapshot artifacts have a short retention time, and so the build will start failing not too long after it begun depending on a pinned snapshot dependency. And so the further workaround needed for this workaround to work is to keep revving up the pinned snapshot we depend on.

Obviously, the clean solution is to stop depending on snapshots altogether, and we do intend to do exactly that as soon as 1.2.0 is released (assuming that it contains the fix for #14, which is the one we need...).

But in any case, I did want to report this because it took me a while to figure out the above details, and it would be nice if they could be fixed. Even if it doesn't get fixed, at least documenting these findings may help others finding themselves in the same situation I did.

The cleanest fix (IMHO) would be for the publishing script to set the compiled code, sources and JD to the same version. If that is impractical for some reason, then a hacky but likely acceptable alternative to fix it might be to let the compiled code be published last out of the three artifacts, so that the highest version getting resolved has compiled code available. And finally, a third option might be to increase the retention time of past snapshots (though that would not really fix the issue, just make it less frequent...).

Thanks a lot for maintaining this project!

-F

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant