How to change the version of a package dependency?

[nartamonov]

I just started using devbox and encountered the following problem. Some packages depend on other packages with specific versions. Sometimes you need to change the version of a dependency to another one. But I haven’t found in the documentation how to do that.

For example, my project is developed in the Scala language, so I have installed the SBT build system. The SBT package is tied to JDK 21, but I want to use JDK 23 because SBT knows how to work with it. My project also relies on the capabilities of JDK 23. So I have the following configuration:

{
  "$schema": "https://raw.githubusercontent.com/jetify-com/devbox/0.13.7/.schema/devbox.schema.json",
  "packages": [
    "[email protected]",
    "temurin-bin-23@latest"
  ]
}

If you run the devbox shell and look at the java --version and JAVA_HOME, it matches what you expect:

nartamonov@dev-machine:~/dev/sigma-graph/sigma-graph$ devbox shell
Starting a devbox shell...
(devbox) nartamonov@dev-machine:~/dev/sigma-graph/sigma-graph$ java --version
openjdk 23 2024-09-17
OpenJDK Runtime Environment Temurin-23+37 (build 23+37)
OpenJDK 64-Bit Server VM Temurin-23+37 (build 23+37, mixed mode, sharing)
(devbox) nartamonov@dev-machine:~/dev/sigma-graph/sigma-graph$ which java
/home/nartamonov/dev/sigma-graph/sigma-graph/.devbox/nix/profile/default/bin/java
(devbox) nartamonov@dev-machine:~/dev/sigma-graph/sigma-graph$ echo $JAVA_HOME
/nix/store/02s8kaspai72g46ylp306pshzzr6s6iq-temurin-bin-23.0.0

However, SBT still runs through JDK 21:

(devbox) nartamonov@dev-machine:~/dev/sigma-graph/sigma-graph$ sbt -v
[sbt_options] declare -a sbt_options=()
[process_args] java_version = '21'
[copyRt] java9_rt = '/home/nartamonov/.sbt/1.0/java9-rt-ext-n_a_21_0_5/rt.jar'
# Executing command line:
/nix/store/f042x32jfm94d3cgaga8d6xl8vy6sg46-openjdk-21.0.5+11/lib/openjdk/bin/java
-Xms2g
-Xmx3g
-Xss2m
-XX:+UseG1GC
-Dscala.color=true
-Dsbt.supershell=true
-Djdk.console=java.base
-Dsbt.script=/home/nartamonov/dev/sigma-graph/sigma-graph/.devbox/nix/profile/default/bin/sbt
-Dscala.ext.dirs=/home/nartamonov/.sbt/1.0/java9-rt-ext-n_a_21_0_5
-jar

Is there a way to solve this problem, or do I need to learn nix and build my own SBT package with the dependencies I want? In this case the devbox idea is significantly devalued.

[yemaney]

Devbox Version 0.13.7

Reason For Issue

Hi. I've looked into this, and I think its more of a sbt question rather than a devbox question. Have never used sbt personally, but I think I found a working resolution, working devbox.json below

I've noticed the config file at .devbox/nix/profile/default/share/sbt/conf/sbtopts has a value that sets a value of -java=home to
a openjdk-21 folder path. I believe this is why sbt is using java21 instead of java23.

$ tail -n 1 .devbox/nix/profile/default/share/sbt/conf/sbtopts
-java-home /nix/store/f042x32jfm94d3cgaga8d6xl8vy6sg46-openjdk-21.0.5+11/lib/openjdk

If you delete this file, sbt will correctly use java23, however that's a rough workaround.

A better option is to override that config value with one of sbt's comand line options.

$ sbt --help | grep java-home
  --java-home <path>         alternate JAVA_HOME
$ echo $JAVA_HOME
/nix/store/16ygp40ximgpl6iwg0vjral2kp1bqn52-temurin-bin-23.0.0
$ sbt --java-home $JAVA_HOME -v
[sbt_options] declare -a sbt_options=()
[process_args] java_version = '23'
[copyRt] java9_rt = '/home/ubuntu/.sbt/1.0/java9-rt-ext-eclipse_adoptium_23/rt.jar'
# Executing command line:
/nix/store/16ygp40ximgpl6iwg0vjral2kp1bqn52-temurin-bin-23.0.0/bin/java
-Dfile.encoding=UTF-8
-Xms1024m
-Xmx1024m
-Xss4M
-XX:ReservedCodeCacheSize=128m
-Dsbt.script=/home/ubuntu/sbtj/.devbox/nix/profile/default/bin/sbt
-Dscala.ext.dirs=/home/ubuntu/.sbt/1.0/java9-rt-ext-eclipse_adoptium_23
-jar
/nix/store/y05q6q62na3153cvwjsywxakkzwzzsdp-sbt-1.10.6/share/sbt/bin/sbt-launch.jar

[info] welcome to sbt 1.10.6 (Eclipse Adoptium Java 23)

devbox.json file

Better yet, you can update the devbox.json init hook to set up an alias for sbt so that you don't have to pass the option everytime.

{
  "$schema": "https://raw.githubusercontent.com/jetify-com/devbox/0.13.7/.schema/devbox.schema.json",
  "packages": [
    "[email protected]",
    "temurin-bin-23@latest"
  ],
  "shell": {
    "init_hook": [
      "alias sbt='sbt --java-home $JAVA_HOME'"
    ],
    "scripts": {
      "test": [
        "echo \"Error: no test specified\" && exit 1"
      ]
    }
  }
}

[nartamonov]

Thank you very much, your way around the problem works. But I think it’s not just about SBT. I found in the nixpkgs repository a discussion of exactly the same problem. And here is what the author of sbt package says:

I think this is one of the quirks that's classified as "it's a feature, not a bug" :) Every Nix package is required to declare all its dependencies. In the case of sbt, it needs to declare the jre / jdk in order to make a derivation, it doesn't care about global packages or global environment variables.

This can be easily overridden, though. Just need to override a pkgs.jre = yourJdk, and sbt will pick it up.

As far as I understand the ideology of Nix, I think he is right. One of the key features of Nix is to combine different packages with each other to get exactly what you need at the output. Therefore, almost all packages are parameterized and accept at the input the definition of other packages on which they depend. And then they tie themselves to these dependencies in different ways. For example, I looked at the definitions of other popular packages that depend on the JDK:

1. Maven. Binds itself to the JDK by specifying the path to the JDK directly in the launch script.
2. Gradle. Similarly, it creates a wrapper script in which the JDK version is fixed.
3. Clojure. Also fixes the JDK version in the wrapper script.
4. Leiningen. Similar.

I'm sure, the packages dependent on NodeJS and other SDKs are designed in much the same way. This is contrary to the current approach of devbox, which tries to work in the same way as traditional package managers that give you a set of packages with fixed dependency versions. Use or not, other options are not usually provided. That’s why developers usually don’t use apt-get, rpm and other utilities to configure their development environment, but instead have invented numerous version managers like sdkman, nvm, etc. Which allow you to get exactly the right combination of versions of tools, that they need!

This is missing from the devbox project. If it wants to become a viable alternative to the above managers, it should come up with a way of composing different versions of packages with each other, but not forcing the user to learn the Nix language. It would be killer feature!

Now, it turns out that the end user, having installed a package dependent on some SDK, should study the definition of this package on Nix and find a way to bypass its binding to the specific version of the SDK. Then what is the point of using devbox if you still have to learn Nix.

[yemaney]

I stand corrected. You make a very good point. Wasn't aware of Nix defining dependencies for each package, but it makes sense in retrospect. It would make sense if devbox supported this as well.

[mskonovalov]

come up with a way of composing different versions of packages with each other, but not forcing the user to learn the Nix language

imho it is probably not the goal of the project.
To me Devbox seems like a simplified abstraction on top of Nix. Yes, you can do basic things without diving into Nix internals but in that case you are limited to what is provided by Nix.
That means if you need a new package that does not exist in Nix or package, that is currently broken or outdated, you need to file a ticket to Nix and wait when it will be fixed for you, or contribute by yourself to speed up the process.
This is the Nix way, which is completely opposite to what tools like sdkman, asdf, etc offer. If you don't want to deal with Nix, there are some quite viable alternatives based on a more traditional concept, e.g. https://github.com/jdx/mise

[balogio]

+1 on being able to force some type of overrides on transient dependencies, or for the defined environment to take precedents over the upstream. Ran into this issue as well, which is not really the expected functionality when I'm also defining the required JDK in the Devbox config. I think this should, at the very least, be called out in the Java env example in the documentation. It'll save some debugging time for the Java crowd hopefully.

If you ask me, this kinda defeats the purpose of Devbox (albeit very easy workaround here for SBT). The point of it being abstracted is for it to be user-friendly with easily defined and reproducible environments, as well as altering the underlying functionality to make it more usable than Nix. Just because "thats how nix works", doesn't mean that's how it should work on Devbox, else what's the point in the abstraction? Would love to see this be taken up as a feature request, something similar to npm overrides in the json config would help bring Devbox to another level: https://docs.npmjs.com/cli/v9/configuring-npm/package-json#overrides .

[mskonovalov]

I don't disagree that it would be nice.
I was hitting my head against the wall for quite a while trying to install npm package as a cli tool that will be available globally inside Devbox (which sounds like a pretty basic thing to do). And didn't find any option to do this but writing your own nix flake that will download the binary from the http URL.
I'm just saying those are the limitations of the approach that Devbox selected.
And I'd not expect significant changes here, as I can imagine how much more complexity it will bring to the tool (imagine how difficult it will be to troubleshoot all possible issues when you have several layers where things can go wrong) but also it may not be easy to intrude into the nix layer and build something on top.
What I think would be a nice way to overcome this without actually changing a lot of things is a Cookbook of most common use cases that you need to frequently deal with. E.g.

1. how to install the binary tool from the http URL into Devbox
2. how to override jre version in Java-based tools
3. how to deal with dependency hell when you add python packages plus a tool like awscli that brings older version of the same packages
   And so on...