Add environment variable for expected build output path

[nmattia]

I would like dfx to specify where it expects a canister's Wasm module to be after build as an environment variable; for instance: $DFX_WASM_OUT.

Problem description

Currently, the wasm module path has to be specified in dfx.json by the user:

{
  "canisters": {
    "my_can": {
      "type": "custom",
      "candid": "src/my_can.did",
      "wasm": "output.wasm",   // <-- set here
      "build": "scripts/build" // <-- referenced here
    }
  }
}

This works but is neither ergonomic nor robust:

• The developer must keep their build script in sync with the dfx.json, duplicating configuration
• The wasm output needs to be copied out of e.g. the motoko or rust build directory (to then often be optimized) and will litter the workspace
• The actual value is not important, but only used to tell dfx where to find the binary file; removing it from dfx.json would be nice

Potential solution

An environment variable like $DFX_WASM_OUT can be set in the build phase so that the developer does not need to keep dfx.json and their build file in sync:

# scripts/build
cargo build --release
ic-cdk-optimizer \
  ./target/wasm32-unknown-unknown/release/my_can.wasm \
  -o "$DFX_WASM_OUT"

This also means the user's workspace won't be littered with an unnecessary output.wasm file.

I believe we can go a bit further, and inverse control here: the actual value of the wasm:field in dfx.json is actually meaningless in most cases, so why not let dfx specify it?

{
  "canisters": {
    "my_can": {
      "type": "custom",
      "candid": "src/my_can.did",
      // we don't care about the actual value of DFX_WASM_OUT
      "build": "cargo build --release && ic-cdk-optimizer ... -o $DFX_WASM_OUT" 
    }
  }
}

Note that the actual value of DFX_WASM_OUT does not even need to stay the same between any two builds. This is similar to the Nix's $out environment variable. Note also that (include a build script is expected to produce many wasm modules during a single run) the canister name doesn't need to (and probably shouldn't) appear in the name of the environment variable, i.e. $DFX_WASM_OUT_{canister.name} is probably a bad idea.

[bitdivine]

canister name doesn't need to (and probably shouldn't) appear in the name of the environment variable
Presumably because other canisters probably can't really use that information, unlike say the did file for another canister. And we don't use wasm as a lib. Ok, yes that sounds fair for commands that work on one canister only.

[bitdivine]

Note: Wasm builds can be different; see for example the production and test builds for almost any of the NNS canisters. The test versions have additional methods to make debugging easier. The II canister has a test build with simplified captcha. Having a single prescribed wasm path is a bit problematic in this context; de facto the output wasm is copied to another location for test builds.

When there are several builds possible, there is the question of how that configuration is managed. In the NNS-dapp we have multiple deployments and for each network specify which build flags we want to enable for that deployment. In the context of dfx, network==deployment (which is problematic when multiple deployments to the same physical network need to interact with each other but let's not delve into that now).

If there is an ephemeral build location, it would be good to be able to ask "where did you put the wasm for canister X?" as it is sometimes necessary to look at the wasm, take a sha256, look at the compiled binary etc. Not often but often enough that we should cover that use case. We do know that canister names are unique and SHOULD contain only a-z0-9_ (hyphen is deprecated) so in particular no . so it is easy enough to make a scheme whereby wasms are stored in e.g. ${project_dir}/wasms/${canister_name}.${feature_flags.sorted().join(".")}.wasm. Doesn't have to be this but this is just an example to demonstrate that it is possible to come up with a clean naming system that's naturalk enough that people who need to poke the binary can do so without needing special tooling.