Unable to import WASM files in certain frameworks

[danilobassi8]

Hi, the hardcoded URL in index.mjs makes it really hard to load the wasm file in certain framework. This can be changed with a few lines of code.
As an example, I'm trying to implement a matrix client with e2e Encryption in Angular.

Angular can be used with Webpack or Esbuild, but they doesn't give you the ability to touch the webpack config, so we can't follow the webpack guide (if you're using it to bundle your Angular app).
The way to use Angular + Web Assembly is to expose .wasm as they are on the final build. You simply tell on the angular.json file something like this:

{
  "project": {
    "architect": {
      "build": {
        "assets": [
          {
            "glob": 'matrix_sdk_crypto_wasm_bg.wasm',                         // what file(s) do we want to expose
            "input": 'node_modules/@matrix-org/matrix-sdk-crypto-wasm/pkg/',  // where are the files located
            "output": '/assets/',                                             // where to copy  on the dist/ folder                                                                             
          },
        ],
      },
    },
  },
  

That basically means, on the final build, put the matrix_sdk_crypto_wasm_bg.wasm file under the /assets/ directory.
And then you simply need to fetch doing something like this: fetch("/assets/matrix_sdk_crypto_wasm_bg.wasm") .. you can do the same if you want to use Angular's built-in HttpClient.

Solution

Instead of hardcoding the URL like this

const moduleUrl = new URL("./pkg/matrix_sdk_crypto_wasm_bg.wasm", import.meta.url); 

we could give the user the ability to send its own URL. This way, you could make this work with almost any framework (I think Vue and Svelte works the same way as Angular)

When testing my app, I manually edited the index.mjs to make it work without breaking backwards compatibility. I can replicate those changes if you want, But I don't know if I manually need to edit index.cjs and index.d.ts by hand too or if those files gets automatically created.

These are my changes if you wan't to replicate it and open a PR:

// old moduleUrl
const defaultURL = new URL("./pkg/matrix_sdk_crypto_wasm_bg.wasm", import.meta.url);


// initAsync now accepts a URL argument. Uses defaultURL if not provided
export async function initAsync(url = defaultURL) {
    if (!modPromise) modPromise = loadModuleAsync(url);
    await modPromise;
}

// use the URL on the loadModuleAsync
async function loadModuleAsync(url) {
    const { instance } = await WebAssembly.instantiateStreaming(fetch(url), {
        // @ts-expect-error: The bindings don't exactly match the 'ExportValue' type
        "./matrix_sdk_crypto_wasm_bg.js": bindings,
    });

    bindings.__wbg_set_wasm(instance.exports);
    // @ts-expect-error: Typescript doesn't know what the module exports are
    instance.exports.__wbindgen_start();
}

[richvdh]

I don't really understand why it doesn't Just Work with the default Webpack settings: Webpack understands new URL("<path>, import.meta.url) and will automatically (a) treat <path> as an asset that needs to be packaged; (b) replace the URL with the correct path. See also: https://webpack.js.org/guides/asset-modules/#url-assets. Could you explain more about why that doesn't work for you?

I'm not completely against an optional parameter to loadModuleAsync but I'd like to make sure it's actually necessary before we do so.

In other news, I hate hate hate the javascript ecosystem. How is this not a 100% solved problem yet? We seem to have spent more time on this project fighting Webpack and trying to so solve other packaging problems than actually writing code.

[danilobassi8]

@richvdh thanks for your quick response!

I'm not an expert on how builders works, but I know that Angular does some kind of magic when building to maintain consistency, either you're using Webpack, Esbuild or Vite.
I opened an issue time ago because I was having a similar issue with web workers, and an Angular dev told me that they were doing some extra processing. As an example, this code:

if(environment.name === 'development'){
    this.worker = new Worker(new URL('../utils/workers/myworker.worker', import.meta.url),
        {name: 'worker', type: 'module'}
     )
} else {
    this.worker = new Worker(new URL('../utils/workers/myworker.worker', 'https://mypage.com')
        {name: 'worker', type: 'module'}
    )
}

was being transpiled as:

if(environment.name === 'development'){
    this.worker = new Worker(new URL('worker-F4AB9Z', import.meta.url), // <--- this changed
        {name: 'worker', type: 'module'}
     )
} else {
    this.worker = new Worker(new URL('../utils/workers/myworker.worker', 'https://mypage.com') // <--- but this doesn't
        {name: 'worker', type: 'module'}
    )
}

I hate hate hate the javascript ecosystem. How is this not a 100% solved problem yet?

I can't agree more. In fact, nowadays there are more and more bundlers out there and they keep adding their own rules, making it really hard to maintain consistency for libraries that needs to import wasm or use web workers.
That's why I think adding a custom URL that defaults to the old one will be the way to avoid custom builders rules.
In my case, the library is trying to fetch using this path: file:///home/danilo/dev/project/node_modules/@matrix-org/matrix-sdk-crypto-wasm/pkg/matrix_sdk_crypto_wasm_bg.wasm