benchmarks: add initial benchmarking support.

[ipuustin]

Add a submodule "webassembly-benchmarks". The benchmark contains a set of wasm files from libsodium. Only the benchmarks which are fast to run (take under 5 seconds to complete on a desktop computer using WasmEdge) were selected for benchmarking.

The benchmark suite initializes the Wasm runtime, runs the benchmark, and cleans up the container. This is done for both WasmEdge and Wasmtime shim crates. Note that the results for the two runtimes shouldn't really be directly compared right now, because Wasmtime shim hasn't been converted to youki/libcontainer-rs yet and that changes the performance due to the extended namespacing. What this helps with is to see what is the performance impact of any given change, for example having the Youki integration done, using a seccomp filter, or setting cgroup options.

The benchmarking library used is Criterion. Run the benchmarks like this:

# cargo bench

If you have gnuplot binary installed, Criterion will generate an HTML report with graphs in target/criterion/report/index.html file. Running the suite takes bit over ten minutes on my desktop.

This PR adds two new direct dependencies: Criterion (Apache2.0/MIT) as dev-dependency and webassembly-benchmarks (ISC) as a submodule.

Note that this PR is just the first step in the benchmarking path -- it does only pretty low-level benchmarking using a limited set of tests. Later on we can add comparison to the normal containers (by compiling and running the libsodium benchmarks as regular processes with the native Youki executor), add or remove Wasm benchmark modules, and improve the test setup. We could for example try running hunderds of containers at once to see how "dense" containers on a node would perform.

This is related to #97 (and should fix the second point in the list).

[ipuustin]

The sources for the benchmark modules are here: https://github.com/jedisct1/libsodium/tree/master/test/default

[Mossaka]

LGTM! Some questions regarding the benchmark:

• What areas do this set of wasm modules test? CPU, Memory etc.
• Does this set of wasm modules require WASI at all?

[Mossaka]

10 minutes of timeout seems too long for me...

[ipuustin]

That's right. I reduced the timeout to one minute. That should be plenty too...

[ipuustin]

LGTM! Some questions regarding the benchmark:

• What areas do this set of wasm modules test? CPU, Memory etc.

Depends on the test. For example, the hash tests should be all about memory access. The encryption tests are probably more computation-intensive. What we are missing is an "empty" Wasm module that would just return. With that we could measure the setup/teardown time. The benefit of using the libsodium tests is that we are getting something that looks more like a real-life workload compared to just computing fibonacci sequence.

• Does this set of wasm modules require WASI at all?

No, these tests don't use WASI.

[Mossaka]

LGTM!

I can merge this in when all CI pipelines are passed.

[cpuguy83]

Some linting issues, also conflict in Cargo.lock.

error: redundant clone
   --> crates/containerd-shim-wasmtime/src/instance.rs:355:21
    |
355 |         let w = wasi.clone();
    |                     ^^^^^^^^ help: remove this
    |
note: cloned value is neither consumed nor mutated
   --> crates/containerd-shim-wasmtime/src/instance.rs:355:17
    |
355 |         let w = wasi.clone();
    |                 ^^^^^^^^^^^^
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
    = note: `-D clippy::redundant-clone` implied by `-D warnings`

error: could not compile `containerd-shim-wasmtime` (lib test) due to previous error
warning: build failed, waiting for other jobs to finish...