Add a regular expression benchmark #224
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This regular expression benchmark measures the time taken to match e-mails, URIs, and IP addresses in an input text. It is written in Rust and significantly adapted from the [regex-benchmark] project. The input text is a bit heavy (6.6MB) and the `benchmark.wasm` is as well (2.8MB); it may be worthwhile at some point to run some of these benchmark files through `wasm-opt`, e.g., to reduce the size. It appears that one iteration of this benchmark takes around half a second for all three phases and the compilation phase is about an order of magnitude larger than the execution phase. I looked into creating a smaller version of the input text, but this would not significantly change the overall benchmark time (since it is concentrated in compilation) and, importantly, I could not figure out a way to have different `stdout` and `stderr` expectations for different workloads. With a smaller workload, the output _will_ be different and currently there is no way to express that. [regex-benchmark]: https://github.com/mariomka/regex-benchmark
|
If people are interested, this workload might be one to add to the |
|
cc: @fitzgen, @alexcrichton, @cfallin, @jameysharp, @elliottt, @jlb6740. Can someone take a look and +1? |
|
Awesome, I've wanted a regex benchmark to study in Sightglass. Thanks! I've opened #226 with a bunch of experiments which somebody could try that should affect compile time, run time, or both, and in some cases may hit very different code generation paths in Cranelift. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This regular expression benchmark measures the time taken to match e-mails, URIs, and IP addresses in an input text. It is written in Rust and significantly adapted from the regex-benchmark project. The input text is a bit heavy (6.6MB) and the
benchmark.wasmis as well (2.8MB); it may be worthwhile at some point to run some of these benchmark files throughwasm-opt, e.g., to reduce the size.It appears that one iteration of this benchmark takes around half a second for all three phases and the compilation phase is about an order of magnitude larger than the execution phase. I looked into creating a smaller version of the input text, but this would not significantly change the overall benchmark time (since it is concentrated in compilation) and, importantly, I could not figure out a way to have different
stdoutandstderrexpectations for different workloads. With a smaller workload, the output will be different and currently there is no way to express that.Closes #216.