Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to benchmarking system #248

Merged
merged 15 commits into from
Dec 11, 2024
Merged

Conversation

matt-graham
Copy link
Collaborator

@matt-graham matt-graham commented Nov 29, 2024

A collection of improvements to benchmarking system made during the DiRAC NVIDIA hackathon

  • Due to behaviour noted in NumPy generate_precomputes implementation has cubic in L memory overhead #246 switches to always using JAX implementation for generating precomputes for spherical transforms to avoid cubic in L overhead.
  • Adds benchmark modules for separate precompute versions of transforms.
  • Some improvements to help text of benchmarking scripts
  • Adds option to run benchmarked functions once initially without recording time to remove effect of initial JIT compilation from reported times
  • Fix to issue causing only last parameter set results being recorded in JSON output
  • JSON output now records additional metadata about host system such as installed package versions and information about GPUs and CPUs available, to make these more useful artefacts to store for longer term comparison.

@jasonmcewen jasonmcewen self-requested a review December 11, 2024 13:24
@matt-graham matt-graham merged commit 8f6e4d5 into main Dec 11, 2024
5 checks passed
@matt-graham matt-graham added the enhancement New feature or request label Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants