Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#3484] Add missing argument to Hash default proc #3485

Merged
merged 2 commits into from
Jan 6, 2025

Conversation

sandbergja
Copy link
Contributor

@sandbergja sandbergja commented Jan 3, 2025

The proc to fill in a default value for a Hash accepts two arguments: the hash itself and then the key. Prior to this commit, our proc was only using the Hash, but was treating it as if it was a key. This had the potential to cause some serious performance problems when solr returned many facets or many facet values, when a huge hash was treated as a string and interpolated into other strings via Hash#inspect.

Adding this missing argument ensures that we send the key/field_name, rather than the entire hash. For the provided test case on my laptop, this commit speed up null facet object generation up from 45 iterations/second to 18,700 iterations/second. It also doubles the performance of our advanced search form.

This commit also adds the rspec-benchmark gem as a dev dependency -- I'm definitely open to figuring out a different way to test this if the additional dependency is not desired.

Closes #3484

The proc to fill in a default value for a Hash accepts
two arguments: the hash itself and then the key.  Prior
to this commit, our proc was only using the Hash, but
was treating it as if it was a key.  This had the potential
to cause some serious performance problems when solr
returned many facets or many facet values, when a huge
hash was treated as a string and interpolated into other
strings via Hash#inspect.

Adding this missing argument ensures that we send the
key/field_name, rather than the entire hash.  For the
provided test case on my laptop, this commit speed up
null facet object generation up from 45 iterations/second
to 18,700 iterations/second.  It also doubles the
performance of our advanced search form.

This commit also adds the rspec-benchmark gem as a
dev dependency -- I'm definitely open to figuring out
a different way to test this if the additional dependency
is not desired.

Closes #3484

Co-authored-by: Christina Chortaria <[email protected]>
Co-authored-by: Kevin Reiss <[email protected]>
Co-authored-by: Max Kadel <[email protected]>
@sandbergja sandbergja marked this pull request as ready for review January 4, 2025 00:01
sandbergja added a commit to pulibrary/orangelight that referenced this pull request Jan 4, 2025
…ch: true

Before this commit, the AdvancedSearchFormComponent
processed any field in the CatalogController that did not
have include_in_advanced_search: false.

In theory, this should not be particularly expensive,
but a subtle bug in Blacklight causes some performance
issues in cases where we try to process fields that aren't
included in the Solr response and when the solr response
contains many facet values -- basically the exact situation
of our advanced search form!  See
projectblacklight/blacklight#3484
and projectblacklight/blacklight#3485
for further details.
Copy link
Contributor

@maxkadel maxkadel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sandbergja!

I have one comment, but it does not need to block this from going in.

Having seen loading single tests get slow in other applications, I'm starting to think more conservatively about loading dependencies for the whole suite.

It's really cool to learn about rspec-benchmark! I'm also excited about the advanced search form loading faster!

spec/spec_helper.rb Outdated Show resolved Hide resolved
Copy link
Contributor

@maxkadel maxkadel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sandbergja!

@maxkadel maxkadel merged commit bdc3c86 into main Jan 6, 2025
13 checks passed
@maxkadel maxkadel deleted the improve-performance-of-null-objects branch January 6, 2025 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Accessing Null Facet Fields through @response.aggregations can be very slow
2 participants