⬆️ gem_bench v2.0.5

[pboling]

Fixes the require_relative > require issue discussed in #349 for the gem_bench gem itself. Does not merit a mention in the changelog, IMO.

Before merging:

• Copy the table printed at the end of the latest benchmark results into the README.md and update this PR
• If this change merits an update to CHANGELOG.md, add an entry following Keep a Changelog guidelines with semantic versioning

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (ef52a1a) to head (0a4d95f).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #351   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            2         2           
  Lines          190       190           
  Branches        90        90           
=========================================
  Hits           190       190           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pboling]

It failed to run the benchmarks because I forgot to enter my MFA code to actually push the new gem version to RubyGems before I made this change. 😬 🤦 It will pass if run again, no other changes needed. The gem is released now.

[JacobEvelyn]

Thanks! I'll wait to re-run this until after #350 is merged.

[JacobEvelyn]

@pboling based on #349 (comment) I expected that with this code, if I add puts MemoWise::VERSION directly after the doff_and_don call I'd see an error, but I do not. What am I misunderstanding?

[pboling]

@JacobEvelyn as it happens, I have another gem, require_bench, which is for mencharking and debugging require and load statements... I'll see if I can figure out how Memowise::VERSION is available at that point.

[pboling]

@JacobEvelyn OK, I figured out exactly what is happening.

I'd like to demostrate the solution with yet another gem of mine, version_gem, which I built to avoid this exact problem, but it will take me a bit to get around to a PR for that.

To briefly summarize - gem_bench uses bundler to figure out what a gem's path is so it can copy it. In order to do this bundler has to load the available "gemspecs". Most gems use a terrible practice (IMO) of requiring their version.rb file into their gemspec, and this makes it impossible for that namespace to not be loaded if bundler has accessed the gem. When require paths don't match up this also results in the nefariously common:

warning: already initialized constant MemoWise::VERSION
warning: previous definition of VERSION was here

I use this version_gem parttern in every gem I've published in the last few years to get around that ^ annoying issue. See: https://gitlab.com/oauth-xx/version_gem/#usage for how to set it up.

I've just reviewed the README over there, and found that I left out the secret sauce that actually solves this issue.

So here it is from gembench's gemspec:
https://github.com/pboling/gem_bench/blob/main/gem_bench.gemspec#L1-L9

# Get the GEMFILE_VERSION without *require* "my_gem/version", for code coverage accuracy
# See: https://github.com/simplecov-ruby/simplecov/issues/557#issuecomment-825171399
load "lib/gem_bench/version.rb"
gem_version = GemBench::Version::VERSION
GemBench::Version.send(:remove_const, :VERSION)

Gem::Specification.new do |spec|
  spec.name = "gem_bench"
  spec.version = gem_version
  # ...
end

You'll note that it only removes the VERSION constant after grabbing a local variable pointer to the value it holds. This means that in the way I've been using it, it does still leave behind namespace pollution. That could be avoidable, perhaps by removing the namespace as well, but perhaps namespace pollution is actually good? This is the first I've thought of it.

In any case, using that pattern you could clean up all of the constants that get defined there, and have an entirely clean gemspec. Also, to follow the pattern, you actually don't need version_gem at all. It includes a few other nice-to-haves that I just happen to want everywhere.

Without fixing this, the MemoWise::VERSION constant will always be loaded by each different version of memo_wise you test, and they will initially all conflict with each other, because initially, in the gemspecs loaded by bundler, they all occupy the same space.

The version_gem library was extracted from the very popular oauth2 library, which I also maintain, a few years ago.

The original reason the version_gem pattern was created was for code coverage accuracy. Because of requiring version.rb in the gemspec it made code coverage tools see that the file had already been loaded, and they were unable to track coverage of the file, which prevented my repos from getting to 100% line coverage.

[JacobEvelyn]

Thanks for all the details, @pboling, and sorry it's taken me so long to get back to this.

After pushing a branch to GitHub that makes your proposed gemspec change, and then using that branch in benchmarks locally, I can finally confirm that MemoWise::VERSION is not defined directly after the doff_and_don call!

The only thing I'm wondering about is why we need to use load in the gemspec instead of just require_relative as before. It seems to work for me either way. Do you know? (I'll also ask in the other GitHub thread where you found this trick.)

EDIT: Reading simplecov-ruby/simplecov#557 more closely, it seems like it might be related to SimpleCov's internals, which don't apply to us here. I'm going to try making this change but continuing to use require_relative instead of load, but please let me know if there's a reason not to do that.

[pboling]

So there were two things the load pattern addressed.

1. Constant already defined warning.

Essentially, require / require_relative will track things they load, so a subsequent load of the same resource will not be double loaded, except that it gets confused by dissimilar paths, so that many things actually do get double loaded. The way I've dealt with this is to use load, so it is never tracked, and a later require / require_relative will act normally. But because I've loaded the file, I then have to remove the constant manually to avoid the warning that the constant was already defined.

If it works fine with require_relative, and doesn't trigger the warning about the redefinition of the constant when the library loads the version.rb file again due to a different path, that's great. I haven't followed how that may have improved in Ruby. Its possible my load pattern may not be needed anymore.

Update: I see you did need to remove the constant when using require_relative... My fear with require_relative, is that this could introduce a bug where another require will not have the effect of the laoding the file because it thinks the file has already been loaded, and it doesn't know that you've removed the constant... thus the VERSION constant may never get loaded.

Update 2: require_relative tracks files loaded the way require does.

Update 3: Only load + remove constant works in all scenarios, see: #370 (comment)

2. Simplecov tracking code coverage.

Not relevant in this use case.

[pboling]

I'll rebase this on top of your latest changes once you've merged #370 ⭐