Improve test coverage using results from mutation testing #70
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A favorite mutation of cargo-mutants is to replace static &str's with "xyzzy". While I think what I want from the Rule metadata is clear to me, it might not be clear to a future contributor (or even my future self).
There were several mutations attempted that all relate to this bug:
* changing += to *=
* changing == to !=
* etc
all in the region that calculates the number of new commits.
It turns out that counting the number of new commits only works the
first time the branch is pulled, and that subsequent updates don't
actually update the local branch. This means that the desired workflow
of
herostratus add ...
herostratus fetch-all
herostratus check-all
... time goes by
herostratus fetch-all
herostratus check-all
doesn't actually work as-expected.
This might be a tricky bug to fix, and I might end up choosing to not
fix it until I finish migrating to gix. The value is in having the
tests, since fetching has turned up being more complex than imagined.
TODO: There should be an integration test covering the desired workflow.
This improves a test coverage gap highlighted by mutation testing. write_config() creates the data-dir if it doesn't exist. So give it a scenario where the data-dir doesn't exist.
The mutation switched one of the == to != in the filtering, which caused _every_ rule to be excluded. So change the test so that not every rule is excluded.
The mutation this resolves is removing the remove_dir_contents implementation with a do-nothing impl.
Mutation: Replace < with ==, which effectively makes the "shorest" item be the last. So re-order the commit subject lines so that the shortest is no longer the last.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Mutation testing revealed a few gaps that I thought were worth filling in, and in doing so, it importantly highlighted a bug in herostratus::git::pull_branch. This alone makes it worth it!
Closes #63