Coverage action

[aGuttman]

Adds new GitHub action that automatically runs all tests and generates a coverage report on new pull requests. Coverage percentage shown as a badge in the README, and a comment detailing coverage of new lines added is generated on the pull request. Comment feature requires that repo wiki be turned on and have at least one page.

Makes use of two marketplace actions.
Badge:
https://github.com/marketplace/actions/coverage-py-badge
Comment:
https://github.com/marketplace/actions/python-coverage-comment
(This comment action would be able to provide the badge as well, but there seems to be a problem with it: py-cov-action/python-coverage-comment-action#15)

Changes:

• .github/workflows/coverage-badge-and-comment.yml
  • Sets up testing environment and runs coverage.py to create a .coverage report on pull requests. Uses the report to produce a coverage badge and comment on the pull request.
• .coveragerc
  • Setting for coverage.py. Needs relative_files = True to work with actions. Omits coverage of test files and emission/net/api/bottle.py, which artificially inflate and deflate the coverage percentage, respectfully.
• .gitignore
  • Added .coverage to gitignore. Not necessary, but figured it would be annoying/confusing to accidentally have some else's coverage report when developing and testing to see your own code coverage.
• README.md
  • Added badge for coverage percentage.
• coverage.svg
  • Coverage percentage badge from GitHub action. Generated by runner and committed back into repo for display in README.

[aGuttman]

I'm missing something with how external PRs work with the comment action. It should be able to do it (py-cov-action/python-coverage-comment-action-v2-example#2) and I thought I had set it up correctly (https://github.com/ewjoachim/python-coverage-comment-action-example/blob/master/.github/workflows/coverage.yml) but it isn't working.

[shankari]

Given the number of changes, I assume that I can squash merge this.

[shankari]

hm, by external PR, I assume you mean a PR from an external fork

@aGuttman Couple of thoughts there:

• the example has VERBOSE: true, which your code does not
• the example already has the github action and is evaluating a new PR, while you are adding the action in this PR
  • in the past, I have noticed issues with running actions in the PRs which are adding them

So add VERBOSE: true and let's see if it fixes it.
Otherwise, I will just merge and you can get back to debugging it once you have focused on the other tasks.

[shankari]

Otherwise looks good.
Let's get this merged!

[aGuttman]

hm, by external PR, I assume you mean a PR from an external fork

@aGuttman Couple of thoughts there:

• the example has VERBOSE: true, which your code does not

• the example already has the github action and is evaluating a new PR, while you are adding the action in this PR

  • in the past, I have noticed issues with running actions in the PRs which are adding them

So add VERBOSE: true and let's see if it fixes it. Otherwise, I will just merge and you can get back to debugging it once you have focused on the other tasks.

Setting verbose didn't produce a comment, but the message left in the run when attempting to post the comment is longer. Maybe helpful to read through it.

[shankari]

Yeah the comment says:

INFO:coverage_comment:Cannot post comment. This is probably because this is an external PR, so it's expected. Ensure you have an additional workflow_run step configured as explained in the documentation (or alternatively, give up on PR comments for external PRs).

[aGuttman]

The workflow run step is, I believe, what happens in the coverage.yml file. The documentation doesn't actually explain it.