Skip to content

Commit

Permalink
tests pass in new project; README updated
Browse files Browse the repository at this point in the history
  • Loading branch information
phitoduck committed May 15, 2024
1 parent c0605cb commit 781b263
Show file tree
Hide file tree
Showing 5 changed files with 36 additions and 28 deletions.
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,4 @@
"color": "green"
}
],
}
}
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This is a project scaffold to help you quickly get up and running with a Python
modern, industry standard tools and practices.

The template comes with an opinionated VS Code settings configuration file to help ensure
the best integration with the tools here, e.g.
the best integration with the tools here, e.g.

- autocompletion
- syntax highlighting
Expand All @@ -31,7 +31,7 @@ cookiecutter https://github.com/mlops-club/cloud-course-python-package-template.
### Notable comments about the file structure

- `src/<package-name>/` - contains the package code. Using an `src/` folder is a best practice.
- `tests/` has minimal boilerplate that
- `tests/` has minimal boilerplate that
- Makes fixtures work using `conftest.py`
- Makes imports such as `from tests.my_file import thing` (via some *light* PYTHONPATH hacking, adding `tests/..` to the PYTHONPATH)
- Contains a `unit_test/` and `functional_test/` directory whose structure is meant to mirror that of `src/<package>/`
Expand All @@ -43,22 +43,22 @@ cookiecutter https://github.com/mlops-club/cloud-course-python-package-template.

This project template makes use of

- [`setuptools`](https://setuptools.pypa.io/en/latest/userguide/index.html) for packaging; a `pip` + `venv` + `setuptools` workflow is encouraged as it is considered the
- [`setuptools`](https://setuptools.pypa.io/en/latest/userguide/index.html) for packaging; a `pip` + `venv` + `setuptools` workflow is encouraged as it is considered the
most officially supported, "vanilla" way to manage Python packages.
- [`pylint`](https://pylint.readthedocs.io/en/stable/) and [`flake8`](https://flake8.pycqa.org/en/latest/) for linting. [`ruff`](https://docs.astral.sh/ruff/) is up and coming, but does not yet support all the plugins and rulesets
that these two do. However, we love `ruff` and are excitedly watching its progress.
- [`flake8`](https://flake8.pycqa.org/en/latest/) supports plugins, and we are using the following
- [`flake8-docstrings`](https://pypi.org/project/flake8-docstrings/) - ensure that functions have docstrings
- [`flake8-pyproject`](https://pypi.org/project/Flake8-pyproject/) - necessary for `flake8` to be configured using a `pyproject.toml` file
- [`radon`](https://radon.readthedocs.io/en/latest/intro.html) - a standalone tool for measuring [cyclomatic complexity](https://radon.readthedocs.io/en/latest/intro.html#cyclomatic-complexity).
- [`radon`](https://radon.readthedocs.io/en/latest/intro.html) - a standalone tool for measuring [cyclomatic complexity](https://radon.readthedocs.io/en/latest/intro.html#cyclomatic-complexity).

Radon out-of-the-box is a `flake8` plugin which enables VS Code and PyCharm to display red squiggly lines on functions whose complexity score is too high.
- [`black`](https://black.readthedocs.io/en/stable/) for formatting. Note that the line length is set to `119` for `black` and other tools that consider line length.
- [`pytest`](https://docs.pytest.org/en/8.2.x/) for unit testing since it is the industry standard testing framework
- [`pytest-cov`](https://pytest-cov.readthedocs.io/en/latest/) for calculating coverage and displaying an HTML coverage report in the browser
- [`pre-commit` framework](https://pre-commit.com/) - a convenient, industry-standard way to install and run `flake8`, `pylint`, and the other tools all at once--and in isolated virtual environments. See `.pre-commit-config.yaml` for the configuration.
- `Makefile` - for running tasks, e.g. `make lint`, `make test`, `make install`.
- `Makefile` - for running tasks, e.g. `make lint`, `make test`, `make install`.

Make is unfortuanately a tool that
comes pre-installed on most OSes, but it comes with its own scripting language which is limited and difficult to use compared to `bash`.
It supports tab-based autocompletion when running commands and can be used to chain commands together, e.g. `make lint test build`.
Expand All @@ -67,10 +67,10 @@ This project template makes use of
still providing the convenience of tab-based autocompletion and chaining commands together.

We are not in love with `Makefile` + `run.sh`, but it has advantages over alternative workflows such as

- `bash` scripts are flexible. Most developers can quickly author simple bash scripts, especially with ChatGPT.
- `Justfile` is *awesome*, but can be difficult to install in CI and most devs do not already have it installed.
- `PyInvoke` is a thing, but requires an initial installation of `invoke` with Python, resulting in extra steps,
- `PyInvoke` is a thing, but requires an initial installation of `invoke` with Python, resulting in extra steps,
e.g. installing in the system Python, or with `pipx`. It also has a slight learning curve over the first
adding that much more friction to new developers joining projects.

Expand All @@ -84,7 +84,7 @@ This project template makes use of
## I need help learning about all of these tools

For an in-depth, step-by-step guide on the history of Python packaging and explanations of all
the tools used in this scaffold, you can take the Udemy course
the tools used in this scaffold, you can take the Udemy course
[Taking Python to Production: A Professional Onboarding Guide](https://www.udemy.com/course/setting-up-the-linux-terminal-for-software-development/?referralCode=EE86AF0C55EE90E3C8AE) where
we develop this template from scratch as well as many other foundational SWE skills, e.g. git, GitHub, CI/CD GitHub Actions, zsh, and testing.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Build, Test, and Publish
name: Build, Test, and Deploy

on:
pull_request:
Expand All @@ -8,8 +8,8 @@ on:
- main
workflow_dispatch:
inputs:
publish_to_main:
description: Publish to PyPI if tests successful
deploy:
description: Deploy if tests successful
required: true
type: boolean
default: false
Expand Down Expand Up @@ -93,6 +93,7 @@ jobs:
- lint-format-and-static-code-checks
- check-version-txt
runs-on: ubuntu-latest
# if - this is a merge to main or push directly to the main branch
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v3
Expand All @@ -105,25 +106,17 @@ jobs:
with:
name: wheel-and-sdist
path: ./dist/
- name: Install twine
run: |
pip install twine
# - name: Publish to TestPyPI
# run: |
# /bin/bash -x run.sh publish:test
# env:
# TEST_PYPI_TOKEN: {% raw %}${{ secrets.TEST_PYPI_TOKEN }}{% endraw %}
# - name: Publish to Prod PyPI
# - name: Deploy
# if: {% raw %}${{ github.event.inputs.deploy == 'true' }}{% endraw %}
# run: |
# /bin/bash -x run.sh publish:prod
# env:
# PROD_PYPI_TOKEN: {% raw %}${{ secrets.PROD_PYPI_TOKEN }}{% endraw %}
# # ...
- name: Push tags
run: |
git tag $(cat version.txt)
git push origin --tags
# https://docs.github.com/en/actions/learn-github-actions/contexts#example-printing-context-information-to-the-log
# Print variables that are available to the workflow. This is useful for debugging and troubleshooting.
# docs: https://docs.github.com/en/actions/learn-github-actions/contexts#example-printing-context-information-to-the-log
dump-contexts-to-log:
runs-on: ubuntu-latest
steps:
Expand Down
12 changes: 11 additions & 1 deletion {{cookiecutter.repo_name}}/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,17 @@

set -e

#####################
# --- Constants --- #
#####################

THIS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
MINIMUM_TEST_COVERAGE_PERCENT=0


##########################
# --- Task Functions --- #
##########################

# install core and development Python dependencies into the currently activated venv
function install {
Expand Down Expand Up @@ -43,7 +53,7 @@ function run-tests {
--cov-report term \
--cov-report xml \
--junit-xml "$THIS_DIR/test-reports/report.xml" \
--cov-fail-under 60 || ((PYTEST_EXIT_STATUS+=$?))
--cov-fail-under "$MINIMUM_TEST_COVERAGE_PERCENT" || ((PYTEST_EXIT_STATUS+=$?))
mv coverage.xml "$THIS_DIR/test-reports/" || true
mv htmlcov "$THIS_DIR/test-reports/" || true
mv .coverage "$THIS_DIR/test-reports/" || true
Expand Down
5 changes: 5 additions & 0 deletions {{cookiecutter.repo_name}}/tests/unit_tests/test__example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
"""Example test file in place so that a freshly created project can immediately pass tests."""

def test__example():
"""An example test to demonstrate how to write unit tests."""
assert True

0 comments on commit 781b263

Please sign in to comment.