From 1d604f9860c027f4277e72a4e653b8406e738bc5 Mon Sep 17 00:00:00 2001 From: Andre Date: Tue, 3 Dec 2024 18:20:14 -0500 Subject: [PATCH] New docs-dev pages --- .DS_Store | Bin 10244 -> 10244 bytes docs-dev/.DS_Store | Bin 10244 -> 10244 bytes ...usage-of-black.rst => guider-of-black.rst} | 56 +++-- ...-branching.rst => guider-of-branching.rst} | 38 ++- ...of-coverage.rst => guider-of-coverage.rst} | 41 +++- docs-dev/guider-of-git.rst | 229 ++++++++++++++++++ ...age-of-poetry.rst => guider-of-poetry.rst} | 47 ++-- docs-dev/guider-of-sphinx.rst | 222 +++++++++++++++++ docs-dev/guider-of-tests.rst | 157 ++++++++++++ .../{usage-of-tox.rst => guider-of-tox.rst} | 64 +++-- docs-dev/index.rst | 22 +- docs-dev/usage-of-git.rst | 3 - docs-dev/usage-of-sphinx.rst | 217 ----------------- .../data-out/outcome_run_2.configuration | 2 +- .../data-out/outcomes.configuration | 2 +- 15 files changed, 787 insertions(+), 313 deletions(-) rename docs-dev/{usage-of-black.rst => guider-of-black.rst} (51%) rename docs-dev/{usage-of-branching.rst => guider-of-branching.rst} (62%) rename docs-dev/{usage-of-coverage.rst => guider-of-coverage.rst} (58%) create mode 100644 docs-dev/guider-of-git.rst rename docs-dev/{usage-of-poetry.rst => guider-of-poetry.rst} (80%) create mode 100644 docs-dev/guider-of-sphinx.rst create mode 100644 docs-dev/guider-of-tests.rst rename docs-dev/{usage-of-tox.rst => guider-of-tox.rst} (53%) delete mode 100644 docs-dev/usage-of-git.rst delete mode 100644 docs-dev/usage-of-sphinx.rst diff --git a/.DS_Store b/.DS_Store index 9942a668d3e0990310b2b0b07e36952bfebc8a7c..e76f568d555f5f5acb01711e5bb5ab83aaabded0 100644 GIT binary patch delta 22 dcmZn(XbIR5D$HqYVq&19U}0vtIZk+)2mnps24(;N delta 22 dcmZn(XbIR5D$HqQVQ8tNU}R~uIZk+)2mnq124w&M diff --git a/docs-dev/.DS_Store b/docs-dev/.DS_Store index 4fd01bcfc56e8d91eb5c2a1bec10f92fe8f3a88d..6792ad7c1b02b4ac9f6860162496e57c65d690c3 100644 GIT binary patch delta 157 zcmZn(XbISGQh?Lg#Kb^H!NSaP@)S|~&4Pln713LlMc8v*#5-5!ozfjVAIFC?bfWrd{52^J*bw?#U+d9+UUT+izx9_{Fl>UznL0 E05M7^GXMYp delta 112 zcmZn(XbISGQh?LQ#KKrd!PwGxvYWX5Wou+iiOc@IVbpt$H{V-c;%C*`_. - ---- - -By following this guide, you can maintain a clean, readable, and standardized codebase using Black. diff --git a/docs-dev/usage-of-branching.rst b/docs-dev/guider-of-branching.rst similarity index 62% rename from docs-dev/usage-of-branching.rst rename to docs-dev/guider-of-branching.rst index 4da1a24..984a891 100644 --- a/docs-dev/usage-of-branching.rst +++ b/docs-dev/guider-of-branching.rst @@ -6,7 +6,8 @@ This project follows a structured branching strategy to manage different version Branches -------- -### Main Branch: ``main`` +Main Branch: ``main`` +--------------------- The ``main`` branch represents the stable, production-ready version of the codebase. It contains the latest official release of the project. Code in ``main`` is fully tested and approved for production use. @@ -14,7 +15,8 @@ The ``main`` branch represents the stable, production-ready version of the codeb - **Purpose**: Serves as the default branch for stable releases. - **Merge Strategy**: Only merges from fully tested and verified branches (such as ``development``) are allowed. -### Development Branch: ``development`` +Development Branch: ``development`` +----------------------------------- The ``development`` branch is where active development for minor versions takes place. Features, bug fixes, and improvements for the upcoming stable release are integrated here before they are fully tested and merged into ``main``. @@ -22,7 +24,8 @@ The ``development`` branch is where active development for minor versions takes - **Purpose**: Acts as the integration branch for ongoing work and testing of the next stable release. - **Merge Strategy**: Developers create feature branches off ``development`` and merge completed features back into ``development`` after testing. -### Feature Branch: ``dev-3.0.0`` +Feature Branch: ``dev-3.0.0`` +----------------------------- The ``dev-3.0.0`` branch is dedicated to developing and testing major updates or changes that are not yet stable enough for integration with ``main``. This branch includes work on significant new features and alterations, such as the introduction of new data sources and schema changes in the database. @@ -31,6 +34,29 @@ The ``dev-3.0.0`` branch is dedicated to developing and testing major updates or - **Merge Strategy**: Only merged into ``main`` after extensive testing and stability checks. May include code merged from ``development`` if the major version requires updates from minor releases. - **Versioning Note**: This branch was initially created from version 2.4.3. Any updates beyond 2.4.3 should be reflected in this branch to maintain continuity. +Documentation Branch: ``gh-pages`` +---------------------------------- + +The ``gh-pages`` branch is a dedicated branch used exclusively for hosting the development documentation of the project. This branch is managed automatically by GitHub Actions, and its content is regenerated every time a new commit is pushed to the ``development`` branch. + +- **Purpose**: Hosts the HTML files generated by Sphinx for the developer documentation. +- **Restrictions**: + - **Do Not Edit Directly**: The branch is overwritten automatically during each deployment. Any manual changes to this branch will be lost. + - All updates to the documentation must be made in the ``docs-dev`` directory on the ``development`` branch. The GitHub Actions workflow then builds and deploys these changes to the ``gh-pages`` branch. +- **Access**: The documentation can be accessed at: + [https://ritchielab.github.io/biofilter/](https://ritchielab.github.io/biofilter/) + +Workflow for Updating Documentation +----------------------------------- + +1. Make changes to the documentation in the ``docs-dev`` directory in the ``development`` branch. +2. Commit and push your changes to the ``development`` branch. +3. GitHub Actions will automatically: + - Build the documentation using Sphinx. + - Deploy the generated HTML files to the ``gh-pages`` branch. +4. Verify the updated documentation. + + Workflow Summary ---------------- @@ -38,13 +64,17 @@ Workflow Summary 2. **Testing and Integration**: When a feature is complete, it is merged into ``development`` for testing and integration with other recent changes. 3. **Stable Release Preparation**: Once the ``development`` branch is stable and ready for release, it is merged into ``main``, creating an official, stable release. 4. **Major Version Development**: Experimental or significant changes go into ``dev-3.0.0`` until they are ready for production. +5. **Documentation Deployment**: Updates to the ``docs-dev`` directory are built and deployed to the ``gh-pages`` branch automatically by GitHub Actions. + Branching Best Practices ------------------------- -- **Keep ``main`` stable**: Only tested, production-ready code should be merged into ``main``. +- **Keep main stable**: Only tested, production-ready code should be merged into ``main``. - **Use Feature Branches**: For each new feature or fix, create a branch off ``development``. This keeps ``development`` organized and easier to manage. - **Document and Review Changes**: Always document significant changes to the codebase. Use pull requests to review and approve changes before merging into shared branches. +- **Do Not Modify ``gh-pages`` Directly**: Always update documentation in the ``docs-dev`` directory in the ``development`` branch. + Conclusion ---------- diff --git a/docs-dev/usage-of-coverage.rst b/docs-dev/guider-of-coverage.rst similarity index 58% rename from docs-dev/usage-of-coverage.rst rename to docs-dev/guider-of-coverage.rst index 9ceb3c8..835bab5 100644 --- a/docs-dev/usage-of-coverage.rst +++ b/docs-dev/guider-of-coverage.rst @@ -1,25 +1,31 @@ Coverage Guide ============== -To measure test coverage in the Biofilter project, we use `Coverage.py`. This guide outlines the steps to install and run `coverage` and generate reports. +To measure test coverage in the Biofilter project, we use `Coverage.py`. This guide outlines the steps to install, run, and generate coverage reports while considering the integration with Poetry and the CI/CD pipeline. 1. Install Coverage.py ----------------------- -Install `coverage` via pip: +`Coverage.py` is already included in the project's development dependencies via Poetry. To ensure it is installed, simply run: .. code-block:: bash - pip install coverage + poetry install + +This command installs all development dependencies, including `Coverage.py`. + +.. note:: + + There is no need to install `coverage` manually using `pip`, as it is managed through Poetry. 2. Run Tests with Coverage -------------------------- -In the Biofilter project directory, use `coverage` to run the tests and collect coverage data: +In the Biofilter project directory, use `coverage` to run the tests and collect coverage data. Since we are using Poetry, the command is: .. code-block:: bash - coverage run -m pytest + poetry run coverage run -m pytest This command uses `coverage` to execute `pytest`, running all tests and collecting information on which parts of the code are covered. @@ -30,7 +36,7 @@ After running the tests, generate a coverage report in the terminal: .. code-block:: bash - coverage report -m + poetry run coverage report -m - The ``-m`` option displays which lines were not covered. @@ -41,7 +47,7 @@ For a more detailed view, generate an HTML report: .. code-block:: bash - coverage html + poetry run coverage html This creates a directory called ``htmlcov`` with an ``index.html`` file. Open this file in a browser for a detailed view of the coverage. @@ -65,31 +71,40 @@ To exclude specific files or certain lines (e.g., docstrings or debugging instru This configuration ensures that coverage focuses only on the relevant parts of the codebase. +Integration with CI/CD +---------------------- + +Test coverage is also verified automatically in the CI/CD pipeline through GitHub Actions. This ensures that every change to the codebase maintains or improves test coverage. Before pushing changes, it is recommended to check the coverage locally using the commands outlined above. + +.. important:: + + Always verify coverage locally before committing to avoid failures during the CI/CD process. + Summary of Commands ------------------- -1. **Install**: +1. **Install dependencies**: (if not already installed) .. code-block:: bash - pip install coverage + poetry install 2. **Run tests with coverage**: .. code-block:: bash - coverage run -m pytest + poetry run coverage run -m pytest 3. **Generate terminal report**: .. code-block:: bash - coverage report -m + poetry run coverage report -m 4. **Generate HTML report**: .. code-block:: bash - coverage html + poetry run coverage html + -These steps will help you measure test coverage in the Biofilter project using `Coverage.py`. diff --git a/docs-dev/guider-of-git.rst b/docs-dev/guider-of-git.rst new file mode 100644 index 0000000..9fc9910 --- /dev/null +++ b/docs-dev/guider-of-git.rst @@ -0,0 +1,229 @@ +GIT Guider +=========== + +This document provides instructions for contributors on how to effectively use Git in the **Biofilter** project. It includes steps for cloning the repository, working with branches, linking changes to GitHub Issues, using GitHub Actions, and other project-specific practices. + +Getting Started +--------------- + +Clone the Repository +~~~~~~~~~~~~~~~~~~~~ + +To start working on the project, clone the GitHub repository to your local machine: + +.. code-block:: bash + + git clone https://github.com/RitchieLab/biofilter.git + cd biofilter + + +Switch to the Development Branch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The project follows a structured branching strategy, with active development happening on the ``development`` branch. Switch to this branch before making any changes: + +.. code-block:: bash + + git checkout development + +.. note:: + The ``main`` branch is reserved for stable, production-ready code. Always use ``development`` for feature work or bug fixes. + + +Working with Branches +--------------------- + +Create a New Feature or Bug Fix Branch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To avoid conflicts and maintain a clean workflow, create a new branch for each feature or bug fix: + +.. code-block:: bash + + git checkout -b feature/ + +For bug fixes, use: + +.. code-block:: bash + + git checkout -b bugfix/ + +Naming Conventions for Branches +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- **Feature branches**: ``feature/`` +- **Bug fix branches**: ``bugfix/`` +- **Hotfix branches**: ``hotfix/`` + +Example: + +.. code-block:: bash + + git checkout -b feature/add-tox-tests + + +Linking Changes to GitHub Issues +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To link your changes to a GitHub Issue, reference the Issue number in your branch name or commit messages: + +- Include the Issue number in your branch name: + + .. code-block:: bash + + git checkout -b feature/12-add-tox-tests + +- Mention the Issue in your commit message: + + .. code-block:: bash + + git commit -m "Fixes #12: Add tox tests for Python 3.10 and 3.12" + + +Commit and Push Changes +------------------------ + +Commit Your Changes +~~~~~~~~~~~~~~~~~~~~ + +Once you've made changes, stage and commit them: + +.. code-block:: bash + + git add . + git commit -m "Brief description of your changes" + +Push Your Changes to GitHub +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Push your branch to the remote repository: + +.. code-block:: bash + + git push origin + +Example: + +.. code-block:: bash + + git push origin feature/add-tox-tests + +Create a Pull Request (PR) +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +After pushing your changes, open a Pull Request on GitHub to merge your branch into the ``development`` branch. Make sure to: +- Assign reviewers. +- Link any relevant Issues in the PR description. +- Provide a detailed summary of your changes. + +GitHub Actions +-------------- + +The Biofilter project uses **GitHub Actions** for Continuous Integration (CI). These actions automatically validate your changes by running tests and building documentation. + +Key GitHub Actions +~~~~~~~~~~~~~~~~~~ + +- **Tests**: Runs `tox` across multiple Python versions. +- **Documentation Build**: Builds and deploys Sphinx documentation for both user and developer guides. + +Workflow Triggers +~~~~~~~~~~~~~~~~~ + +Actions are triggered automatically when: +- A Pull Request is opened or updated. +- Changes are pushed to the ``development`` or ``main`` branches. + +Reviewing Action Results +~~~~~~~~~~~~~~~~~~~~~~~~ + +After pushing your changes, check the **Actions** tab on GitHub to review the results. Address any failing checks before requesting a review. + +Common Commands +--------------- + +Here’s a quick reference for common Git commands used in the project: + +- **Check Current Branch**: + + .. code-block:: bash + + git branch + +- **Fetch Latest Changes**: + + .. code-block:: bash + + git fetch + +- **Pull Updates for a Branch**: + + .. code-block:: bash + + git pull origin + +- **Delete a Local Branch**: + + .. code-block:: bash + + git branch -d + +- **Delete a Remote Branch**: + + .. code-block:: bash + + git push origin --delete + +Best Practices +-------------- + +1. **Always Pull Latest Changes**: + Before starting any work, ensure your ``development`` branch is up to date: + + .. code-block:: bash + + git checkout development + git pull origin development + +2. **Commit Often**: + Make small, incremental commits with descriptive messages. + +3. **Keep Pull Requests Small**: + Focus on one feature or fix per PR to make reviews easier. + +4. **Run Tests Locally**: + Always run tests locally before pushing changes: + + .. code-block:: bash + + poetry run tox + +5. **Follow Branching Strategy**: + Ensure you create branches from ``development`` and merge them back into ``development``. + +Troubleshooting +--------------- + +- **Merge Conflicts**: + If you encounter merge conflicts when pulling updates, resolve them manually and commit the resolved files. + +- **Forgotten to Switch Branch**: + If you accidentally commit to the wrong branch, create a new branch and reset the original: + + .. code-block:: bash + + git branch feature/fix-issue-123 + git reset --hard origin/development + +- **Changes Not Reflecting**: + Ensure you’ve staged and committed all changes: + + .. code-block:: bash + + git status + +Additional Resources +-------------------- + +- `Git Documentation `_ +- `GitHub Actions `_ diff --git a/docs-dev/usage-of-poetry.rst b/docs-dev/guider-of-poetry.rst similarity index 80% rename from docs-dev/usage-of-poetry.rst rename to docs-dev/guider-of-poetry.rst index 3b686d9..eb6225b 100644 --- a/docs-dev/usage-of-poetry.rst +++ b/docs-dev/guider-of-poetry.rst @@ -27,7 +27,8 @@ Why Use Poetry? Setting Up the Project Locally ------------------------------ -### Step 1: Install Poetry +Step 1: Install Poetry +~~~~~~~~~~~~~~~~~~~~~~ Follow the official Poetry installation instructions: @@ -43,14 +44,16 @@ After installation, verify the installation: If you're on Windows, you can use the official installer or refer to the `Poetry documentation `_. -### Step 2: Clone the Repository +Step 2: Clone the Repository +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: bash - git clone https://github.com/.../biofilter.git + git clone https://github.com/RitchieLab/biofilter.git cd biofilter -### Step 3: Install Project Dependencies +Step 3: Install Project Dependencies +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Poetry will automatically create a virtual environment and install all dependencies specified in ``pyproject.toml``: @@ -58,7 +61,14 @@ Poetry will automatically create a virtual environment and install all dependenc poetry install -### Step 4: Activate the Virtual Environment + +.. important:: + + Before running `poetry install`, ensure you are using the desired Python version. This can either be the globally installed Python version or a specific version managed through tools like pyenv. If using pyenv, set the Python version for the project using pyenv local before proceeding with the installation. + + +Step 4: Activate the Virtual Environment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To activate the virtual environment created by Poetry, run: @@ -68,7 +78,8 @@ To activate the virtual environment created by Poetry, run: You can now use the installed dependencies and run project-specific commands within this environment. -### Step 5: Run the Project +Step 5: Run the Project +~~~~~~~~~~~~~~~~~~~~~~~~ For example, to run one of the entry-point scripts defined in the project: @@ -79,7 +90,8 @@ For example, to run one of the entry-point scripts defined in the project: Creating Installation Packages ------------------------------ -### Step 1: Validate the Project Setup +Step 1: Validate the Project Setup +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Ensure the ``pyproject.toml`` file is properly configured with all required metadata, dependencies, and entry points. Example fields to check: - ``name`` @@ -97,7 +109,8 @@ Example entry in ``pyproject.toml``: biofilter = "biofilter_modules.biofilter:main" loki-build = "loki_modules.loki_build:main" -### Step 2: Build the Package +Step 2: Build the Package +~~~~~~~~~~~~~~~~~~~~~~~~~ Run the following command to generate distribution files (``.tar.gz`` and ``.whl``) in the ``dist/`` directory: @@ -109,7 +122,8 @@ This command will create: - A source distribution (``.tar.gz``). - A wheel distribution (``.whl``). -### Step 3: Verify the Package +Step 3: Verify the Package +~~~~~~~~~~~~~~~~~~~~~~~~~~ To test the installation of the package locally, use: @@ -119,7 +133,8 @@ To test the installation of the package locally, use: Replace ```` with the version number of the package. -### Step 4: Publish the Package +Step 4: Publish the Package +~~~~~~~~~~~~~~~~~~~~~~~~~~~ To publish the package to PyPI (or a private repository), use: @@ -168,7 +183,8 @@ Run it: Managing Dependencies ---------------------- -### Adding a Dependency +Adding a Dependency +~~~~~~~~~~~~~~~~~~~ To add a new dependency to the project, use: @@ -176,15 +192,17 @@ To add a new dependency to the project, use: poetry add -### Adding a Development Dependency +Adding a Development Dependency +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To add a dependency for development purposes (e.g., linters, testing frameworks): .. code-block:: bash - poetry add --dev + poetry add --group dev -### Removing a Dependency +Removing a Dependency +~~~~~~~~~~~~~~~~~~~~~ To remove a dependency from the project: @@ -192,6 +210,7 @@ To remove a dependency from the project: poetry remove + Reproducibility --------------- diff --git a/docs-dev/guider-of-sphinx.rst b/docs-dev/guider-of-sphinx.rst new file mode 100644 index 0000000..5a8b884 --- /dev/null +++ b/docs-dev/guider-of-sphinx.rst @@ -0,0 +1,222 @@ +Sphinx Guide +============ + +This document provides instructions for contributors on how to set up, customize, and run Sphinx to generate documentation for the **Biofilter** project. The project uses Sphinx to maintain two separate documentations: one for **user documentation** and another for **developer documentation**. + +Documentation Structure +----------------------- + +The Biofilter project maintains two distinct documentation directories: + +1. **User Documentation**: + - Located in the ``docs/`` directory. + - Hosted on **ReadTheDocs**. + - Contains user-facing guides, usage instructions, and detailed module documentation. + +2. **Developer Documentation**: + - Located in the ``docs-dev/`` directory. + - Hosted on **GitHub Pages**. + - Contains internal technical documentation for contributors, including guides on tools like Tox, Poetry, Black, and branching strategies. + +Each directory has its own Sphinx configuration file (``conf.py``) and operates independently. + + +Prerequisites +------------- + +Ensure you have the following tools installed: + +- Python (>= 3.10) +- Poetry (manages dependencies, including Sphinx) + +.. note:: + + Sphinx is already included as a development dependency in the ``pyproject.toml`` file. You do not need to install it manually. + + +Working with Documentation +-------------------------- + +User Documentation (``docs/``) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To build and preview the user documentation: + +1. Navigate to the ``docs/`` directory: + + .. code-block:: bash + + cd docs + +2. Build the HTML documentation: + + .. code-block:: bash + + make html + +3. Open the generated documentation in a browser: + + .. code-block:: bash + + open _build/html/index.html + +This documentation is automatically built and hosted on **ReadTheDocs**. + + +Developer Documentation (``docs-dev/``) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To build and preview the developer documentation: + +1. Navigate to the ``docs-dev/`` directory: + + .. code-block:: bash + + cd docs-dev + +2. Build the HTML documentation: + + .. code-block:: bash + + make html + +3. Open the generated documentation in a browser: + + .. code-block:: bash + + open _build/html/index.html + +This documentation is hosted on **GitHub Pages**, and any updates are automatically deployed through a GitHub Actions workflow. + +.. important:: + + The ``gh-pages`` branch is used exclusively for hosting the developer documentation. It is automatically managed by GitHub Actions and **should not be modified manually**. + + +Customizing Documentation +-------------------------- + +Configuration Files +~~~~~~~~~~~~~~~~~~~ + +Each documentation directory has its own ``conf.py`` file for configuration. Key sections to customize include: + +1. **Project Information**: + + .. code-block:: python + + project = 'Biofilter' + author = 'Ritchie Lab' + release = '2.4.4' + +2. **Path Settings**: + + Ensure modules are accessible to Sphinx: + + .. code-block:: python + + import os + import sys + sys.path.insert(0, os.path.abspath("..")) + +3. **Extensions**: + + Add or modify Sphinx extensions as needed: + + .. code-block:: python + + extensions = [ + 'sphinx.ext.autodoc', + 'sphinx.ext.viewcode', + 'sphinx.ext.napoleon', + 'sphinx_rtd_theme', + ] + +4. **HTML Theme**: + + Set the theme for the documentation: + + .. code-block:: python + + html_theme = 'sphinx_rtd_theme' # For user documentation + html_theme = 'sphinx_material' # For developer documentation + +Customizing and Regenerating +---------------------------- + +1. **Edit Index Files**: + + - ``docs/index.rst``: Entry point for user documentation. + - ``docs-dev/index.rst``: Entry point for developer documentation. + + Add or modify sections and include additional ``.rst`` files using: + + .. code-block:: rst + + .. toctree:: + :maxdepth: 2 + :caption: Contents: + + usage-guide + developer-guide + +2. **Generate Module Documentation**: + + Use ``sphinx-apidoc`` to create ``.rst`` files for modules: + + .. code-block:: bash + + sphinx-apidoc -o . ../biofilter_modules + +3. **Rebuild Documentation**: + + After making changes, rebuild the documentation: + + .. code-block:: bash + + make clean + make html + +Tips for Contributors +--------------------- + +1. **Docstrings**: + Write detailed docstrings in your codebase. Use Google or NumPy style for compatibility with the ``napoleon`` extension. + +2. **Test Locally**: + Always test your changes locally by building the documentation before pushing updates. + +3. **Branch Management**: + - User documentation updates should be committed to the ``main`` branch. + - Developer documentation updates should be committed to the ``development`` branch. GitHub Actions will handle the deployment to ``gh-pages``. + + +Troubleshooting +--------------- + +- **Sphinx Command Not Found**: + + Ensure you are in the Poetry environment: + + .. code-block:: bash + + poetry shell + +- **Broken Links**: + + Check for broken links using: + + .. code-block:: bash + + make linkcheck + +- **Missing Modules**: + + Verify module paths in ``conf.py`` under ``sys.path``. + +Additional Resources +-------------------- + +- `Sphinx Documentation `_ +- `Google Style Docstrings `_ +- `Napoleon Extension `_ diff --git a/docs-dev/guider-of-tests.rst b/docs-dev/guider-of-tests.rst new file mode 100644 index 0000000..8050438 --- /dev/null +++ b/docs-dev/guider-of-tests.rst @@ -0,0 +1,157 @@ +Testing Guider +============== + +This document outlines the structure, configuration, and execution of tests in the **Biofilter** project using Pytest. It also explains how tests are integrated into the development workflow and the CI/CD pipeline. + +Test Structure +-------------- + +The tests are organized into the following directories under the ``tests/`` folder: + +- **Unit Tests**: + - Located in ``tests/unit/``. + - Focused on small, isolated portions of the codebase, often using fixtures to provide mock data and parameters. + +- **Functional Tests**: + - Located in ``tests/functions/``. + - Test specific functionalities to ensure they behave as expected in real-world scenarios. + +- **Issues Tests**: + - Located in ``tests/issues/``. + - Designed to replicate and debug problems reported via GitHub Issues. + - Create scenarios based on issue descriptions to verify fixes and identify potential regressions. + +VS Code Integration +------------------- + +Configuring VS-Code for Pytest +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To integrate Pytest into VS Code, the ``settings.json`` file in your project’s ``.vscode/`` directory is configured to use Pytest as the default test framework: + +.. code-block:: json + + { + "python.testing.unittestEnabled": false, + "python.testing.pytestEnabled": true, + "python.testing.pytestArgs": ["tests"] + } + +With this configuration, you can: + +- Discover all tests automatically in the ``tests/`` folder. +- Run individual tests or groups of tests directly within VS Code using the built-in testing features. + +Running Tests in VS-Code +~~~~~~~~~~~~~~~~~~~~~~~~ + +1. Open the Testing sidebar in VS Code (Ctrl+Shift+T or Cmd+Shift+T). +2. All discovered tests will appear in the sidebar, organized by directories. +3. Click the play icon next to a test or folder to run individual tests or groups of tests. + +Unit Tests and Fixtures +----------------------- + +Unit Test Isolation +~~~~~~~~~~~~~~~~~~~ + +Unit tests focus on specific code components in isolation. To achieve this, we use **fixtures** that provide initial parameters, mock objects, or sample data for the tests. Fixtures help: + +- Isolate the code being tested. +- Eliminate dependencies on external systems or databases. +- Ensure predictable, reproducible results. + +Example Fixture +^^^^^^^^^^^^^^^ + +Here’s an example of a fixture in the project: + +.. code-block:: python + + import pytest + + @pytest.fixture + def sample_data(): + return { + "param1": 42, + "param2": "example" + } + + def test_example_function(sample_data): + result = example_function(sample_data["param1"]) + assert result == expected_result + +Integration with CI/CD +---------------------- + +The Biofilter project integrates tests into the CI/CD pipeline using GitHub Actions. This ensures that every push and pull request is validated against the test suite. + +Workflow for Tests +~~~~~~~~~~~~~~~~~~ + +1. **Automated Execution**: + - On every commit or pull request to the ``development`` branch, the tests are executed in the GitHub Actions pipeline. + +2. **Coverage Reports**: + - Test coverage is measured during the pipeline execution and can be reviewed to identify untested code. + +3. **Failure Alerts**: + - If a test fails, the CI/CD pipeline is marked as failed, preventing unreviewed changes from being merged into stable branches. + +Running Tests Locally +--------------------- + +To run the tests locally using Pytest, use the following commands: + +Run All Tests +~~~~~~~~~~~~~ + +.. code-block:: bash + + poetry run pytest + +Run Specific Tests +~~~~~~~~~~~~~~~~~~ + +- By directory: + + .. code-block:: bash + + poetry run pytest tests/unit/ + +- By file: + + .. code-block:: bash + + poetry run pytest tests/issues/test_issue_123.py + +- By individual test function: + + .. code-block:: bash + + poetry run pytest -k "test_specific_function" + +Best Practices +-------------- + +1. **Isolate Unit Tests**: + Use fixtures to ensure that unit tests are not dependent on external systems or complex configurations. + +2. **Link Issues Tests to GitHub**: + Ensure that tests in ``tests/issues/`` reference the corresponding GitHub Issue for better traceability. + +3. **Run Tests Locally Before Pushing**: + Always run the full test suite locally to catch errors early. + +4. **Maintain Test Coverage**: + Strive for high test coverage to ensure the robustness of the codebase. + +5. **Document Test Scenarios**: + Document the purpose and setup of tests, especially for complex scenarios in ``tests/issues/``. + + +Additional Resources +-------------------- + +- `Pytest Documentation `_ +- `GitHub Actions Documentation `_ diff --git a/docs-dev/usage-of-tox.rst b/docs-dev/guider-of-tox.rst similarity index 53% rename from docs-dev/usage-of-tox.rst rename to docs-dev/guider-of-tox.rst index 6dcabdc..5b3a74e 100644 --- a/docs-dev/usage-of-tox.rst +++ b/docs-dev/guider-of-tox.rst @@ -1,4 +1,4 @@ -Tox Guider +Tox Guide ========== Tox is a tool used to automate testing across multiple Python versions and environments. It ensures consistency and compatibility in your project's workflows. @@ -13,19 +13,10 @@ Tox allows you to: - Automate dependency installation and testing. - Isolate environments to ensure reproducibility. -Setting Up Tox --------------- - -Tox is already included as a development dependency in the **Biofilter** project. If you need to install it manually, use the following command: - -.. code-block:: bash - - poetry add --group dev tox - Tox Configuration ------------------ -Tox is configured via the `tox.ini` file in the root of the project. Below is an example configuration: +Tox is configured via the ``tox.ini`` file in the root of the project. Below is an example configuration: .. code-block:: ini @@ -47,7 +38,8 @@ Tox is configured via the `tox.ini` file in the root of the project. Below is an Using Tox --------- -### Running All Environments +Running All Environments +~~~~~~~~~~~~~~~~~~~~~~~~ To run tests across all configured Python versions, execute: @@ -57,47 +49,52 @@ To run tests across all configured Python versions, execute: Tox will create isolated environments and execute the tests in each. -### Running Specific Environments +Running Specific Environments +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To test a specific Python version or configuration, use the `-e` flag: +To test a specific Python version or configuration, use the ``-e`` flag: .. code-block:: bash -poetry run tox -e py310 + poetry run tox -e py310 -### Recreating Environments +Recreating Environments +~~~~~~~~~~~~~~~~~~~~~~~~ -If you make changes to the `tox.ini` file or dependencies, recreate the environments: +If you make changes to the ``tox.ini`` file or dependencies, recreate the environments: .. code-block:: bash - poetry run tox --recreate + poetry run tox --recreate Customizing Tox --------------- -### Specifying Python Interpreter Paths +Specifying Python Interpreter Paths +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If Tox cannot find a specific Python version, specify its path in the `tox.ini` file: +If Tox cannot find a specific Python version, specify its path in the ``tox.ini`` file: .. code-block:: ini [testenv:py310] basepython = /path/to/python3.10 -Replace `/path/to/python3.10` with the actual path of your Python interpreter. +Replace ``/path/to/python3.10`` with the actual path of your Python interpreter. -### Parallel Execution +Parallel Execution +~~~~~~~~~~~~~~~~~~ To speed up testing, you can run environments in parallel: .. code-block:: bash - poetry run tox -p auto + poetry run tox -p auto -### Adding Environment Variables +Adding Environment Variables +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To pass environment variables during testing, add them to the `tox.ini` file: +To pass environment variables during testing, add them to the ``tox.ini`` file: .. code-block:: ini @@ -116,8 +113,23 @@ Run Tox with the environment variable set: Cleaning Up Tox --------------- -To clean up all Tox environments, remove the `.tox/` directory: +To clean up all Tox environments, remove the ``.tox/`` directory: .. code-block:: bash rm -rf .tox/ + +CI/CD Integration +------------------ + +Tox is integrated into the project's CI/CD pipeline. Whenever changes are pushed to the repository, Tox automatically validates the codebase across the specified Python versions. This ensures that new changes are compatible and do not introduce regressions. + +.. important:: + + Always run Tox locally before committing to identify and fix any compatibility issues in advance. + +Additional Resources +-------------------- + +- `Tox Documentation `_ +- `Poetry Documentation `_ diff --git a/docs-dev/index.rst b/docs-dev/index.rst index 4aab5de..4c333c4 100644 --- a/docs-dev/index.rst +++ b/docs-dev/index.rst @@ -20,8 +20,8 @@ Documentation for Users ----------------------- For more detailed instructions and usage examples, refer to the user documentation: -- Biofilter User Guide -- LOKI User Guide +- `Biofilter User Guide `_. +- `LOKI User Guide `_. Key Features @@ -136,13 +136,13 @@ Follow these steps to set up the **Biofilter** project for development or testin .. toctree:: :maxdepth: 2 - :caption: Contents: - - usage-of-branching - usage-of-poetry - usage-of-black - usage-of-coverage - usage-of-tox - usage-of-sphinx - usage-of-git + + guider-of-branching + guider-of-poetry + guider-of-black + guider-of-tests + guider-of-coverage + guider-of-tox + guider-of-sphinx + guider-of-git \ No newline at end of file diff --git a/docs-dev/usage-of-git.rst b/docs-dev/usage-of-git.rst deleted file mode 100644 index 0d9d66c..0000000 --- a/docs-dev/usage-of-git.rst +++ /dev/null @@ -1,3 +0,0 @@ -Using git and CI/CD by Actions -============================== - diff --git a/docs-dev/usage-of-sphinx.rst b/docs-dev/usage-of-sphinx.rst deleted file mode 100644 index 00112c5..0000000 --- a/docs-dev/usage-of-sphinx.rst +++ /dev/null @@ -1,217 +0,0 @@ -Sphinx Guider -============= - -This document provides instructions for contributors on how to set up, customize, and run Sphinx to generate documentation for the **Biofilter** project. - -Prerequisites -------------- - -Before working with the documentation, ensure you have the following installed: - -- Python (version >= 3.10) -- Poetry (for dependency management) - -Setting Up the Environment ---------------------------- - -1. **Clone the Repository**: - - .. code-block:: bash - - git clone https://github.com/.../biofilter.git - cd biofilter - -2. **Install Dependencies**: - - Install all project dependencies, including Sphinx, using Poetry: - - .. code-block:: bash - - poetry install - - Activate the virtual environment: - - .. code-block:: bash - - poetry shell - -Initializing Sphinx -------------------- - -1. **Navigate to the ``docs/`` Directory**: - - The documentation is located in the ``docs/`` directory. Navigate to it: - - .. code-block:: bash - - cd docs - -2. **Customize the Configuration**: - - The main configuration file for Sphinx is ``conf.py`` in the ``docs/`` directory. Open and edit it as needed: - - .. code-block:: bash - - nano conf.py - - Key sections you may want to customize: - - - **Project Information**: - - .. code-block:: python - - project = 'Biofilter' - author = 'Ritchie Lab' - release = '2.4.4' - - - **Path Settings**: - - Ensure the project’s modules are accessible to Sphinx: - - .. code-block:: python - - import os - import sys - sys.path.insert(0, os.path.abspath("..")) - - - **Extensions**: - - Add or remove Sphinx extensions as needed: - - .. code-block:: python - - extensions = [ - 'sphinx.ext.autodoc', - 'sphinx.ext.viewcode', - 'sphinx.ext.napoleon', - 'sphinx_rtd_theme', - ] - - - **HTML Theme**: - - Set the theme for the generated documentation: - - .. code-block:: python - - html_theme = 'sphinx_rtd_theme' - -Generating the Documentation ------------------------------ - -1. **Build the HTML Documentation**: - - To generate the HTML version of the documentation: - - .. code-block:: bash - - make html - - The output will be located in the ``_build/html/`` directory. - -2. **View the Documentation**: - - Open the ``index.html`` file in a browser to view the documentation: - - .. code-block:: bash - - xdg-open _build/html/index.html - - On macOS: - - .. code-block:: bash - - open _build/html/index.html - -Customizing Documentation --------------------------- - -1. **Edit ``index.rst``**: - - The ``index.rst`` file serves as the main entry point for your documentation. You can add sections or include additional ``.rst`` files here. - -2. **Generate Module Documentation Automatically**: - - Use ``sphinx-apidoc`` to generate ``.rst`` files for your Python modules: - - .. code-block:: bash - - sphinx-apidoc -o . ../biofilter_modules - - This command scans the ``biofilter_modules`` directory and generates ``.rst`` files for each module. - -3. **Add New Sections**: - - Create new ``.rst`` files for additional sections and include them in ``index.rst`` using: - - .. code-block:: rst - - .. toctree:: - :maxdepth: 2 - :caption: Contents: - - modules - new_section - -Regenerating Documentation ---------------------------- - -If you make changes to the codebase or ``.rst`` files, regenerate the documentation: - -.. code-block:: bash - - make clean - make html - -Tips for Contributors ---------------------- - -1. **Use Docstrings**: - - Write detailed docstrings in your Python code using Google or NumPy style for compatibility with the ``napoleon`` extension. - -2. **Update ``conf.py``**: - - If new directories or modules are added, update the ``sys.path`` in ``conf.py`` to ensure Sphinx can locate them. - -3. **Preview Locally**: - - Always preview the generated documentation locally before submitting changes. - -4. **Commit Changes**: - - If you add or update ``.rst`` files, commit them to the repository: - - .. code-block:: bash - - git add docs/ - git commit -m "Update Sphinx documentation" - -Troubleshooting ---------------- - -- **Sphinx Command Not Found**: - - Ensure you are in the Poetry environment: - - .. code-block:: bash - - poetry shell - -- **Missing Modules in Documentation**: - - Check that the module paths are included in ``conf.py`` under ``sys.path``. - -- **Broken Links**: - - Run the following command to check for broken links: - - .. code-block:: bash - - make linkcheck - -Additional Resources --------------------- - -- `Sphinx Documentation `_ -- `Google Style Docstrings `_ -- `Napoleon Extension `_ diff --git a/tests/issues/b15_biofilter_group_annotation/data-out/outcome_run_2.configuration b/tests/issues/b15_biofilter_group_annotation/data-out/outcome_run_2.configuration index 7d2eed0..cbdf5b8 100644 --- a/tests/issues/b15_biofilter_group_annotation/data-out/outcome_run_2.configuration +++ b/tests/issues/b15_biofilter_group_annotation/data-out/outcome_run_2.configuration @@ -1,5 +1,5 @@ # Biofilter configuration file -# generated Tue, 03 Dec 2024 11:00:39 +# generated Tue, 03 Dec 2024 17:45:38 # Biofilter version 2.4.4 (2024-12-01) # LOKI version 2.2.5 (2019-03-15) diff --git a/tests/issues/l16_build_37_loki/data-out/outcomes.configuration b/tests/issues/l16_build_37_loki/data-out/outcomes.configuration index b81a39a..e8349d5 100644 --- a/tests/issues/l16_build_37_loki/data-out/outcomes.configuration +++ b/tests/issues/l16_build_37_loki/data-out/outcomes.configuration @@ -1,5 +1,5 @@ # Biofilter configuration file -# generated Tue, 03 Dec 2024 11:00:39 +# generated Tue, 03 Dec 2024 17:45:39 # Biofilter version 2.4.4 (2024-12-01) # LOKI version 2.2.5 (2019-03-15)