diff --git a/.gitignore b/.gitignore index 29e50e655..7518a3d74 100644 --- a/.gitignore +++ b/.gitignore @@ -2,12 +2,10 @@ __pycache__ # do not track doc files that are automatically build doc/build doc/source/*.rst +doc/source/*.md # but keep index.rst !doc/source/index.rst -!doc/source/intro.rst -!doc/source/tutorials.rst -!doc/source/architecture.rst # C extensions *.so diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cd9980d97..46a80f38b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -32,9 +32,11 @@ have to solve them yourself. ### 1. Install QUEENS in developer mode Install QUEENS as described in the [README.md](README.md) and run: + ``` pip install -e .[develop] ``` + ### 2. Configure our git-hooks To help you write style-compliant code, we use the [pre-commit](https://pre-commit.com/) package to manage all our git @@ -54,7 +56,7 @@ The code checks are conducted with [Pylint](https://pylint.org/), Compliance with [Google style docstrings](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings) is checked with [ruff](https://github.com/astral-sh/ruff). Complete and meaningful docstrings are required as they are used to generate the -[documentation](#reading-and-writing-documentation). +[documentation](#book-documentation). #### Commit messages Please provide meaningful commit messages based on the diff --git a/README.md b/README.md index fb7560017..9cbdfec50 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,11 @@ -
+ - - - QUEENS logo + + + QUEENS logo +
@@ -24,8 +25,11 @@
+ QUEENS (**Q**uantification of **U**ncertain **E**ffects in **En**gineering **S**ystems) is a Python framework for solver-independent multi-query analyses of large-scale computational models. + + :chart_with_upwards_trend: **QUEENS** offers a large collection of cutting-edge algorithms for deterministic and probabilistic analyses such as: * parameter studies and identification * sensitivity analysis @@ -38,17 +42,22 @@ QUEENS (**Q**uantification of **U**ncertain **E**ffects in **En**gineering **S** * robust data, resource, and error management * easy switching between analysis types * smooth scaling from laptop to HPC cluster + ## :rocket: Getting started + >**Prerequisites**: Unix system and environment management system (we recommend [miniforge](https://conda-forge.org/download/)) + + Clone the QUEENS repository to your local machine. Navigate to its base directory, then: ``` conda env create conda activate queens pip install -e . ``` + ## :crown: Workflow example @@ -97,7 +106,7 @@ if __name__ == "__main__":
-QUEENS logo +QUEENS example
## :busts_in_silhouette: Contributing @@ -107,6 +116,7 @@ Your contributions are welcome! Please follow our [contributing guidelines](http ## :page_with_curl: How to cite If you use QUEENS in your work, please cite the relevant method papers and + ```bib @misc{queens, author = {QUEENS}, @@ -115,6 +125,9 @@ If you use QUEENS in your work, please cite the relevant method papers and howpublished = {\url{https://www.queens-py.org}} } ``` + ## :woman_judge: License + Licensed under GNU LGPL-3.0 (or later). See [LICENSE](LICENSE). + diff --git a/dev-requirements.in b/dev-requirements.in index 3127e5dcf..2ee9356c4 100644 --- a/dev-requirements.in +++ b/dev-requirements.in @@ -20,3 +20,4 @@ commitizen>=3.12.0 docformatter>=1.5.1 yamllint>=1.19.0 ruff +myst-parser diff --git a/dev-requirements.txt b/dev-requirements.txt index e46df7509..0f569ee66 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -1,5 +1,5 @@ # -# This file is autogenerated by pip-compile with Python 3.11 +# This file is autogenerated by pip-compile with Python 3.10 # by the following command: # # pip-compile --output-file=dev-requirements.txt dev-requirements.in @@ -62,6 +62,7 @@ docformatter==1.7.5 # via -r dev-requirements.in docutils==0.21.2 # via + # myst-parser # nbsphinx # pydata-sphinx-theme # sphinx @@ -79,6 +80,10 @@ idna==3.10 # requests imagesize==1.4.1 # via sphinx +importlib-metadata==6.11.0 + # via + # -c requirements.txt + # build isort==5.13.2 # via # -r dev-requirements.in @@ -87,6 +92,7 @@ jinja2==3.1.4 # via # -c requirements.txt # commitizen + # myst-parser # nbconvert # nbsphinx # sphinx @@ -108,6 +114,11 @@ liccheck==0.9.2 # via -r dev-requirements.in licenseheaders==0.8.8 # via -r dev-requirements.in +markdown-it-py==3.0.0 + # via + # -c requirements.txt + # mdit-py-plugins + # myst-parser markupsafe==3.0.2 # via # -c requirements.txt @@ -115,8 +126,16 @@ markupsafe==3.0.2 # nbconvert mccabe==0.7.0 # via pylint +mdit-py-plugins==0.4.2 + # via myst-parser +mdurl==0.1.2 + # via + # -c requirements.txt + # markdown-it-py mistune==3.0.2 # via nbconvert +myst-parser==4.0.0 + # via -r dev-requirements.in nbclient==0.10.0 # via nbconvert nbconvert==7.16.4 @@ -188,6 +207,7 @@ pyyaml==6.0.2 # via # -c requirements.txt # commitizen + # myst-parser # pre-commit # yamllint pyzmq==26.2.0 @@ -228,6 +248,7 @@ soupsieve==2.6 sphinx==8.1.3 # via # -r dev-requirements.in + # myst-parser # nbsphinx # pydata-sphinx-theme sphinxcontrib-applehelp==2.0.0 @@ -250,6 +271,14 @@ tinycss2==1.4.0 # via nbconvert toml==0.10.2 # via liccheck +tomli==2.0.2 + # via + # -c requirements.txt + # build + # pip-tools + # pre-commit-hooks + # pylint + # sphinx tomlkit==0.13.2 # via # commitizen @@ -269,6 +298,7 @@ traitlets==5.14.3 typing-extensions==4.12.2 # via # -c requirements.txt + # astroid # pydata-sphinx-theme untokenize==0.1.1 # via docformatter @@ -290,6 +320,10 @@ wheel==0.44.0 # pip-tools yamllint==1.35.1 # via -r dev-requirements.in +zipp==3.20.2 + # via + # -c requirements.txt + # importlib-metadata # The following packages are considered to be unsafe in a requirements file: # pip diff --git a/doc/source/_ext/create_documentation_files.py b/doc/source/_ext/create_documentation_files.py new file mode 100644 index 000000000..64da2327b --- /dev/null +++ b/doc/source/_ext/create_documentation_files.py @@ -0,0 +1,250 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: LGPL-3.0-or-later +# Copyright (c) 2025, QUEENS contributors. +# +# This file is part of QUEENS. +# +# QUEENS is free software: you can redistribute it and/or modify it under the terms of the GNU +# Lesser General Public License as published by the Free Software Foundation, either version 3 of +# the License, or (at your option) any later version. QUEENS is distributed in the hope that it will +# be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You +# should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, +# see . +# +"""Documentation creation utils.""" + +import pydoc +import re +import sys +from pathlib import Path + +import requests + +from queens.utils.injector import inject +from queens.utils.path_utils import relative_path_from_queens + +sys.path.insert(1, str(relative_path_from_queens("test_utils").resolve())) +from get_queens_example_from_readme import ( # pylint: disable=import-error, wrong-import-position,wrong-import-order + extract_from_markdown_file_by_marker, + get_queens_example_from_readme, +) + + +def get_template_path_by_name(template_name): + """Get template path by name from the template folder. + + Args: + template_name (str): Temple name in the template folder. + + Returns: + pathlib.Path: Path to template + """ + template_path = (Path(__file__).parent / "templates") / template_name + return template_path + + +def relative_to_doc_source(relative_path): + """Relative path from documentation source. + + Args: + relative_path (str): path relative to `doc/source/` + + Returns: + pathlib.Path: Path relative from documentation + """ + return relative_path_from_queens("doc/source/" + relative_path) + + +def create_tutorial_from_readme(): + """Create tutorial from readme.""" + example = get_queens_example_from_readme(".") + tutorial_template = get_template_path_by_name("tutorials.rst.j2") + tutorial = relative_to_doc_source("tutorials.rst") + + inject({"example_text": example.replace("\n", "\n ")}, tutorial_template, tutorial) + + +def remove_markdown_emojis(md_text): + """Remove emojis from markdown text. + + Args: + md_text (str): Markdown text + + Returns: + str: Cleaned text + """ + # Define a regex pattern for matching markdown-style emojis + emoji_pattern = re.compile(r":\w+:") + + for emoji in emoji_pattern.findall(md_text): + + # Remove markdown emojis from the text + md_text = md_text.replace(emoji, "") + + # Replace emojis in reference to other sections + md_text = md_text.replace(emoji[1:-1], "") + + return md_text + + +def prepend_relative_links(md_text, base_url): + """Prepend url for relative links. + + Args: + md_text (str): Text to check + base_url (str): Base URL to add + + Returns: + str: Prepended markdown text + """ + md_link_regex = "\\[([^]]+)]\\(\\s*(.*)\\s*\\)" + for match in re.findall(md_link_regex, md_text): + _, link = match + + # For local reference that started with an emoji + if link.strip().startswith("#-"): + new_link = "#" + link[2:].strip().lower() + md_text = md_text.replace(f"({link})", f"({new_link})") + + # No http links or proper references within the file references + if not link.strip().startswith("http") and not link.strip().startswith("#"): + md_text = md_text.replace(f"({link})", f"({base_url}/{link.strip()})") + + return md_text + + +def clean_markdown(md_text): + """Clean markdown. + + Removes emojis and prepends links. + + Args: + md_text (str): Original markdown text. + + Returns: + str: Markdown text + """ + md_text = remove_markdown_emojis(md_text) + md_text = prepend_relative_links(md_text, "https://www.github.com/queens-py/queens/blob/main") + return md_text + + +def clean_markdown_file(md_path, new_path): + """Load markdown and escape relative links and emojis. + + Args: + md_path (pathlib.Path, str): Path to an existing markdown file + new_path (pathlib.Path, str): Path for the cleaned file + + Returns: + str: file name of the new markdown file + """ + md_text = clean_markdown(relative_path_from_queens(md_path).read_text()) + new_path = Path(new_path) + new_path.write_text(md_text, encoding="utf-8") + return new_path.name + + +def create_development(): + """Create development page.""" + development_template = get_template_path_by_name("development.rst.j2") + development_path = relative_to_doc_source("development.rst") + + md_paths = [] + md_paths.append( + clean_markdown_file( + relative_path_from_queens("CONTRIBUTING.md"), relative_to_doc_source("contributing.md") + ) + ) + md_paths.append( + clean_markdown_file( + relative_path_from_queens("tests/README.md"), relative_to_doc_source("testing.md") + ) + ) + inject({"md_paths": md_paths}, development_template, development_path) + + +def create_intro(): + """Generate landing page.""" + intro_template = get_template_path_by_name("intro.md.j2") + into_path = relative_to_doc_source("intro.md") + + def extract_from_markdown_by_marker(marker_name, md_path): + return clean_markdown(extract_from_markdown_file_by_marker(marker_name, md_path)) + + inject( + { + "readme_path": relative_path_from_queens("README.md"), + "contributing_path": relative_path_from_queens("CONTRIBUTING.md"), + "extract_from_markdown_by_marker": extract_from_markdown_by_marker, + }, + intro_template, + into_path, + ) + + +def create_overview(): + """Create overview of the QUEENS package.""" + overview_template = get_template_path_by_name("overview.rst.j2") + overview_path = relative_to_doc_source("overview.rst") + + queens_base_path = relative_path_from_queens("queens") + + def get_module_description(python_file): + """Get module description. + + Args: + python_file (pathlib.Path): Path to python file. + + Returns: + str: Module description. + """ + module_documentation = pydoc.importfile(str(python_file)).__doc__.split("\n\n") + return "\n\n".join([m.replace("\n", " ") for m in module_documentation[1:]]) + + modules = [] + for path in sorted(queens_base_path.iterdir()): + if path.name.startswith("__") or not path.is_dir(): + continue + + description = get_module_description(path / "__init__.py") + name = path.stem + + modules.append( + { + "name": name.replace("_", " ").title(), + "module": "queens." + name, + "description": description, + } + ) + + inject({"modules": modules, "len": len}, overview_template, overview_path) + + +def download_images(): + """Download images.""" + + def download_file_from_url(url, file_name): + """Download file from an url.""" + url_request = requests.get(url, timeout=10) + with open(file_name, "wb") as f: + f.write(url_request.content) + + download_file_from_url( + "https://raw.githubusercontent.com/queens-py/queens-design/main/logo/queens_logo_night.svg", + relative_to_doc_source("images/queens_logo_night.svg"), + ) + download_file_from_url( + "https://raw.githubusercontent.com/queens-py/queens-design/main/logo/queens_logo_day.svg", + relative_to_doc_source("images/queens_logo_day.svg"), + ) + + +def main(): + """Create all the rst files.""" + create_intro() + create_tutorial_from_readme() + create_development() + create_overview() diff --git a/doc/source/_ext/queens_sphinx_extension.py b/doc/source/_ext/queens_sphinx_extension.py new file mode 100644 index 000000000..0946606de --- /dev/null +++ b/doc/source/_ext/queens_sphinx_extension.py @@ -0,0 +1,38 @@ +# +# SPDX-License-Identifier: LGPL-3.0-or-later +# Copyright (c) 2025, QUEENS contributors. +# +# This file is part of QUEENS. +# +# QUEENS is free software: you can redistribute it and/or modify it under the terms of the GNU +# Lesser General Public License as published by the Free Software Foundation, either version 3 of +# the License, or (at your option) any later version. QUEENS is distributed in the hope that it will +# be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You +# should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, +# see . +# +"""This is a custom extension to create files for sphinx using python.""" + +import os +import sys + +from sphinx.application import Sphinx + +sys.path.insert(0, os.path.abspath(".")) + +import create_documentation_files # pylint: disable=wrong-import-position + + +def run_custom_code(app: Sphinx): # pylint: disable=unused-argument + """Run the custom code.""" + create_documentation_files.main() + + +def setup(app: Sphinx): # pylint: disable=unused-argument + """Setup up sphinx app.""" + app.connect("builder-inited", run_custom_code) + return { + "version": "0.1", + "parallel_read_safe": True, + } diff --git a/doc/source/_ext/templates/development.rst.j2 b/doc/source/_ext/templates/development.rst.j2 new file mode 100644 index 000000000..7e49d1d46 --- /dev/null +++ b/doc/source/_ext/templates/development.rst.j2 @@ -0,0 +1,11 @@ +Development +=========== + +Thanks for your interest in developing QUEENS! + +.. toctree:: + :maxdepth: 1 + + +{% for md_path in md_paths %} + {{ md_path }}{% endfor %} diff --git a/doc/source/_ext/templates/intro.md.j2 b/doc/source/_ext/templates/intro.md.j2 new file mode 100644 index 000000000..3b159c4be --- /dev/null +++ b/doc/source/_ext/templates/intro.md.j2 @@ -0,0 +1,37 @@ +# Introduction + +```{image} images/queens_logo_day.svg +:class: only-light +:width: 500px +:align: center +``` + +```{image} images/queens_logo_night.svg +:class: only-dark +:width: 500px +:align: center +``` + +{{ extract_from_markdown_by_marker("description", readme_path) }} + +## Capabilities +{{ extract_from_markdown_by_marker("capabilities", readme_path) }} + + +## Installation +{{ extract_from_markdown_by_marker("prerequisites", readme_path) }} + +{{ extract_from_markdown_by_marker("installation", readme_path) }} + +For development, install the additional required packages via: +{{ extract_from_markdown_by_marker("installation_develop", contributing_path) }} + +> Note: We recommend using conda/mamba environments and installing performance-critical packages (e.g., numpy, scipy, ...) using `conda install .` The reason for this is the choice of BLAS library (linear algebra packages). Conda (depending on the channel) installs numpy and the [mkl](https://www.intel.com/content/www/us/en/developer/articles/technical/intel-mkl-and-third-party-applications-how-to-use-them-together.html) library from Intel, in contrast to pip which defaults back to the linear algebra package installed on the system. According to certain benchmarks ([here](http://markus-beuckelmann.de/blog/boosting-numpy-blas.html) or [here](https://medium.com/analytics-vidhya/why-conda-install-instead-of-pip-install-ba4c6826a0ae)), the mkl library is able to outperform other linear algebra libraries, especially on Intel devices. Particularly for use cases where linear algebra operations dominate the computational costs, the benefit can be huge. + +## Citing QUEENS +You used QUEENS for a publication? Don't forget to cite +{{ extract_from_markdown_by_marker("citation", readme_path) }} +and the respective methods papers. + +## License +{{ extract_from_markdown_by_marker("license", readme_path) }} diff --git a/doc/source/_ext/templates/overview.rst.j2 b/doc/source/_ext/templates/overview.rst.j2 new file mode 100644 index 000000000..253cc88cf --- /dev/null +++ b/doc/source/_ext/templates/overview.rst.j2 @@ -0,0 +1,16 @@ +Modules Overview +================ + +Here, we present a brief overview of the fundamental modules of QUEENS. + +{% for module in modules %} +{{ module["name"] }} +{{ "-"*len(module["name"]) }} + +.. toctree:: + :maxdepth: 1 + + {{ module["module"] }} + +{{ module["description"] }} +{% endfor %} diff --git a/doc/source/_ext/templates/tutorials.rst.j2 b/doc/source/_ext/templates/tutorials.rst.j2 new file mode 100644 index 000000000..11a9af55d --- /dev/null +++ b/doc/source/_ext/templates/tutorials.rst.j2 @@ -0,0 +1,19 @@ +.. _tutorials: + +Tutorials and examples +================================== + +Work in progress! + +But here a simple example from our `README.md `_: + +.. code-block:: python + +{{ example_text }} + +Resulting in the histogram: + +.. image:: images/monte_carlo_uq.png + :width: 500 + :align: center + :alt: Monte Carlo Histogram diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst deleted file mode 100644 index 5222c1c1f..000000000 --- a/doc/source/architecture.rst +++ /dev/null @@ -1,115 +0,0 @@ ---------------------------- -Architecture Overview ---------------------------- -Here, we present a brief overview of the fundamental architecture of QUEENS. - -Data processor ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.data_processor - - -Distributions ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.distributions - - -Drivers ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.drivers - - -Example simulator functions ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.example_simulator_functions - - -External geometry ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.external_geometry - - -Interfaces ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.interfaces - - -Iterators ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.iterators - - -Models ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.models - - -Parameters ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.parameters - - -Schedulers ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.schedulers - - -Stochastic optimizers ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.stochastic_optimizers - - -Utils ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.utils - - -Variational distributions ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.variational_distributions - - -Visualization ---------------------------- -.. toctree:: - :maxdepth: 4 - - queens.visualization diff --git a/doc/source/conf.py b/doc/source/conf.py index de532a852..5015c3ea8 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -2,7 +2,7 @@ # -*- coding: utf-8 -*- # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -26,8 +26,8 @@ import sys from pathlib import Path -# sys.path.insert(0, str(Path('.')resolve)) -sys.path.insert(0, str(Path("../../").resolve)) +sys.path.insert(0, str(Path("../../").resolve())) +sys.path.append(str(Path("_ext").resolve())) # -- General configuration ------------------------------------------------ @@ -48,6 +48,8 @@ "sphinx.ext.inheritance_diagram", "sphinx.ext.napoleon", "nbsphinx", + "queens_sphinx_extension", + "myst_parser", ] # Custom command to make the returns in the docstring behave like the parameter/argument section. @@ -151,8 +153,8 @@ # html_theme_options = { "logo": { - "image_light": "https://raw.githubusercontent.com/queens-py/queens-design/main/logo/queens_logo_day.svg", - "image_dark": "https://raw.githubusercontent.com/queens-py/queens-design/main/logo/queens_logo_night.svg", + "image_light": "images/queens_logo_day.svg", + "image_dark": "images/queens_logo_night.svg", } } @@ -182,7 +184,7 @@ # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ["_static"] +# html_static_path = ["_static"] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied diff --git a/readme_images/monte_carlo_uq.png b/doc/source/images/monte_carlo_uq.png similarity index 100% rename from readme_images/monte_carlo_uq.png rename to doc/source/images/monte_carlo_uq.png diff --git a/doc/source/images/queens_logo_day.svg b/doc/source/images/queens_logo_day.svg new file mode 100644 index 000000000..96d520b5f --- /dev/null +++ b/doc/source/images/queens_logo_day.svg @@ -0,0 +1,327 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + + + + + + + + + + QUEENS + + diff --git a/doc/source/images/queens_logo_night.svg b/doc/source/images/queens_logo_night.svg new file mode 100644 index 000000000..e9e0ea15c --- /dev/null +++ b/doc/source/images/queens_logo_night.svg @@ -0,0 +1,327 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + + + + + + + + + + QUEENS + + diff --git a/doc/source/index.rst b/doc/source/index.rst index 4a59291c4..4a703ba73 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -1,19 +1,22 @@ -.. pqueens documentation master file, created by - sphinx-quickstart on Tue Apr 4 13:44:16 2017. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. - Welcome to the QUEENS' documentation! -=================================== +===================================== + +**Some useful links**: + +* `Repository on GitHub `_ +* `Website `_ + .. toctree:: - :maxdepth: 2 + :maxdepth: 1 :caption: Contents: intro - architecture + overview tutorials - modules.rst + development + queens + Indices and tables @@ -21,4 +24,3 @@ Indices and tables * :ref:`genindex` * :ref:`modindex` -* :ref:`search` diff --git a/doc/source/intro.rst b/doc/source/intro.rst deleted file mode 100644 index bd950c1e3..000000000 --- a/doc/source/intro.rst +++ /dev/null @@ -1,10 +0,0 @@ -Introduction -=================================== -QUEENS (Quantification of Uncertain Effects in Engineering Systems) is a Python framework for solver-independent multi-query analyses of large-scale computational models. - - -Installation ------------- -For a quick installation guide, QUEENS users should refer to our `README.md `_. - -QUEENS contributors should follow the installation guide and the instructions in our `CONTRIBUTING.md `_. diff --git a/doc/source/tutorials.rst b/doc/source/tutorials.rst deleted file mode 100644 index d9baef81f..000000000 --- a/doc/source/tutorials.rst +++ /dev/null @@ -1,6 +0,0 @@ -.. _tutorials: - -Tutorials and examples -================================== - -Work in progress... diff --git a/queens/data_processor/__init__.py b/queens/data_processor/__init__.py index 1e5e9f708..592712d30 100644 --- a/queens/data_processor/__init__.py +++ b/queens/data_processor/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,9 +12,10 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Data processor. +"""Data Processor. -Extract data from simulation output. +The data processor modules are used to extract data from simulation +model outputs. """ from queens.data_processor.data_processor_csv import DataProcessorCsv diff --git a/queens/distributions/__init__.py b/queens/distributions/__init__.py index f9cbee304..069deb349 100644 --- a/queens/distributions/__init__.py +++ b/queens/distributions/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,7 +12,10 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Distributions.""" +"""Distributions. + +Probability distribution modules of QUEENS. +""" from queens.distributions.bernoulli import BernoulliDistribution from queens.distributions.beta import BetaDistribution diff --git a/queens/drivers/__init__.py b/queens/drivers/__init__.py index a99fe9b1e..0d1083754 100644 --- a/queens/drivers/__init__.py +++ b/queens/drivers/__init__.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # diff --git a/queens/example_simulator_functions/__init__.py b/queens/example_simulator_functions/__init__.py index a863499d4..228e74050 100644 --- a/queens/example_simulator_functions/__init__.py +++ b/queens/example_simulator_functions/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -15,7 +15,7 @@ """Example simulator functions. This package contains a set of example simulator functions, which can be -used as benchmark. +used for benchmarking. """ from queens.example_simulator_functions.agawal09 import agawal09a diff --git a/queens/external_geometry/__init__.py b/queens/external_geometry/__init__.py index 6c5411e32..dc0b8d78d 100644 --- a/queens/external_geometry/__init__.py +++ b/queens/external_geometry/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,9 +12,10 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""External geometry module. +"""External geometry. -Read in external geometry to QUEENS. +Read in external geometry from simulation software to QUEENS for +preprocessing. """ from queens.external_geometry.fourc_dat_geometry import FourcDatExternalGeometry diff --git a/queens/global_settings.py b/queens/global_settings.py index 293bba8c0..917d2ea9e 100644 --- a/queens/global_settings.py +++ b/queens/global_settings.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,7 +12,11 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Global Settings module.""" +"""Global Settings. + +This module provides a context for QUEENS runs with exit functionality +for logging and working with remote resources. +""" import logging from pathlib import Path diff --git a/queens/interfaces/__init__.py b/queens/interfaces/__init__.py index a110820ab..8d8aa0334 100644 --- a/queens/interfaces/__init__.py +++ b/queens/interfaces/__init__.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -17,10 +17,6 @@ This package contains a set of so-called interfaces. The purpose of an interface is essentially the mapping of inputs to outputs. - -The mapping is made by passing the inputs further down to a -*regression_approximation* or a *mf_regression_approximation*, both of -which essentially then evaluate a regression model themselves. """ from queens.interfaces.bmfia_interface import BmfiaInterface diff --git a/queens/iterators/__init__.py b/queens/iterators/__init__.py index dfc999dd2..e2d0aa835 100644 --- a/queens/iterators/__init__.py +++ b/queens/iterators/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -14,12 +14,10 @@ # """Iterators. -The iterator package contains the implementation of several UQ and -optimization methods, each of which is implemented in their own iterator -class. The iterator is therefor one of the central building blocks, as -the iterators orchestrate the evaluations on one or multiple models. -QUEENS also permits nesting of iterators to enable hierarchical methods -or surrogate based UQ approaches. +The iterator package contains methods for parameters studies, +uncertainty quantification, sensitivity analysis, Bayesian inverse +analysis, and optimization. These methods iterate on QUEENS models in a +multi-query fashion. """ from queens.iterators.black_box_variational_bayes import BBVIIterator diff --git a/queens/main.py b/queens/main.py index 139bc4176..8a6ed8a85 100644 --- a/queens/main.py +++ b/queens/main.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,7 +12,11 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Main module of QUEENS containing the high-level control routine.""" +"""QUEENS main. + +Main module of QUEENS containing the high-level control routine for +input file workflow. +""" import logging import sys diff --git a/queens/models/__init__.py b/queens/models/__init__.py index f6c9b4068..47978ce82 100644 --- a/queens/models/__init__.py +++ b/queens/models/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -15,8 +15,8 @@ """Models. The model package contains several types of models to be used in the -context of UQ. Within QUEENS, the model class of object holds and stores -the input and output data, and can evaluate itself to produce data. +context of multi-query analysis. Within QUEENS, the model returns its +evaluations and possibly gradients. """ from queens.models.bmfmc_model import BMFMCModel diff --git a/queens/parameters/__init__.py b/queens/parameters/__init__.py index 9eb6b3a6d..4d1542c27 100644 --- a/queens/parameters/__init__.py +++ b/queens/parameters/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,6 +12,11 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Parameters.""" +"""Parameters. + +In the context of QUEENS, parameters are altered for the multi-query +analysis. They can either be deterministic, random variables or random +fields. +""" from queens.parameters.parameters import Parameters diff --git a/queens/parameters/parameters.py b/queens/parameters/parameters.py index d58097278..7f78dfa5f 100644 --- a/queens/parameters/parameters.py +++ b/queens/parameters/parameters.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,7 +12,7 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Parameters module.""" +"""Parameters.""" import logging diff --git a/queens/schedulers/__init__.py b/queens/schedulers/__init__.py index c23f1ca80..e2b7521c2 100644 --- a/queens/schedulers/__init__.py +++ b/queens/schedulers/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # diff --git a/queens/stochastic_optimizers/__init__.py b/queens/stochastic_optimizers/__init__.py index bed03e843..f537a79f2 100644 --- a/queens/stochastic_optimizers/__init__.py +++ b/queens/stochastic_optimizers/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,7 +12,10 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Stochastic optimizers init.""" +"""Stochastic optimizers. + +This package consists of a set of stochastic optimizers. +""" from queens.stochastic_optimizers.adam import Adam from queens.stochastic_optimizers.adamax import Adamax diff --git a/queens/utils/__init__.py b/queens/utils/__init__.py index b0d0e4a24..5e0ea8651 100644 --- a/queens/utils/__init__.py +++ b/queens/utils/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,4 +12,7 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Utils.""" +"""Utils. + +A large collections of utilities used in all of QUEENS. +""" diff --git a/queens/variational_distributions/__init__.py b/queens/variational_distributions/__init__.py index f33b172ca..4cf603ed1 100644 --- a/queens/variational_distributions/__init__.py +++ b/queens/variational_distributions/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,7 +12,10 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # -"""Variational distributions init.""" +"""Variational distributions. + +A probability distributions package for variational inference. +""" from queens.variational_distributions.full_rank_normal import FullRankNormalVariational from queens.variational_distributions.joint import JointVariational diff --git a/queens/visualization/__init__.py b/queens/visualization/__init__.py index 6ec608e5b..4312c98a9 100644 --- a/queens/visualization/__init__.py +++ b/queens/visualization/__init__.py @@ -1,6 +1,6 @@ # # SPDX-License-Identifier: LGPL-3.0-or-later -# Copyright (c) 2024, QUEENS contributors. +# Copyright (c) 2025, QUEENS contributors. # # This file is part of QUEENS. # @@ -12,3 +12,7 @@ # should have received a copy of the GNU Lesser General Public License along with QUEENS. If not, # see . # +"""Visualization. + +A set of modules to visualize results of different iterators. +""" diff --git a/test_utils/get_queens_example_from_readme.py b/test_utils/get_queens_example_from_readme.py index 504135fe3..8b87a1c2e 100644 --- a/test_utils/get_queens_example_from_readme.py +++ b/test_utils/get_queens_example_from_readme.py @@ -15,29 +15,43 @@ """Extract QUEENS example from the readme.""" +from pathlib import Path + from queens.utils.path_utils import relative_path_from_queens -EXAMPLE_MARKER = "" - -def get_queens_example_from_readme(output_dir): - """Extract the example from the readme. +def extract_from_markdown_file_by_marker(marker_name, md_file): + """Extract section from an markdown file. Args: - output_dir (str): Output directory for the QUEENS run. + marker_name (str): Name of the section to extract. + md_file (str, pathlib.Path): Path to the markdown file + + Returns: + str: section as string """ - readme_path = relative_path_from_queens("README.md") + marker = f"" # Split the example in the readme using the marker - text = readme_path.read_text().split(EXAMPLE_MARKER) + text = Path(md_file).read_text(encoding="utf-8").split(marker) # Only one example should appear if len(text) != 3: - raise ValueError("Could not extract the example from the readme!") + raise ValueError(f"Could not extract the section marked with '{marker}' from {md_file}!") + + return text[1] + + +def get_queens_example_from_readme(output_dir): + """Extract the example from the readme. + + Args: + output_dir (str): Output directory for the QUEENS run. + """ + readme_path = relative_path_from_queens("README.md") - # Extract the source example_source = ( - text[1] + extract_from_markdown_file_by_marker("example", readme_path) .replace("```python", "") .replace("```", "") .replace('output_dir="."', f'output_dir="{output_dir}"') diff --git a/tests/README.md b/tests/README.md index 1e2f1fdf6..7134fe900 100644 --- a/tests/README.md +++ b/tests/README.md @@ -16,23 +16,23 @@ Therefore, we test the QUEENS code base ## :running_woman: Running tests QUEENS is tested using [pytest](https://docs.pytest.org/en/stable/index.html). For a comprehensive list of pytest commands, see [here](https://docs.pytest.org/en/stable/how-to/usage.html). Some additional useful commands to test QUEENS are listed in the following: -| Description | Command | -| ----------- | ----------- | -| Test parallelly using pytest-xdist | `pytest -n ` | -| Test with verbose output | `pytest -ra -v` | -| Test with logging output | `pytest -o log_cli=true --log-cli-level=INFO` | -| Generate a coverage report | `pytest --cov-report=html --cov` | -| Test only the last failed | `pytest --lf` | +| Test | Command | +| ----------------------------- | --------------------------------------------- | +| In parallel with pytest-xdist | `pytest -n ` | +| With verbose output | `pytest -ra -v` | +| With logging output | `pytest -o log_cli=true --log-cli-level=INFO` | +| With coverage report | `pytest --cov-report=html --cov` | +| Only the last failed | `pytest --lf` | ### :bookmark: Pytest markers In QUEENS, tests are organized using pytest markers. This allows you to run all tests in a group with a single command: -| Description| Command | -| ----------- | ----------- | -| Unit tests | `pytest -m unit_tests` | -| Integration tests without 4C | `pytest -m integration_tests` | -| Integration tests with 4C (see below) | `pytest -m integration_tests_fourc` | -|Get a list of all available markers| `pytest --markers` | +| Description | Command | +| ------------------------------- | ----------------------------------- | +| Unit tests | `pytest -m unit_tests` | +| Integration tests | `pytest -m integration_tests` | +| 4C integration test (see below) | `pytest -m integration_tests_fourc` | +| List markers | `pytest --markers` | ### :four_leaf_clover: Integration tests with 4C For the integration tests in QUEENS that require the multiphysics simulation framework [4C](https://github.com/4C-multiphysics/4C), the user needs to create a **symbolic link** to the 4C-executable and store it under `/config`: