Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project: Change to the rye #1210

Merged
merged 1 commit into from
Jul 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 14 additions & 13 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,26 +25,27 @@ jobs:
- name: Acquire sources
uses: actions/checkout@v4

- name: Install Poetry
run: pipx install poetry

- name: Setup Python
uses: actions/setup-python@v5
- name: Install Rye
uses: eifinger/setup-rye@v4
with:
python-version: "3.12"
architecture: x64
cache: poetry
cache-dependency-path: |
pyproject.toml
poetry.lock
enable-cache: true

- name: Install uv
run: rye install uv

- name: Pin python version
run: rye pin 3.12

- name: Install project
run: rye sync

- name: Install project
run: .github/workflows/install.sh testing

- name: Run tests, with coverage
run: |
poetry run pytest --cov=wetterdienst tests/
poetry run coverage xml
rye run pytest --cov=wetterdienst tests/
rye run coverage xml

- name: Upload coverage report to Codecov
uses: codecov/codecov-action@v4
Expand Down
19 changes: 10 additions & 9 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,19 @@ jobs:
- name: Acquire sources
uses: actions/checkout@v4

- name: Install Poetry
run: pipx install poetry

- name: Setup Python
uses: actions/setup-python@v5
- name: Install Rye
uses: eifinger/setup-rye@v4
with:
python-version: "3.12"
architecture: x64
cache: poetry
enable-cache: true

- name: Pin python version
run: rye pin 3.12

- name: Install project
run: rye sync

- name: Build wheel package
run: poetry build --format=wheel
run: rye build --wheel

- name: Upload wheel package
uses: actions/upload-artifact@v4
Expand Down
15 changes: 2 additions & 13 deletions .github/workflows/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,20 +13,9 @@ set -e
set -x

if [ "${flavor}" = "testing" ]; then
poetry install --verbose --no-interaction \
--with=test,dev \
--extras=bufr \
--extras=explorer \
--extras=export \
--extras=import \
--extras=interpolation \
--extras=ipython \
--extras=radar \
--extras=radarplus \
--extras=restapi \
--extras=sql
uv pip install .[bufr,explorer,export,import,interpolation,ipython,radar,radarplus,restapi,sql]

elif [ "${flavor}" = "docs" ]; then
poetry install --verbose --no-interaction --with=docs --extras=interpolation
uv pip install .[docs,interpolation]

fi
27 changes: 12 additions & 15 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,22 +21,19 @@ jobs:
- name: Acquire sources
uses: actions/checkout@v4

- name: Install Poetry
run: pipx install poetry

- name: Setup Python
id: python-setup
uses: actions/setup-python@v5
- name: Install Rye
uses: eifinger/setup-rye@v4
with:
python-version: "3.12"
architecture: x64
cache: poetry
cache-dependency-path: |
pyproject.toml
poetry.lock
enable-cache: true

- name: Install uv
run: rye install uv

- name: Pin python version
run: rye pin 3.12

- name: Install project tooling
run: poetry install --no-interaction --no-root --with=dev
- name: Install project
run: rye sync

- name: Run code-style checks
run: poetry run poe lint
run: rye run lint
27 changes: 14 additions & 13 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,25 @@ jobs:
- name: Acquire sources
uses: actions/checkout@v4

- name: Install Poetry
run: pipx install poetry

- name: Setup Python
uses: actions/setup-python@v5
- name: Install Rye
uses: eifinger/setup-rye@v4
with:
python-version: "3.12"
architecture: x64
cache: poetry
cache-dependency-path: |
pyproject.toml
poetry.lock
enable-cache: true

- name: Install uv
run: rye install uv

- name: Pin python version
run: rye pin 3.12

- name: Install project
run: rye sync

- name: Install project
run: .github/workflows/install.sh testing

- name: Build
run: poetry build
run: rye build

- name: Publish
run: poetry publish --username=__token__ --password=${{ secrets.PYPI_TOKEN }}
run: rye publish --username=__token__ --password=${{ secrets.PYPI_TOKEN }} --yes
45 changes: 11 additions & 34 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,38 +44,19 @@ jobs:
- name: Acquire sources
uses: actions/checkout@v4

# Workaround until pipx is available on M1 machines
# - https://github.com/actions/runner-images/issues/9256
# We install poetry using brew instead of pipx
- name: Install Poetry (Mac M1 only)
if: ${{ startsWith(matrix.os, 'macos-14') }}
run: brew install poetry

- name: Install Poetry
if: ${{ !startsWith(matrix.os, 'macos-14') }}
run: pipx install poetry

- name: Setup Python
if: ${{ !startsWith(matrix.os, 'macos-14') }}
uses: actions/setup-python@v5
- name: Install Rye
uses: eifinger/setup-rye@v4
with:
python-version: ${{ matrix.python-version }}
architecture: x64
cache: poetry
cache-dependency-path: |
pyproject.toml
poetry.lock
enable-cache: true

- name: Setup Python (Mac M1 only)
if: ${{ startsWith(matrix.os, 'macos-14') }}
run: |
brew install pyenv
pyenv install ${{ matrix.python-version }}
pyenv global ${{ matrix.python-version }}
- name: Install uv
run: rye install uv

export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
- name: Pin python version
run: rye pin ${{ matrix.python-version }}

- name: Install project
run: rye sync

- name: Install eccodes (Mac only)
if: ${{ startsWith(matrix.os, 'macos-') }}
Expand All @@ -86,10 +67,6 @@ jobs:
- name: Install project
run: .github/workflows/install.sh testing

- name: Install Jupyter kernel
run: poetry run python -m ipykernel install --name "python3" --user

- name: Run tests
run: |
poetry run pytest -vvv --numprocesses=auto -m "not (explorer or cflake)" tests
poetry run pytest -vvv -m cflake tests
rye run test
8 changes: 1 addition & 7 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,10 @@ build:
# https://docs.readthedocs.io/en/stable/build-customization.html#extend-the-build-process
jobs:
post_install:

# setuptools has been downgraded to 58.2.0, so upgrade again.
- pip install --upgrade setuptools

# Install Poetry.
# https://python-poetry.org/docs/#installing-manually
- pip install poetry==1.8.3

- pip install uv
# Install project's dependencies.
# Tell Poetry to use the RTD virtual environment.
# see https://github.com/python-poetry/poetry/issues/9025
- VIRTUAL_ENV=$READTHEDOCS_VIRTUALENV_PATH .github/workflows/install.sh docs

Expand Down
12 changes: 7 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ RUN \
git build-essential python3-dev python3-pip python3-venv python3-wheel \
python3-h5py ca-certificates pkg-config libhdf5-dev

RUN pip install uv

# Install wradlib.
RUN python -m pip install --prefer-binary wradlib
RUN uv pip install --system wradlib

# Install Wetterdienst.

Expand All @@ -27,17 +29,17 @@ COPY dist/wetterdienst-*.whl /tmp/
RUN --mount=type=cache,id=pip,target=/root/.cache/pip \
true \
&& WHEEL=$(ls -r /tmp/wetterdienst-*-py3-none-any.whl | head -n 1) \
&& pip install --prefer-binary versioningit \
&& pip install --use-pep517 --prefer-binary ${WHEEL}[bufr,cratedb,duckdb,explorer,influxdb,interpolation,postgresql,radar,radarplus,restapi]
&& uv pip install --system versioningit \
&& uv pip install --system ${WHEEL}[bufr,cratedb,duckdb,explorer,influxdb,interpolation,postgresql,radar,radarplus,restapi]

# TODO: for linux/arm64 we currently cant install zarr as it depends on numcodecs which has no wheels
# and building it from source takes too long
# see also: https://github.com/zarr-developers/numcodecs/issues/288
RUN WHEEL=$(ls -r /tmp/wetterdienst-*-py3-none-any.whl | head -n 1) && \
if [ "$(uname -m)" = "x86_64" ]; then \
pip install --use-pep517 --prefer-binary ${WHEEL}[export]; \
uv pip install --system ${WHEEL}[export]; \
else \
pip install --use-pep517 --prefer-binary ${WHEEL}[export_without_zarr]; \
uv pip install --system ${WHEEL}[export_without_zarr]; \
fi


Expand Down
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ rdwd_ for the R language, which originally drew our interest in this project.
Our long-term goal is to provide access to multiple weather services as well as other
related agencies such as river measurements. With ``wetterdienst`` we try to use modern
Python technologies all over the place. The library is based on polars_ (we <3 pandas_, it is still part of some
IO processes) across the board, uses Poetry_ for package administration and GitHub Actions for all things CI.
IO processes) across the board, uses rye_ for package administration and GitHub Actions for all things CI.
Our users are an important part of the development as we are not currently using the
data we are providing and only implement what we think would be the best. Therefore
contributions and feedback whether it be data related or library related are very
Expand All @@ -98,7 +98,7 @@ source.
.. _rdwd: https://github.com/brry/rdwd
.. _polars: https://www.pola.rs/
.. _pandas: https://pandas.pydata.org/
.. _Poetry: https://python-poetry.org/
.. _rye: https://rye.astral.sh/

Data
****
Expand Down
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ def _get_project_meta():
with open("../pyproject.toml") as pyproject:
file_contents = pyproject.read()

return tomlkit.parse(file_contents)["tool"]["poetry"]
return tomlkit.parse(file_contents)["project"]


# -- Project information -----------------------------------------------------
pkg_meta = _get_project_meta()

project = str(pkg_meta["name"])
copyright = "Copyright (C) 2018-2023 earthobservations" # noqa: A001
author = ", ".join([author.split("<")[0].strip() for author in pkg_meta["authors"]])
author = ", ".join(author["name"] for author in pkg_meta["authors"])
version = str(pkg_meta["version"])

# -- General configuration ---------------------------------------------------
Expand Down
10 changes: 4 additions & 6 deletions docs/contribution/development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,13 @@ Acquire sources and prerequisites
cd wetterdienst

# Prerequisites
brew install --cask firefox
brew install git python geckodriver poetry
brew install git python rye

# Other OS
# You can also get installers and/or release archives for Linux, macOS
# and Windows at
#
# - https://python-poetry.org/docs/#installation
# - https://www.mozilla.org/en-US/firefox/new/
# - https://github.com/mozilla/geckodriver/releases
# - https://rye.astral.sh/


************
Expand All @@ -46,7 +43,8 @@ Using Poetry

.. code-block:: bash

poetry install --with=test,dev,docs --extras=sql --extras=export --extras=restapi --extras=explorer --extras=interpolation
rye pin 3.12
rye install --with=test,dev,docs --extras=sql --extras=export --extras=restapi --extras=explorer --extras=interpolation
poetry shell


Expand Down
Loading
Loading