Merge pull request #126 from pastas/dev

Release v1.5.0

.github/workflows/ci.yml

@@ -32,20 +32,13 @@ jobs:
           pip install .
           pip install ".[lint]"
 
-      - name: Run black
-        uses: psf/black@stable
-        with:
-          options: "--check --diff"
-          jupyter: true
-
-      - name: Run isort
-        run: |
-          isort --check --diff --profile=black ./pastastore
+      - name: ruff-lint
+        uses: chartboost/ruff-action@v1
 
-      - name: Run flake8
-        run: |
-          flake8 --count --show-source --exit-zero --max-line-length=88 ./pastastore
-          flake8 --count --exit-zero --max-line-length=88 --statistics ./pastastore
+      - name: ruff-format
+        uses: chartboost/ruff-action@v1
+        with:
+          args: "format --check"
 
   test:
     name: Tests
@@ -54,15 +47,22 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.9", "3.10", "3.11"]
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
         pastas-version:
           [
             "git+https://github.com/pastas/[email protected]",
             "git+https://github.com/pastas/[email protected]",
             "git+https://github.com/pastas/[email protected]",
-            "git+https://github.com/pastas/pastas.git@v1.4.0",
+            "git+https://github.com/pastas/pastas.git@v1.5.0",
             "git+https://github.com/pastas/pastas.git@dev",
           ]
+        exclude:
+          - python-version: "3.12"
+            pastas-version: "git+https://github.com/pastas/[email protected]"
+          - python-version: "3.12"
+            pastas-version: "git+https://github.com/pastas/[email protected]"
+          - python-version: "3.12"
+            pastas-version: "git+https://github.com/pastas/[email protected]"
 
     steps:
       - uses: actions/checkout@v4
@@ -74,24 +74,34 @@ jobs:
           cache: "pip"
           cache-dependency-path: pyproject.toml
 
-      - name: Install dependencies
+      - name: Install dependencies < PY312
+        if: ${{ matrix.python-version != '3.12'}}  
         run: |
           pip install --upgrade pip
+          pip install numpy
           pip install ${{ matrix.pastas-version }}
           pip install -e .[test]
 
+      - name: Install dependencies == PY312
+        if: ${{ matrix.python-version == '3.12'}}  
+        run: |
+          pip install --upgrade pip
+          pip install numpy
+          pip install ${{ matrix.pastas-version }}
+          pip install -e .[test_py312]
+
       - name: Run pytest
         run: |
           pytest -m "not bgmap"
 
       - name: Test backgroundmap once
-        if: ${{ matrix.python-version == '3.9' && matrix.pastas-version == 'git+https://github.com/pastas/pastas.git@dev' }}
+        if: ${{ matrix.python-version == '3.11' && matrix.pastas-version == 'git+https://github.com/pastas/pastas.git@dev' }}
         run: |
           pytest -m "bgmap"
 
       - name: Run codacy-coverage-reporter
         if: ${{ github.repository == 'pastas/pastastore' && success() }}
         uses: codacy/codacy-coverage-reporter-action@master
         with:
-          project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
+          project-token: ${{ secrets.CODACY_API_TOKEN }}
           coverage-reports: coverage.xml

connectors.md

@@ -5,8 +5,7 @@ Connectors are briefly described below.
 
 - PasConnector (works out of the box, **preferred choice**)
 - DictConnector (works out of the box)
-- ArcticConnector (not actively tested, requires `arctic` and MongoDB, **best performance**)
-- PystoreConnector (not actively tested, requires `pystore` and `python-snappy`)
+- ArcticDBConnector (requires `arcticdb`, **best performance**)
 
 ## PasConnector
 
@@ -16,7 +15,7 @@ series and pastas Models on disk.
 
 ```python
 # requires a name, and path to a folder
-conn = pst.PasConnector("my_db", path="./pastas_db")
+conn = pst.PasConnector("./pastas_db", name="my_db")
 ```
 
 ## DictConnector
@@ -30,54 +29,14 @@ storage container. All "stored" data will be lost if you restart the kernel.
 conn = pst.DictConnector("my_temporary_db")
 ```
 
-## ArcticConnector
+## ArcticDBConnector
 
-Store data in MongoDB using Arctic. Only works if there is an instance of
-MongoDB running and the `arctic` python package is installed. This Connector
-has the best performance, both in terms of read/write speeds and data
-compression. Currently, `arctic` does not work with pandas>1.1 because it
-has reference to the deprecated `pandas.Panel` object. This can be fixed by
-installing a custom version of arctic in which this bug has been addressed.
-This can be done by cloning a Pull Request and installing that version of arctic.
+Store on disk using ArcticDB. Only works if the `arcticdb` python package is installed.
+This Connector has the best overall performance in terms of read/write speeds and data
+compression.
 
 ```python
-# provide a name and a connection string to a running instance of MongoDB
-connstr = "mongodb://localhost:27017/"  # local instance of mongodb
-conn = pst.ArcticConnector("my_db", connstr)
-```
-
-## PystoreConnector
-
-Store data on disk as parquet files using compression. Only works if
-`python-snappy` and `pystore` are installed. Does not require separate database
-software running somewhere, but installation of `python-snappy` is a little
-more challenging. Slightly less performant than ArcticConnector, but faster
-than PasConnector.
-
-```python
-# provide a name and a path to a folder on disk
-conn = pst.PystoreConnector("my_db", path="./pastas_db")
+# requires a name, and path to a folder
+path = "lmdb://./pastas_db/"  # path to folder
+conn = pst.ArcticDBConnector(path, name="my_db")
 ```
-
-## Connector Dependencies
-
-This module has several dependencies (depending on which connector is used):
-
-If using `Dictconnector` or `PasConnector`:
-
-- No additional dependencies are required.
-
-If using `ArcticConnector`:
-
-- Arctic requires MongoDB, e.g. install the Community edition
-    ([Windows](https://fastdl.mongodb.org/win32/mongodb-win32-x86_64-2012plus-4.2.1-signed.msi),
-    [MacOS](https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-4.2.1.tgz)).
-- OR, if you wish to use Docker for running MongoDB see the installation
-    instructions [here](https://github.com/pastas/pastastore/tree/master/dockerfiles#running-mongodb-from-docker).
-
-If using `PystoreConnector`:
-
-- PyStore uses [Snappy](http://google.github.io/snappy/), a fast and efficient
-    compression/decompression library from Google. You'll need to install Snappy on
-    your system before installing PyStore. See links for installation instructions
-    here: <https://github.com/ranaroussi/pystore#dependencies>

docs/conf.py

@@ -1,3 +1,4 @@
+# ruff: noqa: D100
 # -*- coding: utf-8 -*-
 #
 # Configuration file for the Sphinx documentation builder.

docs/connectors.rst

@@ -63,57 +63,6 @@ are stored as pandas.DataFrames. Models are stored as pickled dictionaries
 and *do not* contain the time series themselves. These are picked up from
 the other libraries when the model is loaded from the database.
 
-Arctic
-------
-Note: this Connector is not actively tested!
-
-The :ref:`ArcticConnector` is an object that creates a
-connection with a MongoDB database. This can be an existing or a new database.
-For each of the datasets a collection or library is created. These are named
-using the following convention: `<database name>.<library name>`.
-
-The Arctic implementation uses the following structure:
-
-.. code-block::
-
-   +-- database
-   |   +-- collections or libraries (i.e. oseries, stresses, models)
-   |   |   +-- documents... (i.e. individual time series or models)
-
-The data is stored within these libraries. Observations and stresses time series
-are stored as pandas.DataFrames. Models are stored in JSON (actually binary
-JSON) and *do not* contain the time series themselves. These are picked up from
-the other libraries when the model is loaded from the database.
-
-Pystore
--------
-Note: this Connector is not actively tested!
-
-The :ref:`PystoreConnector` is an object that links
-to a location on disk. This can either be an existing or a new Pystore. A new
-store is created with collections (or libraries) that hold the different 
-datasets:
-
-* observation time series
-* stresses time series
-* models
-
-The Pystores have the following structure:
-
-.. code-block:: bash
-
-   +-- store
-   |   +-- collections or libraries... (i.e. oseries, stresses, models)
-   |   |   +-- items... (i.e. individual time series or models)
-
-
-The time series data is stored as Dask DataFrames which can be easily converted
-to pandas DataFrames. The models are stored as JSON (not including the
-time series) in the metadata file belonging to an item. The actual data in the
-item is an empty DataFrame serving as a placeholder. This slightly 'hacky'
-design allows the models to be saved in a PyStore. The time series are picked
-up from their respective stores when the model is loaded from disk.
-
 Custom Connectors
 -----------------
 It should be relatively straightforward to write your own custom connector

docs/examples.rst

@@ -58,38 +58,6 @@ a `PastaStore` object::
    store = pst.PastaStore(conn)
 
 
-Using Arctic
-------------
-
-The following snippet shows how to create an `ArcticConnector` and initialize
-a `PastaStore` object. Please note that a MongoDB instance must be running for
-this to work::
-
-   import pastastore as pst
-
-   # define arctic connector
-   connstr = "mongodb://localhost:27017/"
-   conn = pst.ArcticConnector("my_db", connstr)
-
-   # create project for managing Pastas data and models
-   store = pst.PastaStore(conn)
-
-
-Using Pystore
--------------
-
-To use the `PystoreConnector` the steps are identical, the only difference is
-that the user has to provide a path to a location on disk instead of a
-connection string to a database::
-
-   # define pystore connector
-   path = "./data/pystore"
-   conn = pst.PystoreConnector("my_db", path)
-
-   # create project for managing Pastas data and models
-   store = pst.PastasProject(conn)
-
-
 The PastaStore object
 ---------------------