Skip to content

Commit

Permalink
Merge pull request #52 from peopledoc/procrastinate
Browse files Browse the repository at this point in the history
Rename the project Procrastinate
  • Loading branch information
Joachim Jablon authored Aug 15, 2019
2 parents 1a63988 + 28398b4 commit 70078ab
Show file tree
Hide file tree
Showing 41 changed files with 219 additions and 195 deletions.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Cabbage
# procrastinate

Kind of like Celery but based on elephants (Postgres) instead of rabbits (RabbitMQ)

Expand All @@ -10,13 +10,13 @@ Launch a postgres DB the way you want, e.g. using docker:

```console
$ docker-compose up -d
$ export PGDATABASE=cabbage PGHOST=localhost PGUSER=postgres
$ export PGDATABASE=procrastinate PGHOST=localhost PGUSER=postgres
$ createdb && psql -v ON_ERROR_STOP=ON -f init.sql
```

## Installation for development

Cabbage officially is compatible with *``Python 3.6``* and above, using Postgres 10.
procrastinate officially is compatible with *``Python 3.6``* and above, using Postgres 10.

```console
$ pip install -r requirements.txt
Expand Down Expand Up @@ -53,12 +53,12 @@ With a running database, in the dev virtualenv:
### Launch a worker

```console
$ python -m cabbage_demo worker sums
$ python -m cabbage_demo worker products
$ python -m procrastinate_demo worker sums
$ python -m procrastinate_demo worker products
```

### Schedule some tasks

```console
$ python -m cabbage_demo client
$ python -m procrastinate_demo client
```
22 changes: 0 additions & 22 deletions cabbage/exceptions.py

This file was deleted.

6 changes: 0 additions & 6 deletions cabbage_demo/cabbage_app.py

This file was deleted.

2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

# -- Project information -----------------------------------------------------

project = "Cabbage"
project = "procrastinate"
copyright = "2019, Peopledoc"
author = "Peopledoc"

Expand Down
33 changes: 21 additions & 12 deletions docs/discussions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ Another nice thing is the ability to easily browse (and possibly edit) jobs in
the queue, because interacting with data in a standard database a lot easier
than implementing a new specific protocol (say, AMQP).

This makes the building of tools around cabbage quite easier.
This makes the building of tools around Procrastinate quite easier.

Finally, the idea that the core operations around tasks are handled by the
database itself using stored procedures eases the possibility of porting
cabbage to another language, while staying compatible with Python-based jobs.
Procrastinate to another language, while staying compatible with Python-based jobs.

.. _Postgres: https://www.postgresql.org/
.. _RabbitMQ: https://www.rabbitmq.com/
Expand All @@ -33,7 +33,7 @@ There are 14 standards...

https://xkcd.com/927/

We are aware that Cabbage is an addition to an already crowded market of
We are aware that Procrastinate is an addition to an already crowded market of
Python task queues, and the aim is not to replace them all, but to provide
an alternative that fits our need, where we could not find one we were
completely satifified with.
Expand All @@ -48,8 +48,8 @@ some projects that really stand out, to name a few:
- Dramatiq_ + dramatiq-pg_: Dramatiq is another very nice Python task queue
that does things quite well, and it happens that there is a third party
addition for using postgres as a backend. In fact, it was built around the
same time as we started cabbage, and the paradigm it uses makes it hard to
integrate a few of the feature we really wanted to use Cabbage for, namely
same time as we started Procrastinate, and the paradigm it uses makes it hard to
integrate a few of the feature we really wanted to use Procrastinate for, namely
locks.


Expand All @@ -58,25 +58,25 @@ some projects that really stand out, to name a few:
.. _dramatiq-pg: https://pypi.org/project/dramatiq-pg/


How stable is cabbage ?
-----------------------
How stable is Procrastinate ?
-----------------------------

Not quite stable. There a lot of things we would like to do before we start
advertising the project, and so far, it's not used anywhere.

We'd love if you were to try out cabbage in a non-production non-critical
We'd love if you were to try out Procrastinate in a non-production non-critical
project of yours and provide us with feedback.

Glossary
--------

Cabbage uses several words with a specific meaning in the documentation and
Procrastinate uses several words with a specific meaning in the documentation and
reference materials, as well as in the user code and it's own code.

Let's lay down a few words their meaning.

Asynchronous
In cabbage, most of the time asynchronoucity is metionned, it's not about
In Procrastinate, most of the time asynchronoucity is metionned, it's not about
Python's ``async/await`` features, but about the fact a job is launched
in the code, and then the calling code moves on, not waiting for the
completion of the job. Because of this, asynchronous tasks should have a
Expand Down Expand Up @@ -107,14 +107,23 @@ Worker
by one and executing them, and then wait for the queue to fill again.

App
This is meant to be the main entrypoint of Cabbage. The app knows
This is meant to be the main entrypoint of Procrastinate. The app knows
all the tasks of your project, and thanks to the job store, it knows how
to launch jobs to execute your tasks.

Job Store
The job store resposability is to store and retrieve jobs. In cabbage, the
The job store resposability is to store and retrieve jobs. In Procrastinate, the
job store will store your jobs in your ``postgres`` database.

Wasn't this project named "Cabbage" ?
-------------------------------------

Yes, in early development, we planned to call this "cabbage" in reference to
celery, but even if the name was available on PyPI, by the time we stopped
procrastinating and wanted to register it, it had been taken. Given this project
is all about "launching tasks in an undertermined moment in the future", the new
name felt quite adapted too. Also, now you know why the project is named this way.

Thanks PeopleDoc
----------------

Expand Down
34 changes: 17 additions & 17 deletions docs/howto.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,11 @@ We can solve this problem by using locks::
job_descritption.defer("d")

In this case, our jobs might still be executed by any of the workers,
but cabbage will not select a job for completion as long as there is
a job currently processing with the same lock. Note that cabbage will
but Procrastinate will not select a job for completion as long as there is
a job currently processing with the same lock. Note that Procrastinate will
use postgres to search the jobs table for suitable jobs, meaning that
even if the database contains a high proportion of locked tasks, it will barely
affect cabbages's capacity to quickly find the free tasks. Also, identical
affect Procrastinates's capacity to quickly find the free tasks. Also, identical
jobs will always be started in creation order, so we can be assured our
tasks will run sequentially and in order.

Expand All @@ -72,7 +72,7 @@ There is no mechanism in place to expire locks yet, but if a task fails
without the whole Python process crashing, it will free its lock.

Launch a job in the future
---------------------------
--------------------------

If a job is configured with a date in the future, it will run at the
first opportunity after that date. Let's launch the task that will
Expand All @@ -96,7 +96,7 @@ Define a retry strategy on a task

We sometimes know in advance that a task may fail randomly. For example a task
fetching resources on another network. You can define a retry strategy on a
task and cabbage will enforce it.
task and Procrastinate will enforce it.
Available strategies are:

- Define a number of attempts::
Expand All @@ -118,16 +118,16 @@ Available strategies are:

- You can get a more precise strategy using a RetryStrategy instance::

from cabbage import RetryStrategy
from procrastinate import RetryStrategy

@app.task(retry=cabbage.RetryStrategy(max_attempts=10, wait=5))
@app.task(retry=procrastinate.RetryStrategy(max_attempts=10, wait=5))
def my_other_task():
print("Hello world")

- If you want to go for a fully fledged custom retry strategy, you can implement your
own retry strategy::

class MyRetryStrategy(cabbage.BaseRetryStrategy):
class MyRetryStrategy(procrastinate.BaseRetryStrategy):
growth: Optional[str] = "linear"

def get_schedule_in(self, attempts: int) -> int:
Expand All @@ -146,7 +146,7 @@ app / machine reboots.
Add a task middleware
---------------------

As of today, Cabbage has no specific way of ensuring a piece of code runs
As of today, Procrastinate has no specific way of ensuring a piece of code runs
before or after every job. That being said, you can always decide to use
your own decorator instead of ``@app.task`` and have this decorator
implement the actions you need and delegate the rest to ``@app.task``.
Expand All @@ -164,16 +164,16 @@ It might look like this::

Then, define all of your tasks using this ``@task`` decorator.

Test your code that uses cabbage
--------------------------------
Test your code that uses Procrastinate
--------------------------------------

Cabbage defines an `InMemoryJobStore` that will speed-up your tests,
Procrastinate defines an `InMemoryJobStore` that will speed-up your tests,
remove dependency to Postgres and allow you to have tasks run in a
controlled way.

To use it, you can do::

app = cabbage.App(job_store=cabbage.testing.InMemoryJobStore())
app = procrastinate.App(job_store=procrastinate.testing.InMemoryJobStore())

# Run the jobs your tests created, then stop
# the worker:
Expand All @@ -186,13 +186,13 @@ To use it, you can do::
app.job_store.reset()


Deploy Cabbage in a real environment
------------------------------------
Deploy Procrastinate in a real environment
------------------------------------------

We haven't done that yet, no advice to give.

Monitor cabbage in a real environment
-------------------------------------
Monitor Procrastinate in a real environment
-------------------------------------------

We're in the process of writing an admin website and Rest API.
We'll update this section.
21 changes: 13 additions & 8 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
Cabbage: Postgresql-based Task Queue for Python
===============================================
Procrastinate: Postgresql-based Task Queue for Python
=========================================================

Cabbage is an open-source Python 3.6+ distributed task processing library,
leveraging postgresql to store task definitions, manage locks and
Procrastinate is an open-source Python 3.6+ distributed task processing
library, leveraging postgresql to store task definitions, manage locks and
dispatch tasks.

Here's an example::

# Make a app in your code
app = cabbage.App(job_store=cabbage.PostgresJobStore())
app = procrastinate.App(job_store=procrastinate.PostgresJobStore())

# Then define tasks
@app.task(queue="sums")
Expand All @@ -20,17 +20,22 @@ Here's an example::
sum.defer(a=3, b=5)

# Somewhere in your program, launch a worker
worker = cabbage.Worker(
worker = procrastinate.Worker(
app=app,
queues=["sums"]
)
worker.run()

There are quite a few interesting features that Cabbage adds to the mix.
There are quite a few interesting features that Procrastinate adds to the mix.
You can head to the Quickstart section for a general tour or
to the How-To sections for specific features. The Discussion
section should hopefully answer your questions. Otherwise,
feel free to open an `issue <https://github.com/peopledoc/cabbage/issues>`_.
feel free to open an `issue <https://github.com/peopledoc/procrastinate/issues>`_.

*Note to my future self: add a quick note here on why this project is named*
"Procrastinate_".

.. _Procrastinate: https://en.wikipedia.org/wiki/Procrastination

.. toctree::
:maxdepth: 2
Expand Down
12 changes: 6 additions & 6 deletions docs/reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,23 @@ API Reference
App
---

.. autoclass:: cabbage.App
.. autoclass:: procrastinate.App
:members: task, run_worker

Job stores
----------

.. autoclass:: cabbage.PostgresJobStore
.. autoclass:: procrastinate.PostgresJobStore

.. autoclass:: cabbage.testing.InMemoryJobStore
.. autoclass:: procrastinate.testing.InMemoryJobStore
:members: reset

Retry strategies
----------------

.. automodule:: cabbage.retry
.. automodule:: procrastinate.retry

.. autoclass:: cabbage.RetryStrategy
.. autoclass:: procrastinate.RetryStrategy

.. autoclass:: cabbage.BaseRetryStrategy
.. autoclass:: procrastinate.BaseRetryStrategy
:members: get_schedule_in
Loading

0 comments on commit 70078ab

Please sign in to comment.