On conflict clause / upserts (#816)

* initial commit

* fix tests

* version pin litestar

* use typing extensions for Literal

* make suggested change

* undo

* Update piccolo/query/mixins.py

Co-authored-by: sinisaos <[email protected]>

* Update piccolo/query/mixins.py

Co-authored-by: sinisaos <[email protected]>

* add one extra comma

* first attempt at docs

* add `NotImplementedError` for unsupported methods

* fix typo in sqlite version number

* fix tests

* get tests running for sqlite

* add test for `do nothing`

* add test for violating non target constraint

* remove old comment

* allow multiple on conflict clauses

* `target` -> `targets`

It accepts a list, so targets makes more sense.

* add docstring for `test_do_nothing`

* add tests for multiple ON CONFLICT clauses

* add docs for multiple ``on_conflict`` clauses

* add docs for using `all_columns`

* fix typo in test name

* add test for using `all_columns`

* add test for using an enum to specify the action

* add a test to make sure `DO UPDATE` with no values raises an exception

* rename `targets` back to `target`

* integrate @sinisaos tests

* move `on_conflict` to its own page

* refactor the `where` clause

---------

Co-authored-by: sinisaos <[email protected]>

docs/src/piccolo/query_clauses/as_of.rst

@@ -3,6 +3,8 @@
 as_of
 =====
 
+.. note:: Cockroach only.
+
 You can use ``as_of`` clause with the following queries:
 
 * :ref:`Select`
@@ -21,5 +23,3 @@ This generates an ``AS OF SYSTEM TIME`` clause. See `documentation <https://www.
 This clause accepts a wide variety of time and interval `string formats <https://www.cockroachlabs.com/docs/stable/as-of-system-time.html#using-different-timestamp-formats>`_.
 
 This is very useful for performance, as it will reduce transaction contention across a cluster.
-
-Currently only supported on Cockroach Engine. 

docs/src/piccolo/query_clauses/index.rst

@@ -26,6 +26,7 @@ by modifying the return values.
     ./freeze
     ./group_by
     ./offset
+    ./on_conflict
     ./output
     ./returning
 

docs/src/piccolo/query_clauses/on_conflict.rst

@@ -0,0 +1,229 @@
+.. _on_conflict:
+
+on_conflict
+===========
+
+.. hint:: This is an advanced topic, and first time learners of Piccolo
+    can skip if they want.
+
+You can use the ``on_conflict`` clause with the following queries:
+
+* :ref:`Insert`
+
+Introduction
+------------
+
+When inserting rows into a table, if a unique constraint fails on one or more
+of the rows, then the insertion fails.
+
+Using the ``on_conflict`` clause, we can instead tell the database to ignore
+the error (using ``DO NOTHING``), or to update the row (using ``DO UPDATE``).
+
+This is sometimes called an **upsert** (update if it already exists else insert).
+
+Example data
+------------
+
+If we have the following table:
+
+.. code-block:: python
+
+    class Band(Table):
+        name = Varchar(unique=True)
+        popularity = Integer()
+
+With this data:
+
+.. csv-table::
+    :file: ./on_conflict/bands.csv
+
+Let's try inserting another row with the same ``name``, and we'll get an error:
+
+.. code-block:: python
+
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... )
+    Unique constraint error!
+
+``DO NOTHING``
+--------------
+
+To ignore the error:
+
+.. code-block:: python
+
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO NOTHING"
+    ... )
+
+If we fetch the data from the database, we'll see that it hasn't changed:
+
+.. code-block:: python
+
+    >>> await Band.select().where(Band.name == "Pythonistas").first()
+    {'id': 1, 'name': 'Pythonistas', 'popularity': 1000}
+
+
+``DO UPDATE``
+-------------
+
+Instead, if we want to update the ``popularity``:
+
+.. code-block:: python
+
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO UPDATE",
+    ...     values=[Band.popularity]
+    ... )
+
+If we fetch the data from the database, we'll see that it was updated:
+
+.. code-block:: python
+
+    >>> await Band.select().where(Band.name == "Pythonistas").first()
+    {'id': 1, 'name': 'Pythonistas', 'popularity': 1200}
+
+``target``
+----------
+
+Using the ``target`` argument, we can specify which constraint we're concerned
+with. By specifying ``target=Band.name`` we're only concerned with the unique
+constraint for the ``band`` column. If you omit the ``target`` argument, then
+it works for all constraints on the table.
+
+.. code-block:: python
+    :emphasize-lines: 5
+
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO NOTHING",
+    ...     target=Band.name
+    ... )
+
+If you want to target a composite unique constraint, you can do so by passing
+in a tuple of columns:
+
+.. code-block:: python
+    :emphasize-lines: 5
+
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO NOTHING",
+    ...     target=(Band.name, Band.popularity)
+    ... )
+
+You can also specify the name of a constraint using a string:
+
+.. code-block:: python
+    :emphasize-lines: 5
+
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO NOTHING",
+    ...     target='some_constraint'
+    ... )
+
+``values``
+----------
+
+This lets us specify which values to update when a conflict occurs.
+
+By specifying a :class:`Column <piccolo.columns.base.Column>`, this means that
+the new value for that column will be used:
+
+.. code-block:: python
+    :emphasize-lines: 6
+
+    # The new popularity will be 1200.
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO UPDATE",
+    ...     values=[Band.popularity]
+    ... )
+
+Instead, we can specify a custom value using a tuple:
+
+.. code-block:: python
+    :emphasize-lines: 6
+
+    # The new popularity will be 1111.
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO UPDATE",
+    ...     values=[(Band.popularity, 1111)]
+    ... )
+
+If we want to update all of the values, we can use :meth:`all_columns<piccolo.table.Table.all_columns>`.
+
+.. code-block:: python
+    :emphasize-lines: 5
+
+    >>> await Band.insert(
+    ...     Band(id=1, name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO UPDATE",
+    ...     values=Band.all_columns()
+    ... )
+
+``where``
+---------
+
+This can be used with ``DO UPDATE``. It gives us more control over whether the
+update should be made:
+
+.. code-block:: python
+    :emphasize-lines: 6
+
+    >>> await Band.insert(
+    ...     Band(id=1, name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO UPDATE",
+    ...     values=[Band.popularity],
+    ...     where=Band.popularity < 1000
+    ... )
+
+Multiple ``on_conflict`` clauses
+--------------------------------
+
+SQLite allows you to specify multiple ``ON CONFLICT`` clauses, but Postgres and
+Cockroach don't.
+
+.. code-block:: python
+
+    >>> await Band.insert(
+    ...     Band(name="Pythonistas", popularity=1200)
+    ... ).on_conflict(
+    ...     action="DO UPDATE",
+    ...     ...
+    ... ).on_conflict(
+    ...     action="DO NOTHING",
+    ...     ...
+    ... )
+
+Learn more
+----------
+
+* `Postgres docs <https://www.postgresql.org/docs/current/sql-insert.html#SQL-ON-CONFLICT>`_
+* `Cockroach docs <https://www.cockroachlabs.com/docs/v2.0/insert.html#on-conflict-clause>`_
+* `SQLite docs <https://www.sqlite.org/lang_UPSERT.html>`_
+
+Source
+------
+
+.. currentmodule:: piccolo.query.methods.insert
+
+.. automethod:: Insert.on_conflict
+
+.. autoclass:: OnConflictAction
+    :members:
+    :undoc-members:

docs/src/piccolo/query_clauses/on_conflict/bands.csv

@@ -0,0 +1,2 @@
+id,name,popularity
+1,Pythonistas,1000

docs/src/piccolo/query_types/insert.rst

@@ -3,33 +3,47 @@
 Insert
 ======
 
-This is used to insert rows into the table.
-
-.. code-block:: python
-
-    >>> await Band.insert(Band(name="Pythonistas"))
-    [{'id': 3}]
-
-We can insert multiple rows in one go:
+This is used to bulk insert rows into the table:
 
 .. code-block:: python
 
     await Band.insert(
+        Band(name="Pythonistas")
         Band(name="Darts"),
         Band(name="Gophers")
     )
 
 -------------------------------------------------------------------------------
 
-add
----
+``add``
+-------
 
-You can also compose it as follows:
+If we later decide to insert additional rows, we can use the ``add`` method:
 
 .. code-block:: python
 
-    await Band.insert().add(
-        Band(name="Darts")
-    ).add(
-        Band(name="Gophers")
-    )
+    query = Band.insert(Band(name="Pythonistas"))
+
+    if other_bands:
+        query = query.add(
+            Band(name="Darts"),
+            Band(name="Gophers")
+        )
+
+    await query
+
+-------------------------------------------------------------------------------
+
+Query clauses
+-------------
+
+on_conflict
+~~~~~~~~~~~
+
+See :ref:`On_Conflict`.
+
+
+returning
+~~~~~~~~~
+
+See :ref:`Returning`.

piccolo/apps/asgi/commands/new.py

@@ -12,7 +12,7 @@
 SERVERS = ["uvicorn", "Hypercorn"]
 ROUTERS = ["starlette", "fastapi", "blacksheep", "litestar"]
 ROUTER_DEPENDENCIES = {
-    "litestar": ["litestar>=2.0.0a3"],
+    "litestar": ["litestar==2.0.0a3"],
 }