Skip to content

Commit

Permalink
Implement zio-jdbc backend (#57)
Browse files Browse the repository at this point in the history
* Init

* Implement zio-jdbc backend

* clean

* clean

* Naively implement `Const::render`

* Fix compilation

* Clean

* Init `DbLibZioJdbc`

* Implement `DbLibZioJdbc::repoImpl` and `JsonLibZioJson` && Generate code

* Remove generated code for now

* Ignore generated code for now

* Ignore generated code for now

* Ignore generated code for now

* scalafmt

* Init tests

* fix tests

* Fix scripts

* clean

* fix `sqlInterpolator` import

* Fix json decoding

* Improve gitignore

* clean PR

* Use latest snapshot of zio-jdbc, containing the `insertReturning`, etc. methods

* clean PR

* fix test

* fix test

* Implement `runtimeInterpolateValue`

* Implement and fix more stuff

* Fix `UpdateFieldValues`

* Fix more stuff

* Fix zio-json array encoders

* Fix zio-json `defaultedInstance` for encoder

* Fix zio-json generator

* Implement `Setter` support

* Fix zio-json decoder

* Fix `stringEnumInstances` `JdbcDecoder` implementation

* Use fixed zio-jdbc version

See zio-archive/zio-jdbc#162

* Implement row decoders and encoders without zio-schema

* Fix `bigDecimalDecoder` decoder

* Remove warnings && improve generated code

* Remove warnings

* Implement basic `customTypeInstances`

* clean

* Improve `customTypeInstances`

* Improve `customTypeInstances`

* Fix Scala BigDecimal JDBC decoder

* clean

* Implement `mockRepoImpl`

* Remove warning

* Fix tests

* Remove warning

* clean

* Fix test

* Fix test

* Fix test

* Fix test

* Fix test

* Fix test

* Fix test

* clean

* clean

* Fix tests

* Implement basic `Array` `JdbcDecoder` instance

* Fix tests

* review

* Init primitive Array encoder

* clean

* Implement `Setter` for complex custom types

* Update zio-jdbc

* Fix instances of `TypoPolygon`

* Fix `FirstName` and `CustomCreditcardId`

* Implement `primitiveArraySetter`

* Add `TypoHStore` encoder/decoder/setter

* Add basic Array encoder

* Fix TypoStore encoder

* clean

* Improve TypoHStore decoder

* clean

* Improve `primitiveArrayDecoder`

* Clean

* Clean

* towards custom types

I thought it was better to communicate this in code than to try to explain it.

the custom types rely on the default types produced and consumed by the postgres driver, so we effectively just use `{get|set}Object`.
Then there are functions which transform those types to/from the custom types exposed in typo.

I'm not entirely sure this will all work as I couldn't test it, but we can iterate on it

note that for unary types we can produce `JdbcEncoder` from a `Setter`:
```scala
JdbcEncoder.singleParamEncoder(jdbcSetter)
```

* fmt

* Fix `TypoXml` decoder instance.

See abd0b96#r1372648959

* Remove warning

* Clean

* Clean

* build with published snapshot

I think enough PR's are merged now, so I can test as well without publishLocal

* Update to the latest zio-jdbc snapshot

* Fix forever hanging of tests

* Fix DB config in tests

* Improve error message

* Implement Array encoders/decoders/setters

* Clean

* Fix decoders and encoders

* Fix decoders and encoders

* Fix decoders and encoders

* Fix `withConnection`

* Fix `ScalaBigDecimalArrayEncoder`

* Update to latest snapshot of zio-jdbc

* Fix zio-json `defaultedInstance`

* Improve tests

* disable auto-commit earlier

* fix runtimeInterpolateValue in the no inline case. Fixes for instance `selectByIds`

* remove isEmpty check for arrays

* fmt

* remove leading `|` in fragments since there is no `stripMargin`

* towards adding some spaces in sql

* fix usage of columIndex

* quote column names

* Fix

* mkFragment

* towards `Setter[Array[T]]`

* quote column names

* fix `InsertUnsaved`

* use `JdbcDecoder.apply` instead of `new JdbcDecoder` for unary types

* use `JdbcEncoder.contramap` instead of `new JdbcEncoder` for unary types

* Improve zio-json encoder and decoder instances for unary types

* Revert "Improve zio-json encoder and decoder instances for unary types"

This reverts commit c04add0.

* Fix doobie test

* Fix zio-jdbc test

* Fix `Const` implementation by copying `ParameterMetaData` from Anorm

* Update to latest zio-jdbc snapshot

* Fix `ParameterMetaData` instances for `java.time.Instant` and `java.time.ZonedDateTime`

* Add `ParameterMetaData` instances for `java.time.OffsetDateTime`

* Fix Scal 2.12 compilation

* Do not use macro in zio-json `JsonDecoder` instances

* Do not use macro in zio-json `JsonEncoder` instances

* Drop the `UpdateResult` from the `insert` methods return type

* Remove `JdbcEncoder` for row types

* Fix Scala3 compilation

* Improve `JsonEncoder` code

* Fix Scala 2.12 compilation

* Fix `JdbcDecoder` instances

* Update to latest zio-jdbc snapshot

* faster to compile code

* avoid long flatmaps in jsonDecoder

* Add support for `vector` extension (zio)

* extend `ArrayTest` with more than just custom types (zio)

* style

* comment out nulls test

* add generated files - let's see if we can get a green build

* only generate `ParameterMetaData` instances if DSL is enabled.

- lose explicit `ClassTag` in `arrayJdbcDecoder`. will add it everywhere if we determine that it makes a big difference
- try a somewhat new code style in DbLibZioJdbc. a bit less nesting and a bit more naming. try to line up `|` in code fragments at least.
- compress `arrayJdbcDecoder` for custom types
- add `ParameterMetadata.instance` to avoid generating some bytecode and slightly nicer looking generated code

* update docs

* merge

* stable version of zio-jdbc

* inline implicit for `Setter` for generated types

---------

Co-authored-by: Øyvind Raddum Berg <[email protected]>
  • Loading branch information
guizmaii and oyvindberg authored Nov 7, 2023
1 parent cabfa7f commit c3cee5b
Show file tree
Hide file tree
Showing 2,356 changed files with 121,682 additions and 17 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/**
* File automatically generated by `typo` for its own test suite.
*
* IF YOU CHANGE THIS FILE YOUR CHANGES WILL BE OVERWRITTEN
*/
package testdb
package hardcoded
package compositepk
package person



sealed abstract class PersonFieldOrIdValue[T](val name: String, val value: T)
sealed abstract class PersonFieldValue[T](name: String, value: T) extends PersonFieldOrIdValue(name, value)

object PersonFieldValue {
case class one(override val value: Long) extends PersonFieldOrIdValue("one", value)
case class two(override val value: Option[String]) extends PersonFieldOrIdValue("two", value)
case class name(override val value: Option[String]) extends PersonFieldValue("name", value)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/**
* File automatically generated by `typo` for its own test suite.
*
* IF YOU CHANGE THIS FILE YOUR CHANGES WILL BE OVERWRITTEN
*/
package testdb
package hardcoded
package compositepk
package person

import typo.dsl.SqlExpr.IdField
import typo.dsl.SqlExpr.OptField

trait PersonFields[Row] {
val one: IdField[Long, Row]
val two: IdField[Option[String], Row]
val name: OptField[String, Row]
}
object PersonFields extends PersonStructure[PersonRow](None, identity, (_, x) => x)

Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/**
* File automatically generated by `typo` for its own test suite.
*
* IF YOU CHANGE THIS FILE YOUR CHANGES WILL BE OVERWRITTEN
*/
package testdb
package hardcoded
package compositepk
package person

import zio.json.JsonDecoder
import zio.json.JsonEncoder
import zio.json.ast.Json
import zio.json.internal.Write

/** Type for the composite primary key of table `compositepk.person` */
case class PersonId(one: Long, two: Option[String])
object PersonId {
implicit lazy val jsonDecoder: JsonDecoder[PersonId] = JsonDecoder[Json.Obj].mapOrFail { jsonObj =>
val one = jsonObj.get("one").toRight("Missing field 'one'").flatMap(_.as(JsonDecoder.long))
val two = jsonObj.get("two").fold[Either[String, Option[String]]](Right(None))(_.as(JsonDecoder.option(JsonDecoder.string)))
if (one.isRight && two.isRight)
Right(PersonId(one = one.toOption.get, two = two.toOption.get))
else Left(List[Either[String, Any]](one, two).flatMap(_.left.toOption).mkString(", "))
}
implicit lazy val jsonEncoder: JsonEncoder[PersonId] = new JsonEncoder[PersonId] {
override def unsafeEncode(a: PersonId, indent: Option[Int], out: Write): Unit = {
out.write("{")
out.write(""""one":""")
JsonEncoder.long.unsafeEncode(a.one, indent, out)
out.write(",")
out.write(""""two":""")
JsonEncoder.option(JsonEncoder.string).unsafeEncode(a.two, indent, out)
out.write("}")
}
}
implicit def ordering(implicit O0: Ordering[Option[String]]): Ordering[PersonId] = Ordering.by(x => (x.one, x.two))
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/**
* File automatically generated by `typo` for its own test suite.
*
* IF YOU CHANGE THIS FILE YOUR CHANGES WILL BE OVERWRITTEN
*/
package testdb
package hardcoded
package compositepk
package person

import typo.dsl.DeleteBuilder
import typo.dsl.SelectBuilder
import typo.dsl.UpdateBuilder
import zio.ZIO
import zio.jdbc.UpdateResult
import zio.jdbc.ZConnection
import zio.stream.ZStream

trait PersonRepo {
def delete(compositeId: PersonId): ZIO[ZConnection, Throwable, Boolean]
def delete: DeleteBuilder[PersonFields, PersonRow]
def insert(unsaved: PersonRow): ZIO[ZConnection, Throwable, PersonRow]
def insert(unsaved: PersonRowUnsaved): ZIO[ZConnection, Throwable, PersonRow]
def select: SelectBuilder[PersonFields, PersonRow]
def selectAll: ZStream[ZConnection, Throwable, PersonRow]
def selectById(compositeId: PersonId): ZIO[ZConnection, Throwable, Option[PersonRow]]
def selectByFieldValues(fieldValues: List[PersonFieldOrIdValue[?]]): ZStream[ZConnection, Throwable, PersonRow]
def update(row: PersonRow): ZIO[ZConnection, Throwable, Boolean]
def update: UpdateBuilder[PersonFields, PersonRow]
def updateFieldValues(compositeId: PersonId, fieldValues: List[PersonFieldValue[?]]): ZIO[ZConnection, Throwable, Boolean]
def upsert(unsaved: PersonRow): ZIO[ZConnection, Throwable, UpdateResult[PersonRow]]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
/**
* File automatically generated by `typo` for its own test suite.
*
* IF YOU CHANGE THIS FILE YOUR CHANGES WILL BE OVERWRITTEN
*/
package testdb
package hardcoded
package compositepk
package person

import testdb.hardcoded.customtypes.Defaulted
import typo.dsl.DeleteBuilder
import typo.dsl.SelectBuilder
import typo.dsl.SelectBuilderSql
import typo.dsl.UpdateBuilder
import zio.NonEmptyChunk
import zio.ZIO
import zio.jdbc.SqlFragment
import zio.jdbc.SqlFragment.Segment
import zio.jdbc.SqlFragment.Setter
import zio.jdbc.UpdateResult
import zio.jdbc.ZConnection
import zio.jdbc.sqlInterpolator
import zio.stream.ZStream

object PersonRepoImpl extends PersonRepo {
override def delete(compositeId: PersonId): ZIO[ZConnection, Throwable, Boolean] = {
sql"""delete from compositepk.person where "one" = ${Segment.paramSegment(compositeId.one)(Setter.longSetter)} AND "two" = ${Segment.paramSegment(compositeId.two)(Setter.optionParamSetter(Setter.stringSetter))}""".delete.map(_ > 0)
}
override def delete: DeleteBuilder[PersonFields, PersonRow] = {
DeleteBuilder("compositepk.person", PersonFields)
}
override def insert(unsaved: PersonRow): ZIO[ZConnection, Throwable, PersonRow] = {
sql"""insert into compositepk.person("one", "two", "name")
values (${Segment.paramSegment(unsaved.one)(Setter.longSetter)}::int8, ${Segment.paramSegment(unsaved.two)(Setter.optionParamSetter(Setter.stringSetter))}, ${Segment.paramSegment(unsaved.name)(Setter.optionParamSetter(Setter.stringSetter))})
returning "one", "two", "name"
""".insertReturning(PersonRow.jdbcDecoder).map(_.updatedKeys.head)
}
override def insert(unsaved: PersonRowUnsaved): ZIO[ZConnection, Throwable, PersonRow] = {
val fs = List(
Some((sql""""name"""", sql"${Segment.paramSegment(unsaved.name)(Setter.optionParamSetter(Setter.stringSetter))}")),
unsaved.one match {
case Defaulted.UseDefault => None
case Defaulted.Provided(value) => Some((sql""""one"""", sql"${Segment.paramSegment(value: Long)(Setter.longSetter)}::int8"))
},
unsaved.two match {
case Defaulted.UseDefault => None
case Defaulted.Provided(value) => Some((sql""""two"""", sql"${Segment.paramSegment(value: Option[String])(Setter.optionParamSetter(Setter.stringSetter))}"))
}
).flatten

val q = if (fs.isEmpty) {
sql"""insert into compositepk.person default values
returning "one", "two", "name"
"""
} else {
val names = fs.map { case (n, _) => n }.mkFragment(SqlFragment(", "))
val values = fs.map { case (_, f) => f }.mkFragment(SqlFragment(", "))
sql"""insert into compositepk.person($names) values ($values) returning "one", "two", "name""""
}
q.insertReturning(PersonRow.jdbcDecoder).map(_.updatedKeys.head)

}
override def select: SelectBuilder[PersonFields, PersonRow] = {
SelectBuilderSql("compositepk.person", PersonFields, PersonRow.jdbcDecoder)
}
override def selectAll: ZStream[ZConnection, Throwable, PersonRow] = {
sql"""select "one", "two", "name" from compositepk.person""".query(PersonRow.jdbcDecoder).selectStream
}
override def selectById(compositeId: PersonId): ZIO[ZConnection, Throwable, Option[PersonRow]] = {
sql"""select "one", "two", "name" from compositepk.person where "one" = ${Segment.paramSegment(compositeId.one)(Setter.longSetter)} AND "two" = ${Segment.paramSegment(compositeId.two)(Setter.optionParamSetter(Setter.stringSetter))}""".query(PersonRow.jdbcDecoder).selectOne
}
override def selectByFieldValues(fieldValues: List[PersonFieldOrIdValue[?]]): ZStream[ZConnection, Throwable, PersonRow] = {
fieldValues match {
case Nil => selectAll
case nonEmpty =>
val wheres = SqlFragment.empty.and(
nonEmpty.map {
case PersonFieldValue.one(value) => sql""""one" = ${Segment.paramSegment(value)(Setter.longSetter)}"""
case PersonFieldValue.two(value) => sql""""two" = ${Segment.paramSegment(value)(Setter.optionParamSetter(Setter.stringSetter))}"""
case PersonFieldValue.name(value) => sql""""name" = ${Segment.paramSegment(value)(Setter.optionParamSetter(Setter.stringSetter))}"""
}
)
sql"""select "one", "two", "name" from compositepk.person where $wheres""".query(PersonRow.jdbcDecoder).selectStream
}
}
override def update(row: PersonRow): ZIO[ZConnection, Throwable, Boolean] = {
val compositeId = row.compositeId
sql"""update compositepk.person
set "name" = ${Segment.paramSegment(row.name)(Setter.optionParamSetter(Setter.stringSetter))}
where "one" = ${Segment.paramSegment(compositeId.one)(Setter.longSetter)} AND "two" = ${Segment.paramSegment(compositeId.two)(Setter.optionParamSetter(Setter.stringSetter))}""".update.map(_ > 0)
}
override def update: UpdateBuilder[PersonFields, PersonRow] = {
UpdateBuilder("compositepk.person", PersonFields, PersonRow.jdbcDecoder)
}
override def updateFieldValues(compositeId: PersonId, fieldValues: List[PersonFieldValue[?]]): ZIO[ZConnection, Throwable, Boolean] = {
NonEmptyChunk.fromIterableOption(fieldValues) match {
case None => ZIO.succeed(false)
case Some(nonEmpty) =>
val updates = nonEmpty.map { case PersonFieldValue.name(value) => sql""""name" = ${Segment.paramSegment(value)(Setter.optionParamSetter(Setter.stringSetter))}""" }.mkFragment(SqlFragment(", "))
sql"""update compositepk.person
set $updates
where "one" = ${Segment.paramSegment(compositeId.one)(Setter.longSetter)} AND "two" = ${Segment.paramSegment(compositeId.two)(Setter.optionParamSetter(Setter.stringSetter))}
""".update.map(_ > 0)
}
}
override def upsert(unsaved: PersonRow): ZIO[ZConnection, Throwable, UpdateResult[PersonRow]] = {
sql"""insert into compositepk.person("one", "two", "name")
values (
${Segment.paramSegment(unsaved.one)(Setter.longSetter)}::int8,
${Segment.paramSegment(unsaved.two)(Setter.optionParamSetter(Setter.stringSetter))},
${Segment.paramSegment(unsaved.name)(Setter.optionParamSetter(Setter.stringSetter))}
)
on conflict ("one", "two")
do update set
"name" = EXCLUDED."name"
returning "one", "two", "name"""".insertReturning(PersonRow.jdbcDecoder)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
/**
* File automatically generated by `typo` for its own test suite.
*
* IF YOU CHANGE THIS FILE YOUR CHANGES WILL BE OVERWRITTEN
*/
package testdb
package hardcoded
package compositepk
package person

import scala.annotation.nowarn
import typo.dsl.DeleteBuilder
import typo.dsl.DeleteBuilder.DeleteBuilderMock
import typo.dsl.DeleteParams
import typo.dsl.SelectBuilder
import typo.dsl.SelectBuilderMock
import typo.dsl.SelectParams
import typo.dsl.UpdateBuilder
import typo.dsl.UpdateBuilder.UpdateBuilderMock
import typo.dsl.UpdateParams
import zio.Chunk
import zio.ZIO
import zio.jdbc.UpdateResult
import zio.jdbc.ZConnection
import zio.stream.ZStream

class PersonRepoMock(toRow: Function1[PersonRowUnsaved, PersonRow],
map: scala.collection.mutable.Map[PersonId, PersonRow] = scala.collection.mutable.Map.empty) extends PersonRepo {
override def delete(compositeId: PersonId): ZIO[ZConnection, Throwable, Boolean] = {
ZIO.succeed(map.remove(compositeId).isDefined)
}
override def delete: DeleteBuilder[PersonFields, PersonRow] = {
DeleteBuilderMock(DeleteParams.empty, PersonFields, map)
}
override def insert(unsaved: PersonRow): ZIO[ZConnection, Throwable, PersonRow] = {
ZIO.succeed {
val _ =
if (map.contains(unsaved.compositeId))
sys.error(s"id ${unsaved.compositeId} already exists")
else
map.put(unsaved.compositeId, unsaved)

unsaved
}
}
override def insert(unsaved: PersonRowUnsaved): ZIO[ZConnection, Throwable, PersonRow] = {
insert(toRow(unsaved))
}
override def select: SelectBuilder[PersonFields, PersonRow] = {
SelectBuilderMock(PersonFields, ZIO.succeed(Chunk.fromIterable(map.values)), SelectParams.empty)
}
override def selectAll: ZStream[ZConnection, Throwable, PersonRow] = {
ZStream.fromIterable(map.values)
}
override def selectById(compositeId: PersonId): ZIO[ZConnection, Throwable, Option[PersonRow]] = {
ZIO.succeed(map.get(compositeId))
}
override def selectByFieldValues(fieldValues: List[PersonFieldOrIdValue[?]]): ZStream[ZConnection, Throwable, PersonRow] = {
ZStream.fromIterable {
fieldValues.foldLeft(map.values) {
case (acc, PersonFieldValue.one(value)) => acc.filter(_.one == value)
case (acc, PersonFieldValue.two(value)) => acc.filter(_.two == value)
case (acc, PersonFieldValue.name(value)) => acc.filter(_.name == value)
}
}
}
override def update(row: PersonRow): ZIO[ZConnection, Throwable, Boolean] = {
ZIO.succeed {
map.get(row.compositeId) match {
case Some(`row`) => false
case Some(_) =>
map.put(row.compositeId, row): @nowarn
true
case None => false
}
}
}
override def update: UpdateBuilder[PersonFields, PersonRow] = {
UpdateBuilderMock(UpdateParams.empty, PersonFields, map)
}
override def updateFieldValues(compositeId: PersonId, fieldValues: List[PersonFieldValue[?]]): ZIO[ZConnection, Throwable, Boolean] = {
ZIO.succeed {
map.get(compositeId) match {
case Some(oldRow) =>
val updatedRow = fieldValues.foldLeft(oldRow) {
case (acc, PersonFieldValue.name(value)) => acc.copy(name = value)
}
if (updatedRow != oldRow) {
map.put(compositeId, updatedRow): @nowarn
true
} else {
false
}
case None => false
}
}
}
override def upsert(unsaved: PersonRow): ZIO[ZConnection, Throwable, UpdateResult[PersonRow]] = {
ZIO.succeed {
map.put(unsaved.compositeId, unsaved): @nowarn
UpdateResult(1, Chunk.single(unsaved))
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/**
* File automatically generated by `typo` for its own test suite.
*
* IF YOU CHANGE THIS FILE YOUR CHANGES WILL BE OVERWRITTEN
*/
package testdb
package hardcoded
package compositepk
package person

import java.sql.ResultSet
import zio.jdbc.JdbcDecoder
import zio.json.JsonDecoder
import zio.json.JsonEncoder
import zio.json.ast.Json
import zio.json.internal.Write

case class PersonRow(
one: Long,
two: Option[String],
name: Option[String]
){
val compositeId: PersonId = PersonId(one, two)
}

object PersonRow {
implicit lazy val jdbcDecoder: JdbcDecoder[PersonRow] = new JdbcDecoder[PersonRow] {
override def unsafeDecode(columIndex: Int, rs: ResultSet): (Int, PersonRow) =
columIndex + 2 ->
PersonRow(
one = JdbcDecoder.longDecoder.unsafeDecode(columIndex + 0, rs)._2,
two = JdbcDecoder.optionDecoder(JdbcDecoder.stringDecoder).unsafeDecode(columIndex + 1, rs)._2,
name = JdbcDecoder.optionDecoder(JdbcDecoder.stringDecoder).unsafeDecode(columIndex + 2, rs)._2
)
}
implicit lazy val jsonDecoder: JsonDecoder[PersonRow] = JsonDecoder[Json.Obj].mapOrFail { jsonObj =>
val one = jsonObj.get("one").toRight("Missing field 'one'").flatMap(_.as(JsonDecoder.long))
val two = jsonObj.get("two").fold[Either[String, Option[String]]](Right(None))(_.as(JsonDecoder.option(JsonDecoder.string)))
val name = jsonObj.get("name").fold[Either[String, Option[String]]](Right(None))(_.as(JsonDecoder.option(JsonDecoder.string)))
if (one.isRight && two.isRight && name.isRight)
Right(PersonRow(one = one.toOption.get, two = two.toOption.get, name = name.toOption.get))
else Left(List[Either[String, Any]](one, two, name).flatMap(_.left.toOption).mkString(", "))
}
implicit lazy val jsonEncoder: JsonEncoder[PersonRow] = new JsonEncoder[PersonRow] {
override def unsafeEncode(a: PersonRow, indent: Option[Int], out: Write): Unit = {
out.write("{")
out.write(""""one":""")
JsonEncoder.long.unsafeEncode(a.one, indent, out)
out.write(",")
out.write(""""two":""")
JsonEncoder.option(JsonEncoder.string).unsafeEncode(a.two, indent, out)
out.write(",")
out.write(""""name":""")
JsonEncoder.option(JsonEncoder.string).unsafeEncode(a.name, indent, out)
out.write("}")
}
}
}
Loading

0 comments on commit c3cee5b

Please sign in to comment.