Code

Custom database backend#164

Code

Merged

cswinter merged 53 commits intomastercswinter/LocustDB:masterfrom geistesblitzcswinter/LocustDB:geistesblitzCopy head branch name to clipboardMar 20, 2024

+5,971-1,913

Commits

Commits on Mar 10, 2024

• Float multiplication

  cswinter

  committed
  dc6982b

  Copy full SHA for dc6982b

• PyO3 -> 0.19

  cswinter

  committed
  e66fc59

  Copy full SHA for e66fc59

• SQLParser upgrade, unquoting

  cswinter

  committed
  54bc7a5

  Copy full SHA for 54bc7a5

• cors

  cswinter

  committed
  3410ab4

  Copy full SHA for 3410ab4

• Ingestion improvements, multi query

  cswinter

  committed
  8a9b113

  Copy full SHA for 8a9b113

• Storage v2 prototype

  cswinter

  committed
  59db069

  Copy full SHA for 59db069

• WAL compaction

  cswinter

  committed
  343109d

  Copy full SHA for 343109d

• Add missing disk_store/v2.rs file

  cswinter

  committed
  5ba5111

  Copy full SHA for 5ba5111

• WandB data import

  cswinter

  committed
  99ebedb

  Copy full SHA for 99ebedb

• db inspection tool

  cswinter

  committed
  2419550

  Copy full SHA for 2419550

• Implement partition loading for dbv2

  cswinter

  committed
  d96816c

  Copy full SHA for d96816c

• Rip out RocksDB backend

  cswinter

  committed
  4654196

  Copy full SHA for 4654196

• Multiple columns in single partition file

  cswinter

  committed
  36f7a69

  Copy full SHA for 36f7a69

• db_bench command

  cswinter

  committed
  4e5bdf3

  Copy full SHA for 4e5bdf3

• add db_bench results

  cswinter

  committed
  378f1bb

  Copy full SHA for 378f1bb

• large table bench, faster rng

  cswinter

  committed
  dfa1f3f

  Copy full SHA for dfa1f3f

• more efficient logging

  cswinter

  committed
  01ac854

  Copy full SHA for 01ac854

• logging client optimizations

  cswinter

  committed
  1eee24e

  Copy full SHA for 1eee24e

• don't drop events

  cswinter

  committed
  f237e71

  Copy full SHA for f237e71

• columnar ingestion

  cswinter

  committed
  3cc1667

  Copy full SHA for 3cc1667

• fix db_inspector

  cswinter

  committed
  9258aee

  Copy full SHA for 9258aee

• add queries to db_bench, measure query disk read

  cswinter

  committed
  9acb54d

  Copy full SHA for 9acb54d

• When loading joint partition, populate cache for all columns

  cswinter

  committed
  0029237

  Copy full SHA for 0029237

• Fix subpartition key collision

  cswinter

  committed
  5292afd

  Copy full SHA for 5292afd

• fix lru cache, new bench options

  cswinter

  committed
  e6ee3f4

  Copy full SHA for e6ee3f4

• Compaction

  cswinter

  committed
  ff2228f

  Copy full SHA for ff2228f

• fix sparse column ingestion

  cswinter

  committed
  08b93c6

  Copy full SHA for 08b93c6

• Remove RocksDB remnants, cleanup

  cswinter

  committed
  454916c

  Copy full SHA for 454916c

• fix parser tests

  cswinter

  committed
  7142b72

  Copy full SHA for 7142b72

• fix tests

  cswinter

  committed
  6641baa

  Copy full SHA for 6641baa

• ensure filesystem safe column names

  cswinter

  committed
  37dae96

  Copy full SHA for 37dae96

Commits on Mar 12, 2024

• bugfixes

  cswinter

  committed
  0d0e1ce

  Copy full SHA for 0d0e1ce

Commits on Mar 13, 2024

• Add CORS options

  cswinter

  committed
  0fd6583

  Copy full SHA for 0fd6583

• Add option for address to bind server to

  cswinter

  committed
  a5bf241

  Copy full SHA for a5bf241

• don't print all tables/columns on startup

  cswinter

  committed
  22086ff

  Copy full SHA for 22086ff

• Fix read of bitvec data sections

  cswinter

  committed
  4d55981

  Copy full SHA for 4d55981

• remove debug logging

  cswinter

  committed
  e67a5da

  Copy full SHA for e67a5da

• fix bug caused by streaming stage with nonstreaming input

  cswinter

  committed
  225aaee

  Copy full SHA for 225aaee

Commits on Mar 15, 2024

• fix order by for null/nonexistant column

  cswinter

  committed
  c9d7b95

  Copy full SHA for c9d7b95

Commits on Mar 16, 2024

• fix f64 lz4 decode

  cswinter

  committed
  f03dbba

  Copy full SHA for f03dbba

• Test with small batch size and fix streaming bug

  cswinter

  committed
  33b84db

  Copy full SHA for 33b84db

• Add default for batch size arg

  cswinter

  committed
  8df626b

  Copy full SHA for 8df626b

Commits on Mar 18, 2024

• Rework streaming execution

  cswinter

  committed
  7f72802

  Copy full SHA for 7f72802

• columns endpoint

  cswinter

  committed
  8277536

  Copy full SHA for 8277536

• limit and offset for columns endpoint

  cswinter

  committed
  cdf14ff

  Copy full SHA for cdf14ff

Commits on Mar 19, 2024

• preserve row ordering during compaction, tons of bugfixes

  cswinter

  committed
  565863b

  Copy full SHA for 565863b

• fix limit select of null column

  cswinter

  committed
  de16cd5

  Copy full SHA for de16cd5

Commits on Mar 20, 2024

• ingestion test and storage fixes

  cswinter

  committed
  a7982a5

  Copy full SHA for a7982a5

• actualy add ingestion test, fixes

  cswinter

  committed
  7c6ffab

  Copy full SHA for 7c6ffab

• Simplify meta db writing

  cswinter

  committed
  66a4be8

  Copy full SHA for 66a4be8

• delete dead code

  cswinter

  committed
  bd298ea

  Copy full SHA for bd298ea

• eliminate query_col inefficiency

  cswinter

  committed
  9c03ca8

  Copy full SHA for 9c03ca8

• more efficient metastore datastructure

  cswinter

  committed
  132f1df

  Copy full SHA for 132f1df