v0.2.1

Highlights

• Python bindings:

  • You can now query intermediate results, simply referencing a variable:

    intermediate = con.sql("select * from tbl where a > 2;")
    con.sql("select * from intermediate where b > 4;")

  • show() was added to print results without other libraries

• SQL commands:

  • COPY TO:
    • Add support for remote URLs, including GCS and S3. For more information, see #1280
    • Tables can now be copied with COPY TO (instead of only queries). For more information see #1280

• SQL functions:

  • All functions received support for passing in identifiers and named arguments. For more information, see #1314
  • csv_scan:
    • Add support for GCS and S3 with credentials
  • ndjson_scan:
    • Add support for GCS and S3 with credentials
  • parquet_scan:
    • Add support for GCS and S3 with credentials
  • generate_series:
    • Added support for floats

• Recipes:

  • We've added a new examples directory. Our first example demonstrates how to connect to glaredb with R and query a Snowflake database. Check it out here: https://github.com/GlareDB/glaredb/blob/main/examples/R/out-of-process.R
  • We added an example of using our Python bindings with OpenAI! See it here: https://github.com/GlareDB/glaredb/blob/main/py-glaredb/examples/openai_ex.py

What's Changed

• chore: add semicolon to sql snippet by @greyscaled in #1299
• chore: Add example R script for connecting to glaredb by @scsmithr in #1296
• feat: composable plans by @universalmind303 in #1300
• feat: Support URLs for COPY .. TO ... by @vrongmeal in #1280
• feat: Add support for float in generate_series by @adhish20 in #1298
• chore: move builtins.rs to sqlbuiltins by @adhish20 in #1302
• chore: Add Dependabot configuration for Cargo by @0xf333 in #1301
• fix csv_scan for object_stores by @universalmind303 in #1303
• use pr-target instead of pr by @universalmind303 in #1305
• chore: Add example for running openai on data from postgres by @scsmithr in #1309
• feat: Support object store in *_scan functions. by @vrongmeal in #1306
• fix: stop parsing datasource and tunnel limits from cloud response by @f0ssel in #1313
• chore: move "client" stuff to metastoreproto by @adhish20 in #1311
• fix: Fix not being able to read huggingface parquets by @scsmithr in #1319
• feat: Support named arguments and idents in table functions. by @vrongmeal in #1314
• fix: Explicitly use Ident in table function parameters. by @vrongmeal in #1322
• fix: Add xtask alias back (for zip) by @vrongmeal in #1325
• chore: Update deltalake to da1012. by @vrongmeal in #1326
• feat: adding show() method to python execution results by @adhish20 in #1315
• chore: Use SchemaReference from datafusion instead of our own. by @vrongmeal in #1327
• chore: simplify feat template by @greyscaled in #1328
• build: Bump version to 0.2.1 by @greyscaled in #1332

New Contributors

• @0xf333 made their first contribution in #1301

Full Changelog: v0.2.0...v0.2.1

v0.2.0

Highlights

• python bindings:

  • GlareDB can now be used in Python with Pandas, Polars and external data sources. Install the module from PyPI with

  pip install glaredb

  The instance of glaredb is currently limited to in-memory. We're working on connecting to remote instances of glaredb for a future release. For more information, we have examples at https://github.com/GlareDB/glaredb/tree/main/py-glaredb/examples.

• COPY TO:

  • With COPY TO, a local copy of a query can be persisted. We're adding support for remote persistence in the future. For example, you can save a local copy in test.csv of the query select 1 with: copy (select 1) to local (location 'test.csv');
  • Currently, json, parquet and csv formats are supported. The default format is CSV, but you can specify a different format as follows:

  copy (select 1) to local format parquet (location 'test.parquet');

• ndjson_scan:

  • You can now read newline delimited JSON files, ex: select * from ndjson_scan('./your/file.json');

• misc:

  • Added timestamp support for native tables

What's Changed

• chore: Prevent python linking for macos by @scsmithr in #1198
• remove python dep from docker by @universalmind303 in #1199
• feat: Allow reading from the python environment by @scsmithr in #1195
• feat: Add pandas support by @scsmithr in #1201
• feat: Use multi-threaded runtime in python by @scsmithr in #1203
• chore: Break up slt tests into batches by @scsmithr in #1205
• chore: Add slt tests for parquet_scan and csv_scan by @scsmithr in #1206
• chore: add link to protoc installation by @greyscaled in #1209
• feat: Make data dir optional in python bindings by @scsmithr in #1211
• chore: Use raw github link in csv_scan test by @scsmithr in #1212
• feat: add support for scanning ndjson by @universalmind303 in #1213
• ci: setup python workflow by @universalmind303 in #1210
• remove py-glaredb from default members by @universalmind303 in #1224
• build(deps): Bump deps by @scsmithr in #1241
• check permissions on tmp dir by @universalmind303 in #1239
• chore: Bump rust version in docker image by @scsmithr in #1250
• chore: Remove starting metastore from pgproto test script by @scsmithr in #1252
• fix: Downgrade datadriven crate by @scsmithr in #1253
• fix: Fix explain queries by @scsmithr in #1251
• feat: COPY TO support by @vrongmeal in #1159
• refactor: xtask --> justfile by @universalmind303 in #1242
• ci: Keep py-glaredb and glaredb versions in sync by @universalmind303 in #1255
• fix: Allow create tables with timestamps by @vrongmeal in #1257
• chore: Add pytest by @scsmithr in #1258
• feat: support compute_engine authentication parameter by @f0ssel in #1256
• chore: Add more info for pypi by @scsmithr in #1263
• build(deps): Bump deps by @scsmithr in #1277
• refactor: use native execs instead of custom execs by @universalmind303 in #1262
• chore: Add test cases for parsing compute engine from conn string by @scsmithr in #1285
• fix: don't fetch filetype eagerly by @adhish20 in #1286
• feat: Add s3 to object store registry create by @scsmithr in #1291
• feat(py): add lazy execution to python by @universalmind303 in #1292
• chore: Add tests asserting logical plan reuse in python by @scsmithr in #1293
• build: Bump version to 0.2.0 by @scsmithr in #1295

New Contributors

• @adhish20 made their first contribution in #1286

Full Changelog: v0.0.16...v0.2.0

v0.0.16

Highlights

• table functions:
  • csv_scan(file) to read from CSV files locally, or remotely
  • All _scan functions can now handle file:// and relative URLs
• build:
  • Releases now contain a binary of glaredb for Windows
• misc:
  • Initial work on Python bindings for glaredb. More to come in the next release.

What's Changed

• feat: add "csv_scan" function by @universalmind303 in #1168
• chore: Log error if failed to find catalog by @scsmithr in #1173
• Revert "chore: Log error if failed to find catalog (#1173)" by @scsmithr in #1175
• build(deps): Bump deps by @scsmithr in #1181
• feat: Windows builds by @scsmithr in #1183
• chore: Add discord link to readme by @scsmithr in #1185
• feat: initial py bindings by @universalmind303 in #1182
• fix: Add python3 to dockerfile by @scsmithr in #1191
• fix: allow file & relative urls for *_scan by @universalmind303 in #1193
• release: v0.0.16 by @greyscaled in #1188

Full Changelog: v0.0.15...v0.0.16

v0.0.15

Highlights

• table create/insert:
  • added native table support (ie: CREATE TABLE my_table ( my_col text );
• table functions:
  • parquet_scan (ie: SELECT * FROM parquet_scan('path/to/file.parquet')
  • generate_series (ie: SELECT * FROM generate_series(1, 4)
• misc:
  • added credentials as a database object (see: #1140)
  • an --ignore-auth flag was added when running glaredb locally for development convenience
• fixes:
  • list_schemas no longer returns hidden schemas for BigQuery

What's Changed

• build(deps): Bump deps by @scsmithr in #1139
• chore: Allow ignoring auth messages by @scsmithr in #1145
• feat: Add native (internal) table support using delta-rs by @scsmithr in #1129
• feat: Add support for credentials objects by @vrongmeal in #1140
• feat: Add generate_series function by @scsmithr in #1154
• feat: add "parquet_scan" function by @universalmind303 in #1160
• fix: Don't return hidden data sets from bigquery by @scsmithr in #1162
• add support for reading from "file://" for parquet_scan by @universalmind303 in #1164
• feat: Stream writes to tables by @scsmithr in #1165
• chore: Remove native table inner by @scsmithr in #1169
• chore: bump version to 0.0.15 by @f0ssel in #1170

Full Changelog: v0.0.14...v0.0.15

v0.0.14

What's Changed

• feat: Support temporary tables by @vrongmeal in #1089
• build(deps): Bump deps by @scsmithr in #1114
• feat: Allow specifying user and password as commandline by @scsmithr in #1118
• build(deps): Update datafusion and object_store by @scsmithr in #1121
• feat: Add list_schemas and list_tables table functions by @vrongmeal in #1122
• chore: add documentation for demo postgres data by @f0ssel in #1123
• chore: add docs for demo user setup by @f0ssel in #1124
• chore: Add demo data source to readme by @scsmithr in #1125
• feat: Delta lake data source (initial implementation) by @scsmithr in #1119
• chore: Bump version to 0.0.14 by @scsmithr in #1130

Full Changelog: v0.0.13...v0.0.14

v0.0.13

Highlights

• data sources:
  • add functions for reading data sources directly, without connecting them
    • read_mongodb
    • read_mysql
    • read_bigquery
    • read_snowflake
    • read_postgres
    • there is a new catalog table glare_catalog.functions with columns for function_name and function_type
  • can now connect to postgres with TLS
• local:
  • there's a new 1-line install script in our README
  • glaredb local SQL shell now has sytnax highlighting and multiline support
• fixes:
  • corrected data source and ssh tunnel limits in GlareDB Cloud

What's Changed

• chore: Run ssh tunnel tests separately by @scsmithr in #1071
• feat: Make context provider async by @vrongmeal in #1066
• feat: Support EXPLAIN in async DF planner by @vrongmeal in #1072
• build(deps): Bump deps by @scsmithr in #1079
• fix: max datasource and tunnel counting by @greyscaled in #1082
• chore: Move metastore related protos and types to separate crates by @scsmithr in #1083
• build(deps): bump clap from 4.3.0 to 4.3.1 by @dependabot in #1080
• chore: Remove setting version to "pre-release" by @scsmithr in #1087
• feat: Add functions for reading data sources by @scsmithr in #1086
• feat(cli): add highlighting & multiline sql support by @universalmind303 in #1091
• chore: Run ci on pull_request by @scsmithr in #1092
• chore(readme): manual installation; typo by @greyscaled in #1093
• feat: Support connecting to external postgres dbs with tls by @scsmithr in #1094
• release: v0.0.13 by @greyscaled in #1095

New Contributors

• @universalmind303 made their first contribution in #1091

Full Changelog: v0.0.12...v0.0.13

v0.0.12

Highlights

• glaredb repo:
  • the repo is now open source, with ongoing improvements to make contributions and local development easier!
• glaredb local:
  • new -mode for formatting output

What's Changed

• chore: Update readme (and other docs) by @scsmithr in #1048
• chore: Remove gitguardian workflow by @scsmithr in #1051
• fix: readme logo in dark mode by @cmoog in #1049
• chore: Manually abort on panic for slt tests by @scsmithr in #1052
• chore: Use xtask when building release in docker image by @scsmithr in #1055
• feat: Add local client commands to specify output formatting by @scsmithr in #1056
• fix: Fix local limits by @scsmithr in #1057
• build(deps): Bump base64 and proptest by @scsmithr in #1050
• chore: Combine data sources into a single crate by @scsmithr in #1061
• feat: Make gcs and s3 credentials optional by @scsmithr in #1062
• chore: Update readme by @scsmithr in #1068
• release: v0.0.12 by @greyscaled in #1069

New Contributors

• @cmoog made their first contribution in #1049

Full Changelog: v0.0.11...v0.0.12

v0.0.11

Highlights

• fix:
  • SSH tunnels not being able to be created

What's Changed

• chore: Add release workflow by @scsmithr in #1018
• fix: Misc ssh tunnel fixes by @scsmithr in #1038
• build(deps): Update github actions deps by @scsmithr in #1046
• release: v0.0.11 by @greyscaled in #1047

Full Changelog: v0.0.10...v0.0.11

v0.0.10

What's Changed

• fix: Allow docker image pull to fail by @scsmithr in #1027
• feat: Add xtask by @scsmithr in #1028
• feat: Switch to rustls by @scsmithr in #1029
• fix: Add custom cert resolver by @scsmithr in #1031
• chore: Remove openssl dependencies by @scsmithr in #1036
• release: v0.0.10 by @greyscaled in #1035

Full Changelog: v0.0.9...v0.0.10

v0.0.9

Highlights

• tunnels
  • add ALTER TUNNEL [IF EXISTS] <tunnel_name> ROTATE KEYS command to rotate pub/priv keys for tunnels
  • set maximum number of tunnels (plan-based)

What's Changed

• build(deps): Bump deps by @scsmithr in #1007
• chore: Add gitguardian workflow by @scsmithr in #1010
• chore: Add SLT framework to setup pre and post test hooks by @vrongmeal in #1011
• feat: Add command for running glaredb locally by @scsmithr in #1012
• feat: Add clickbench scripts and queries by @scsmithr in #1014
• chore: Simplify CI by @scsmithr in #1016
• feat: ALTER TUNNEL .. ROTATE KEYS support. by @vrongmeal in #1017
• feat: restrict max tunnels by @f0ssel in #1021
• chore: Add aliases for some cargo actions by @vrongmeal in #1023
• release: v0.0.9 by @greyscaled in #1026

Full Changelog: v0.0.8...v0.0.9