Database Lab Engine v3.1.0

Database Lab Engine v3.1.0

DLE 3.1 has many improvements and several new features, such as native support of pgBackRest and the ability to configure the timezone when using CLI.

In this release, the development team has focused on the Database Lab Engine community, making it easier to get help or contribute. The team greets all new contributors: @Nikolay Devxx, @asotolongo, @Tanya301, @denis-boost, @pietervincken, @ane4ka

Community news

• 🌠 DLE repository on GitHub now has 1,100+ stars; many thanks to everyone who supports the project in any way
• 💥 Pieter Vincken has published a blog post describing their experience of using DLE: "Testing with production data made easy"
• 📈 The Twitter account has reached 400 followers: @Database_Lab
• 🎉 DLE now has 15 contributors. More contributions are welcome! See "good first issues"
• 🥇 Please consider various ways to contribute – read CONTRIBUTING.md

The README.md has been significantly improved. It now has four translations, with more coming in the future. Many thanks to our README translation contributors! It helps engineers worldwide understand DLE concepts and improve the development experience by building powerful dev&test environments.

• Ukrainian / Українська (🙏 @denis-boost)
• Spanish / Español (🙏 @asotolongo)
• Russian / Pусский (🙏 @Tanya301)
• German / Deutsch (🙏 @ane4ka)

Migration notes

⚠️ Prior to version 3.1, the default ZFS version DLE's Docker images were using was 0.8.x. For DLE 3.1, the default Docker image (postgresai/dblab-server:3.1.0) uses ZFS 2.1.2. If you need to upgrade DLE from a previous version that worked with a ZFS pool 0.8.x (default for previous DLE versions), consider the following options:

• Option 1: upgrade your system to use ZFS 2.1, optionally upgrade your pool (zpool upgrade dblab_pool), and then upgrade DLE to use the default image postgresai/dblab-server:3.1.0
• Option 2: postpone the ZFS upgrade, stay on ZFS 0.8, and upgrade DLE to version 3.1 using a special image, postgresai/dblab-server:3.1.0-zfs0.8

Changelog

New features

• Native support for pgBackRest as a tool to restore data from archives (physical mode, including continuously updated state) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/442, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/454 (:bow: contribution by @Nikolay Devxx)
• Allow configuring timezone in DLE CLI configuration to improve the experience of using DLE in CI/CD pipelines https://gitlab.com/postgres-ai/database-lab/-/merge_requests/486, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/502
• Automatically clean up failed clones with their sessions (for example, errors during the Postgres clone configuration – in the past, in such cases, DLE lost control over such clones and they could not be removed, even using an API request) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/405, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/404
• For logical mode, during dump/restore, DLE now cleans up only those files or directories that are specified to be restored https://gitlab.com/postgres-ai/database-lab/-/merge_requests/410
• Improved support of multiple DLEs running on the same machine: Take the pool name into account when determining the clone existence https://gitlab.com/postgres-ai/database-lab/-/merge_requests/471
• Rework README, add translations https://gitlab.com/postgres-ai/database-lab/-/merge_requests/411, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/423, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/467, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/478, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/481, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/497
• CONTRIBUTING.md: Describe components and Makefile targets https://gitlab.com/postgres-ai/database-lab/-/merge_requests/413

Improvements and fixes

• Reset the snapshot state atomically to avoid the moments in time when the DLE does not have a snapshot https://gitlab.com/postgres-ai/database-lab/-/merge_requests/406
• Skip connection if the DLE container is already connected to an internal network. This solves the problem when in some cases, DLE cannot start because the DLE container is already connected to the bridge network https://gitlab.com/postgres-ai/database-lab/-/merge_requests/416
• Optimize snapshot list. This has to improve the performance of /snapshot API calls which could take many seconds in the past https://gitlab.com/postgres-ai/database-lab/-/merge_requests/426
• ZFS version 2.1.2 inside DLE container (optional, 0.8 remains the default in DLE 3.x) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/343, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/477
• Do not specify the logerrors extension by default https://gitlab.com/postgres-ai/database-lab/-/merge_requests/485
• Replace the "standby_mode" parameter with "recovery_target" in the DLE configuration for the promotion stage https://gitlab.com/postgres-ai/database-lab/-/merge_requests/484
• Use the correct path to the data directory for the promote instance https://gitlab.com/postgres-ai/database-lab/-/merge_requests/503
• Fix wrong server version in the DLE response https://gitlab.com/postgres-ai/database-lab/-/merge_requests/508
• Avoid losing the snapshot list on configuration reload https://gitlab.com/postgres-ai/database-lab/-/merge_requests/506
• In logical mode, dump only databases to which user have access https://gitlab.com/postgres-ai/database-lab/-/merge_requests/501
• UI: Add clone port to clones list https://gitlab.com/postgres-ai/database-lab/-/merge_requests/447
• UI: Correct status detection https://gitlab.com/postgres-ai/database-lab/-/merge_requests/449
• UI: Add the total size of clones to instance page header https://gitlab.com/postgres-ai/database-lab/-/merge_requests/450
• UI: Fetch failed on connection refused calls https://gitlab.com/postgres-ai/database-lab/-/merge_requests/459
• UI: Adjust DLE stats labels https://gitlab.com/postgres-ai/database-lab/-/merge_requests/461
• UI (Platform only): Fix command page scroll https://gitlab.com/postgres-ai/database-lab/-/commit/992b0ecc4f43a578eeadd3818d8824e470b78009
• UI (Platform only): Fix markdown parser https://gitlab.com/postgres-ai/database-lab/-/merge_requests/496

Internal

• Set up GPG to have signed commits in GitPod https://gitlab.com/postgres-ai/database-lab/-/merge_requests/407
• Use GitHub CodeQL Analysis https://gitlab.com/postgres-ai/database-lab/-/merge_requests/409
• Define rules to reduce the running time of integration tests - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/414
• Upgrade the default Docker image - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/412, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/476
• Add .whitesource configuration file - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/420
• Rework LICENSE files - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/429
• Don't count .js in repo stats - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/430
• Rework CI - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/425, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/431, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/453, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/457, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/493, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/495, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/499
• Describe how dataStateAt is determined - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/356
• Isolate Database Lab Engine sources - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/443
• Update dependencies for security reasons - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/439, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/445, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/456, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/458, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/466, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/451, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/452, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/469, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/468, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/472, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/504
• Make UI source code publicly available https://gitlab.com/postgres-ai/database-lab/-/merge_requests/421
• Update Go to 1.18 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/498

How to get help

To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.

Provide feedback, contribute

Any feedback will be greatly appreciated:

• Twitter: @Database_Lab
• Database Lab Community Slack: https://slack.postgres.ai/

If you want to help the project grow, consider various ways of contributing:

• Star the GitHub repo: https://github.com/postgres-ai/database-lab-engine
• Spread the word using Twitter (please mention @Database_Lab!) or your social network of choice
• Consider translating the README.md to one or several languages you know
• Consider implementing one of the features that will make DLE better: see CONTRIBUTING.md and the list of "good first issues" on GitLab.

Diff between versions v3.0.0 and v3.1.0-rc.1: v3.0.0...v3.1.0-rc.1