Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MongoDB: Improve UX by using ctk load table mongodb://... #79

Merged
merged 4 commits into from
Nov 13, 2023

Conversation

amotl
Copy link
Member

@amotl amotl commented Nov 13, 2023

About

This patch adjusts the MongoDB adapter added with GH-76 to have the same interface like GH-77, in order to facilitate convenient data transfers from MongoDB.

Synopsis

# Define target to be CrateDB schema=testdrive, table=demo.
export CRATEDB_SQLALCHEMY_URL=crate://crate@localhost:4200/testdrive/demo

# Load data from database=testdrive, collection=demo.
ctk load table mongodb://localhost:27017/testdrive/demo

# Verify data has been transferred.
crash --command "SELECT * FROM testdrive.demo;"

Details

The patch effectively automates the procedure outlined within the previous README on behalf of a mongodb_copy() primitive.

https://github.com/crate-workbench/cratedb-toolkit/blob/c31df22b59c8019245d55b2bf10496818f3a0ecc/cratedb_toolkit/io/mongodb/api.py#L12-L70

Reviewers Note

This PR currently can not be reviewed well, because it has multiple ancestors not merged yet. Other PRs will need to be unstacked first.

@amotl amotl changed the base branch from amo/mongodb to amo/cloud-first November 13, 2023 01:32
@amotl amotl force-pushed the amo/mongodb-ctk-load-table branch 4 times, most recently from 89bd493 to c4fa9aa Compare November 13, 2023 02:10
Copy link

codecov bot commented Nov 13, 2023

Codecov Report

Attention: 14 lines in your changes are missing coverage. Please review.

Files Coverage Δ
cratedb_toolkit/io/influxdb.py 100.00% <100.00%> (ø)
cratedb_toolkit/io/mongodb/export.py 56.36% <100.00%> (+29.44%) ⬆️
cratedb_toolkit/io/mongodb/extract.py 93.22% <100.00%> (+20.80%) ⬆️
cratedb_toolkit/io/mongodb/translate.py 93.05% <100.00%> (+31.08%) ⬆️
cratedb_toolkit/io/mongodb/util.py 100.00% <100.00%> (ø)
cratedb_toolkit/model.py 98.11% <100.00%> (+1.33%) ⬆️
cratedb_toolkit/testing/testcontainers/mongodb.py 100.00% <100.00%> (ø)
cratedb_toolkit/testing/testcontainers/util.py 80.00% <100.00%> (+12.07%) ⬆️
cratedb_toolkit/util/cr8.py 100.00% <100.00%> (ø)
cratedb_toolkit/io/mongodb/cli.py 71.69% <80.00%> (+2.60%) ⬆️
... and 4 more

📢 Thoughts on this report? Let us know!

@amotl amotl force-pushed the amo/mongodb-ctk-load-table branch 2 times, most recently from 91efd25 to c31df22 Compare November 13, 2023 02:40
Base automatically changed from amo/cloud-first to main November 13, 2023 08:11
@amotl amotl force-pushed the amo/mongodb-ctk-load-table branch from c31df22 to 0e85b58 Compare November 13, 2023 08:24
Use `influxdb` vs. `mongodb` for recently added test cases.
- Downgrade to Python 3.11
- Downgrade to pymongo<4 on MongoDB 2
- Skip testing against MongoDB 2, because `auth failed`
@amotl amotl force-pushed the amo/mongodb-ctk-load-table branch from 0e85b58 to 48b4f18 Compare November 13, 2023 08:40
@amotl amotl marked this pull request as ready for review November 13, 2023 08:45
@amotl amotl merged commit d62a71d into main Nov 13, 2023
@amotl amotl deleted the amo/mongodb-ctk-load-table branch November 13, 2023 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant