Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor README.md for PGXN #168

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Refactor README.md for PGXN #168

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f4d6a63
Refactor README.md for PGXN
mkgrgis Oct 4, 2023
bef888f
Fix spell errors, README.md
mkgrgis Oct 5, 2023
9a785e5
Fix spelling, README.md
mkgrgis Oct 5, 2023
f6fed1d
Merge branch 'master' into readme_fix
mkgrgis Jan 17, 2024
c519375
Merge branch 'EnterpriseDB:master' into readme_fix
mkgrgis May 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 60 additions & 7 deletions INSTALL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,32 @@
Notes about installation Mongo Foreign Data Wrapper
===================================================

To compile the [MongoDB][1] foreign data wrapper for [PostgreSQL](https://www.postgresql.org/), `mongo-c` and `json-c`
libraries are needed. To build and install `mongo-c` and `json-c` libraries, there
are two ways. You can either use script `autogen.sh` or you can manually
To compile the [MongoDB][1] foreign data wrapper for [PostgreSQL](https://www.postgresql.org/),
`mongo-c` and `json-c` libraries are needed. To build and install `mongo-c` and `json-c`
libraries, there are two ways. You can either use script `autogen.sh` or you can manually
perform all required steps listed.

### Notes about new MongoDB C Driver support
### Notes about MongoDB `meta` C Driver support

The current implementation is based on the driver version 1.17.3 of MongoDB.

[MongoDB][1] provided completely new library for driver called
MongoDB's meta driver and also old driver called legacy.
Command sequence for installing both legacy and meta drivers
is similar with small differences. You can select driver by compile time option.

In order to use MongoDB driver 1.17.0+, take the following steps:

* clone `libmongoc` version 1.17.0+
(https://github.com/mongodb/mongo-c-driver) and follow the install
directions given there. `libbson` is now maintained in a subdirectory
of the `libmongoc`.
(https://github.com/mongodb/mongo-c-driver/tree/master/src/libbson).
* ensure pkg-config / pkgconf is installed on your system.
* run `make -f Makefile.meta && make -f Makefile.meta install`
* if you get an error when trying to `CREATE EXTENSION `mongo_fdw`;`,
then try running `ldconfig`

## Installation using script
Number of manual steps needs to be performed to compile and install required
mongo-c and json-c libraries. If you want to avoid the manual steps, there is a
Expand All @@ -23,13 +41,19 @@ respectively. If these variables are not set then these libraries will be
installed in the default location. Please note that you need to have the
required permissions on the directory where you want to install the libraries.

* autogen.sh
Build with [MongoDB][1]'s legacy branch driver (Deprecated in mongo_fdw 5.4.0)
* autogen.sh --with-legacy

Build [MongoDB][1]'s master branch driver
* autogen.sh --with-master

The script autogen.sh will do all the necessary steps to build with mongo-c
driver accordingly.
The script autogen.sh will do all the necessary steps to build with legacy and
meta driver accordingly.

## Steps for manual installation
### mongo-c
#### meta driver

1. Download and extract source code of mongoc driver for version `1.17.3`

```sh
Expand Down Expand Up @@ -61,6 +85,21 @@ driver accordingly.

For more details on installation of mongo-c driver, you can refer [here][3].

#### Legacy driver

Deprecation Notice:
The legacy driver support has been deprecated in mongo_fdw 5.4.0 and is
expected to be removed entirely in a future release.

* Checkout, extract legacy branch

```sh
wget https://github.com/mongodb/mongo-c-driver/archive/v0.8.tar.gz
tar -zxf v0.8.tar.gz
rm -rf mongo-c-driver
mv mongo-c-driver-0.8 mongo-c-driver
```

### json-c

1. Download and extract source code
Expand Down Expand Up @@ -93,6 +132,20 @@ For more details on installation of mongo-c driver, you can refer [here][3].

For more details on installation of json-c library, you can refer [here][4].

### How to compile against mongo-c Meta or Legacy driver?
To compile against legacy driver, 'Makefile.legacy' must be used and
'Makefile.meta' must be used to compile against the meta driver. For example,
this can be achieved by copying required Makefile as shown below:
For meta,

cp Makefile.meta Makefile

For legacy (Deprecated in mongo_fdw 5.4.0),

cp Makefile.legacy Makefile

The default compilation is with Meta driver.

## Mongo_fdw configuration, compilation and installation
The `PKG_CONFIG_PATH` environment variable must be set to mongo-c-driver source
directory for successful compilation as shown below,
Expand Down
97 changes: 62 additions & 35 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Contents
8. [Character set handling](#character-set-handling)
9. [Examples](#examples)
10. [Limitations](#limitations)
11. [Contributing](#contributing)
11. [Contributing and testing](#contributing-and-testing)
12. [Useful links](#useful-links)

Features
Expand Down Expand Up @@ -120,6 +120,8 @@ Usage
Controls whether `mongo_fdw` uses exact rows from
remote collection to obtain cost estimates.

The following options are _only supported with meta driver_:

- **authentication_database** as *string*, optional

Database against which user will be
Expand Down Expand Up @@ -192,7 +194,8 @@ Usage
performing a sort locally. This option can also be set for an individual
table, and if any of the tables involved in the query has set it to
false then the ORDER BY will not be pushed down. The table-level value
of the option takes precedence over the server-level option value.
of the option takes precedence over the server-level option value.lue of the option takes precedence over the server-level
option value.

## CREATE USER MAPPING options

Expand Down Expand Up @@ -225,11 +228,11 @@ command:
Similar to the server-level option, but can be
configured at table level as well.

- **enable_aggregate_pushdown** as *boolean*, optional, default `true`
- **enable_aggregate_pushdown** as *boolean*, optional, default corresponding server-level value

Similar to the server-level option, but can be configured at table level as well.

- **enable_order_by_pushdown** as *boolean*, optional, default `true`
- **enable_order_by_pushdown** as *boolean*, optional, default corresponding server-level value

Similar to the server-level option, but can be configured at table level as well.

Expand All @@ -256,9 +259,11 @@ functions, `mongo_fdw` provides the following user-callable utility functions:
Identifier case handling
------------------------

PostgreSQL folds identifiers to lower case by default, MongoDB use JSON notation of identifiers.
PostgreSQL folds identifiers to lower case by default, MongoDB use JSON notation of
identifiers without any default character case transformations.

All transformation rules and problems **yet not described**.
Hence if the BSON document key contains uppercase letters or occurs within a nested
document, PostgreSQLrequires the corresponding column names to be quoted.

Generated columns
-----------------
Expand Down Expand Up @@ -305,37 +310,42 @@ execution plans for a query, just run `EXPLAIN`.
Once for a database you need, as PostgreSQL superuser.

```sql
CREATE EXTENSION mongo_fdw;
CREATE EXTENSION mongo_fdw;
```

### Create a foreign server with appropriate configuration:

Once for a foreign data source you need, as PostgreSQL superuser.

```sql
CREATE SERVER "MongoDB server" FOREIGN DATA WRAPPER mongo_fdw OPTIONS (
address '127.0.0.1',
port '27017'
);
CREATE SERVER "MongoDB server"
FOREIGN DATA WRAPPER mongo_fdw
OPTIONS (
address '127.0.0.1',
port '27017'
);
```

### Grant usage on foreign server to normal user in PostgreSQL:

Once for a normal user (non-superuser) in PostgreSQL, as PostgreSQL superuser. It is a good idea to use a superuser only where really necessary, so let's allow a normal user to use the foreign server (this is not required for the example to work, but it's security recommendation).

```sql
GRANT USAGE ON FOREIGN SERVER "MongoDB server" TO pguser;
GRANT USAGE ON FOREIGN SERVER "MongoDB server" TO pguser;
```
Where `pguser` is a sample user for works with foreign server (and foreign tables).

### User mapping

Create an appropriate user mapping:
```sql
CREATE USER MAPPING FOR pguser SERVER "MongoDB server" OPTIONS (
username 'mongo_user',
password 'mongo_pass'
);
CREATE USER MAPPING
FOR pguser
SERVER "MongoDB server"
OPTIONS (
username 'mongo_user',
password 'mongo_pass'
);
```
Where `pguser` is a sample user for works with foreign server (and foreign tables).

Expand All @@ -345,23 +355,27 @@ All `CREATE FOREIGN TABLE` SQL commands can be executed as a normal PostgreSQL u
Create a foreign table referencing the MongoDB collection:

```sql
-- Note: first column of the table must be "_id" of type "name".
CREATE FOREIGN TABLE warehouse (
_id name,
warehouse_id int,
warehouse_name text,
warehouse_created timestamptz
) SERVER "MongoDB server" OPTIONS (
database 'db',
collection 'warehouse'
);
-- Note: first column of the table must be "_id" of type "name".
CREATE FOREIGN TABLE warehouse (
_id name,
warehouse_id int,
warehouse_name text,
warehouse_created timestamptz
)
SERVER "MongoDB server"
OPTIONS (
database 'db',
collection 'warehouse'
);
```

### Typical examples with [MongoDB][1]'s equivalent statements.

#### `SELECT`
```sql
SELECT * FROM warehouse WHERE warehouse_id = 1;
SELECT *
FROM warehouse
WHERE warehouse_id = 1;
```
```
_id | warehouse_id | warehouse_name | warehouse_created
Expand Down Expand Up @@ -401,7 +415,9 @@ db.warehouse.insert
```
#### `DELETE`
```sql
DELETE FROM warehouse WHERE warehouse_id = 2;
DELETE
FROM warehouse
WHERE warehouse_id = 2;
```
```
db.warehouse.remove
Expand All @@ -413,7 +429,9 @@ db.warehouse.remove
```
#### `UPDATE`
```sql
UPDATE warehouse SET warehouse_name = 'UPS_NEW' WHERE warehouse_id = 1;
UPDATE warehouse
SET warehouse_name = 'UPS_NEW'
WHERE warehouse_id = 1;
```
```
db.warehouse.update
Expand Down Expand Up @@ -447,21 +465,30 @@ ANALYZE warehouse;
Limitations
-----------

- If the BSON document key contains uppercase letters or occurs within
a nested document, ``mongo_fdw`` requires the corresponding column names
to be declared in double quotes.

- Note that PostgreSQL limits column names to 63 characters by
default. If you need column names that are longer, you can increase the
`NAMEDATALEN` constant in `src/include/pg_config_manual.h`, compile,
and re-install.

Contributing
------------
Contributing and testing
------------------------

Have a fix for a bug or an idea for a great new feature? Great! Check
out the contribution guidelines [here][3].

### Running regression test.
Run `mongodb_init.sh` file to load required collections.
```sh
source mongodb_init.sh
```
Finally, run regression.
```sh
make USE_PGXS=1 installcheck
```
However, make sure to set the `MONGO_HOST`, `MONGO_PORT`, `MONGO_USER_NAME`,
and `MONGO_PWD` environment variables correctly. The default settings can
be found in the `mongodb_init.sh` script.

Useful links
------------

Expand Down