feat: Enable monitoring via Prometheus and Grafana (rebased) (#48)

.github/workflows/paradedb-test-eks.yml

@@ -86,12 +86,12 @@ jobs:
       - name: Install the CloudNativePG Operator
         run: |
           helm repo add cnpg https://cloudnative-pg.github.io/charts
-          helm upgrade --install cnpg --namespace cnpg-system --create-namespace cnpg/cloudnative-pg
+          helm upgrade --atomic --install cnpg --namespace cnpg-system --create-namespace cnpg/cloudnative-pg
 
-      - name: Wait for CNPG Webhook Service to be Ready
+      - name: Install the Prometheus Operator
         run: |
-          kubectl wait --namespace cnpg-system --for=condition=available --timeout=120s deployment/cnpg-cloudnative-pg
-          kubectl get svc -n cnpg-system cnpg-webhook-service
+          helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
+          helm upgrade --atomic --install prometheus-community --namespace prometheus-community --create-namespace prometheus-community/prometheus-operator-crds
 
       - name: Test Helm Dependency Update
         working-directory: charts/paradedb/

.github/workflows/tests-cluster-chainsaw.yaml

@@ -6,7 +6,7 @@ on:
       - 'gh-pages'
 
 jobs:
-  test-cluster-chainsaw:
+  test-cluster-standalone:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout

README.md

@@ -30,21 +30,50 @@ The [ParadeDB](https://github.com/paradedb/paradedb) Helm Chart is based on the
 
 Kubernetes, and specifically the CloudNativePG operator, is the recommended approach for deploying ParadeDB in production, with high availability. ParadeDB also provides a [Docker image](https://hub.docker.com/r/paradedb/paradedb) and [prebuilt binaries](https://github.com/paradedb/paradedb/releases) for Debian, Ubuntu and Red Hat Enterprise Linux.
 
-The chart is also available on [ArtifactHub](https://artifacthub.io/packages/helm/paradedb/paradedb).
+The chart is also available on [Artifact Hub](https://artifacthub.io/packages/helm/paradedb/paradedb).
 
 ## Getting Started
 
 First, install [Helm](https://helm.sh/docs/intro/install/). The following steps assume you have a Kubernetes cluster running v1.25+. If you are testing locally, we recommend using [Minikube](https://minikube.sigs.k8s.io/docs/start/).
 
+### Installing the Prometheus Stack
+
+The ParadeDB Helm chart supports monitoring via Prometheus and Grafana. To enable this, you need to have the Prometheus CRDs installed before installing the CloudNativePG operator. If you do not yet have the Prometheus CRDs installed on your Kubernetes cluster, you can install it with:
+
+```bash
+helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
+helm upgrade --atomic --install prometheus-community \
+--create-namespace \
+--namespace prometheus-community \
+--values https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/main/docs/src/samples/monitoring/kube-stack-config.yaml \
+prometheus-community/kube-prometheus-stack
+```
+
 ### Installing the CloudNativePG Operator
 
-Skip this step if the CNPG operator is already installed in your cluster.
+Skip this step if the CloudNativePG operator is already installed in your cluster. If you do not wish to monitor your cluster, omit the `--set` commands.
 
-```console
+```bash
 helm repo add cnpg https://cloudnative-pg.github.io/charts
-helm upgrade --install cnpg \
+helm upgrade --atomic --install cnpg \
+--create-namespace \
 --namespace cnpg-system \
+--set monitoring.podMonitorEnabled=true \
+--set monitoring.grafanaDashboard.create=true \
+cnpg/cloudnative-pg
+```
+
+### Installing the CloudNativePG Operator
+
+Skip this step if the CloudNativePG operator is already installed in your cluster. If you do not wish to monitor your cluster, omit the `--set` commands.
+
+```bash
+helm repo add cnpg https://cloudnative-pg.github.io/charts
+helm upgrade --atomic --install cnpg \
 --create-namespace \
+--namespace cnpg-system \
+--set monitoring.podMonitorEnabled=true \
+--set monitoring.grafanaDashboard.create=true \
 cnpg/cloudnative-pg
 ```
 
@@ -57,46 +86,56 @@ type: paradedb
 mode: standalone
 
 cluster:
-  instances: 2
+  instances: 3
   storage:
     size: 256Mi
 ```
 
-Then, launch the ParadeDB cluster.
+Then, launch the ParadeDB cluster. If you do not wish to monitor your cluster, omit the `--set` command.
 
 ```bash
 helm repo add paradedb https://paradedb.github.io/charts
-helm upgrade --install paradedb \
---namespace paradedb-database \
+helm upgrade --atomic --install paradedb \
+--namespace paradedb \
 --create-namespace \
 --values values.yaml \
+--set cluster.monitoring.enabled=true \
 paradedb/paradedb
 ```
 
-If `--values values.yaml` is omitted, the default values will be used. For additional configuration options for the `values.yaml` file, please refer to the [ParadeDB Helm Chart documentation](https://artifacthub.io/packages/helm/paradedb/paradedb#values). For advanced cluster configuration options, please refer to the [CloudNativePG Cluster Chart documentation](charts/paradedb/README.md).
-
-A more detailed guide on launching the cluster can be found in the [Getting Started docs](<./charts/paradedb/docs/Getting Started.md>). To get started with ParadeDB, we suggest you follow the [quickstart guide](/documentation/getting-started/quickstart).
+If `--values values.yaml` is omitted, the default values will be used. For additional configuration options for the `values.yaml` file, including configuring backups and PgBouncer, please refer to the [ParadeDB Helm Chart documentation](https://artifacthub.io/packages/helm/paradedb/paradedb#values). For advanced cluster configuration options, please refer to the [CloudNativePG Cluster Chart documentation](charts/paradedb/README.md).
 
 ### Connecting to a ParadeDB CNPG Cluster
 
 The command to connect to the primary instance of the cluster will be printed in your terminal. If you do not modify any settings, it will be:
 
 ```bash
-kubectl --namespace paradedb-database exec --stdin --tty services/paradedb-rw -- bash
+kubectl --namespace paradedb exec --stdin --tty services/paradedb-rw -- bash
 ```
 
-This will launch a shell inside the instance. You can connect via `psql` with:
+This will launch a Bash shell inside the instance. You can connect to the ParadeDB database via `psql` with:
 
 ```bash
 psql -d paradedb
 ```
 
+### Connecting to the Grafana Dashboard
+
+To connect to the Grafana dashboard for your cluster, we suggested port forwarding the Kubernetes service running Grafana to localhost:
+
+```bash
+kubectl --namespace prometheus-community port-forward svc/prometheus-community-grafana 3000:80
+```
+
+You can then access the Grafana dasbhoard at [http://localhost:3000/](http://localhost:3000/) using the credentials `admin` as username and `prom-operator` as password. These default credentials are
+defined in the [`kube-stack-config.yaml`](https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/main/docs/src/samples/monitoring/kube-stack-config.yaml) file used as the `values.yaml` file in [Installing the Prometheus CRDs](#installing-the-prometheus-stack) and can be modified by providing your own `values.yaml` file.
+
 ## Development
 
 To test changes to the Chart on a local Minikube cluster, follow the instructions from [Getting Started](#getting-started), replacing the `helm upgrade` step by the path to the directory of the modified `Chart.yaml`.
 
 ```bash
-helm upgrade --install paradedb --namespace paradedb-database --create-namespace ./charts/paradedb
+helm upgrade --atomic --install paradedb --namespace paradedb --create-namespace ./charts/paradedb
 ```
 
 ## License

charts/paradedb/README.md

@@ -4,21 +4,36 @@ The [ParadeDB](https://github.com/paradedb/paradedb) Helm Chart is based on the
 
 Kubernetes, and specifically the CloudNativePG operator, is the recommended approach for deploying ParadeDB in production, with high availability. ParadeDB also provides a [Docker image](https://hub.docker.com/r/paradedb/paradedb) and [prebuilt binaries](https://github.com/paradedb/paradedb/releases) for Debian, Ubuntu and Red Hat Enterprise Linux.
 
-The chart is also available on [ArtifactHub](https://artifacthub.io/packages/helm/paradedb/paradedb).
+The chart is also available on [Artifact Hub](https://artifacthub.io/packages/helm/paradedb/paradedb).
 
 ## Getting Started
 
 First, install [Helm](https://helm.sh/docs/intro/install/). The following steps assume you have a Kubernetes cluster running v1.25+. If you are testing locally, we recommend using [Minikube](https://minikube.sigs.k8s.io/docs/start/).
 
+### Installing the Prometheus Stack
+
+The ParadeDB Helm chart supports monitoring via Prometheus and Grafana. To enable this, you need to have the Prometheus CRDs installed before installing the CloudNativePG operator. If you do not yet have the Prometheus CRDs installed on your Kubernetes cluster, you can install it with:
+
+```bash
+helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
+helm upgrade --atomic --install prometheus-community \
+--create-namespace \
+--namespace prometheus-community \
+--values https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/main/docs/src/samples/monitoring/kube-stack-config.yaml \
+prometheus-community/kube-prometheus-stack
+```
+
 ### Installing the CloudNativePG Operator
 
-Skip this step if the CNPG operator is already installed in your cluster.
+Skip this step if the CloudNativePG operator is already installed in your cluster. If you do not wish to monitor your cluster, omit the `--set` commands.
 
-```console
+```bash
 helm repo add cnpg https://cloudnative-pg.github.io/charts
-helm upgrade --install cnpg \
---namespace cnpg-system \
+helm upgrade --atomic --install cnpg \
 --create-namespace \
+--namespace cnpg-system \
+--set monitoring.podMonitorEnabled=true \
+--set monitoring.grafanaDashboard.create=true \
 cnpg/cloudnative-pg
 ```
 
@@ -31,50 +46,59 @@ type: paradedb
 mode: standalone
 
 cluster:
-  instances: 2
+  instances: 3
   storage:
     size: 256Mi
 ```
 
-Then, launch the ParadeDB cluster.
+Then, launch the ParadeDB cluster. If you do not wish to monitor your cluster, omit the `--set` command.
 
 ```bash
 helm repo add paradedb https://paradedb.github.io/charts
-helm upgrade --install paradedb \
---namespace paradedb-database \
+helm upgrade --atomic --install paradedb \
+--namespace paradedb \
 --create-namespace \
 --values values.yaml \
+--set cluster.monitoring.enabled=true \
 paradedb/paradedb
 ```
 
-If `--values values.yaml` is omitted, the default values will be used. For additional configuration options for the `values.yaml` file, please refer to the [ParadeDB Helm Chart documentation](https://artifacthub.io/packages/helm/paradedb/paradedb#values). For advanced cluster configuration options, please refer to the [CloudNativePG Cluster Chart documentation](charts/paradedb/README.md).
-
-A more detailed guide on launching the cluster can be found in the [Getting Started docs](<./docs/Getting Started.md>). To get started with ParadeDB, we suggest you follow the [quickstart guide](/documentation/getting-started/quickstart).
+If `--values values.yaml` is omitted, the default values will be used. For additional configuration options for the `values.yaml` file, including configuring backups and PgBouncer, please refer to the [ParadeDB Helm Chart documentation](https://artifacthub.io/packages/helm/paradedb/paradedb#values). For advanced cluster configuration options, please refer to the [CloudNativePG Cluster Chart documentation](charts/paradedb/README.md).
 
 ### Connecting to a ParadeDB CNPG Cluster
 
 The command to connect to the primary instance of the cluster will be printed in your terminal. If you do not modify any settings, it will be:
 
 ```bash
-kubectl --namespace paradedb-database exec --stdin --tty services/paradedb-rw -- bash
+kubectl --namespace paradedb exec --stdin --tty services/paradedb-rw -- bash
 ```
 
-This will launch a shell inside the instance. You can connect via `psql` with:
+This will launch a Bash shell inside the instance. You can connect to the ParadeDB database via `psql` with:
 
 ```bash
 psql -d paradedb
 ```
 
+### Connecting to the Grafana Dashboard
+
+To connect to the Grafana dashboard for your cluster, we suggested port forwarding the Kubernetes service running Grafana to localhost:
+
+```bash
+kubectl --namespace prometheus-community port-forward svc/prometheus-community-grafana 3000:80
+```
+
+You can then access the Grafana dasbhoard at [http://localhost:3000/](http://localhost:3000/) using the credentials `admin` as username and `prom-operator` as password. These default credentials are
+defined in the [`kube-stack-config.yaml`](https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/main/docs/src/samples/monitoring/kube-stack-config.yaml) file used as the `values.yaml` file in [Installing the Prometheus CRDs](#installing-the-prometheus-stack) and can be modified by providing your own `values.yaml` file.
+
 ## Development
 
 To test changes to the Chart on a local Minikube cluster, follow the instructions from [Getting Started](#getting-started), replacing the `helm upgrade` step by the path to the directory of the modified `Chart.yaml`.
 
 ```bash
-helm upgrade --install paradedb --namespace paradedb-database --create-namespace ./charts/paradedb
+helm upgrade --atomic --install paradedb --namespace paradedb --create-namespace ./charts/paradedb
 ```
 
-Cluster Configuration
----------------------
+## Cluster Configuration
 
 ### Database types
 
@@ -83,6 +107,7 @@ To use the ParadeDB Helm Chart, specify `paradedb` via the `type` parameter.
 ### Modes of operation
 
 The chart has three modes of operation. These are configured via the `mode` parameter:
+
 * `standalone` - Creates new or updates an existing CNPG cluster. This is the default mode.
 * `replica` - Creates a replica cluster from an existing CNPG cluster. **_Note_ that this mode is not yet supported.**
 * `recovery` - Recovers a CNPG cluster from a backup, object store or via pg_basebackup.
@@ -99,8 +124,10 @@ providers are supported:
 * Google Cloud Storage
 
 Additionally you can specify the following parameters:
+
 * `backups.retentionPolicy` - The retention policy for backups. Defaults to `30d`.
 * `backups.scheduledBackups` - An array of scheduled backups containing a name and a crontab schedule. Example:
+
 ```yaml
 backups:
   scheduledBackups:
@@ -113,13 +140,11 @@ Each backup adapter takes it's own set of parameters, listed in the [Configurati
 below. Refer to the table for the full list of parameters and place the configuration under the appropriate key: `backup.s3`,
 `backup.azure`, or `backup.google`.
 
-Recovery
---------
+## Recovery
 
 There is a separate document outlining the recovery procedure here: **[Recovery](docs/recovery.md)**
 
-Examples
---------
+## Examples
 
 There are several configuration examples in the [examples](examples) directory. Refer to them for a basic setup and
 refer to  the [CloudNativePG Documentation](https://cloudnative-pg.io/documentation/current/) for more advanced configurations.
@@ -179,15 +204,15 @@ refer to  the [CloudNativePG Documentation](https://cloudnative-pg.io/documentat
 | cluster.monitoring.customQueries | list | `[]` | Custom Prometheus metrics Will be stored in the ConfigMap |
 | cluster.monitoring.customQueriesSecret | list | `[]` | The list of secrets containing the custom queries |
 | cluster.monitoring.disableDefaultQueries | bool | `false` | Whether the default queries should be injected. Set it to true if you don't want to inject default queries into the cluster. |
-| cluster.monitoring.enabled | bool | `false` | Whether to enable monitoring |
+| cluster.monitoring.enabled | bool | `true` | Whether to enable monitoring |
 | cluster.monitoring.podMonitor.enabled | bool | `true` | Whether to enable the PodMonitor |
 | cluster.monitoring.podMonitor.metricRelabelings | list | `[]` | The list of metric relabelings for the PodMonitor. Applied to samples before ingestion. |
 | cluster.monitoring.podMonitor.relabelings | list | `[]` | The list of relabelings for the PodMonitor. Applied to samples before scraping. |
 | cluster.monitoring.prometheusRule.enabled | bool | `true` | Whether to enable the PrometheusRule automated alerts |
 | cluster.monitoring.prometheusRule.excludeRules | list | `[]` | Exclude specified rules |
 | cluster.postgresGID | int | `-1` | The GID of the postgres user inside the image, defaults to 26 |
 | cluster.postgresUID | int | `-1` | The UID of the postgres user inside the image, defaults to 26 |
-| cluster.postgresql.parameters | object | `{}` | PostgreSQL configuration options (postgresql.conf) |
+| cluster.postgresql.parameters | object | `{"cron.database_name":"postgres"}` | PostgreSQL configuration options (postgresql.conf) |
 | cluster.postgresql.pg_hba | list | `[]` | PostgreSQL Host Based Authentication rules (lines to be appended to the pg_hba.conf file) |
 | cluster.postgresql.pg_ident | list | `[]` | PostgreSQL User Name Maps rules (lines to be appended to the pg_ident.conf file) |
 | cluster.postgresql.shared_preload_libraries | list | `[]` | Lists of shared preload libraries to add to the default ones |