Add MySQL integration

.github/workflows/platform-test.yml

@@ -42,6 +42,7 @@ jobs:
     name: Platform Test
     needs: list-tests
     runs-on: ubuntu-latest
+    if: ${{ needs.list-tests.outputs.tests != '[]' }}
     strategy:
       matrix:
         test: ${{ fromJson(needs.list-tests.outputs.tests) }}

CONTRIBUTING.md

@@ -42,10 +42,9 @@ Each component has its own guide for contributing:
 <!-- [Main Helm chart](./charts/k8s-monitoring/README.md#Contributing) -->
 
 -   [Feature subcharts](./charts/k8s-monitoring/docs/Features.md#Contributing)
+-   [Integrations](./charts/feature-integrations/README.md#Contributing)
 -   [Destinations](./charts/k8s-monitoring/docs/destinations/README.md#Contributing)
 
-<!-- [Integrations](./charts/feature-integrations/README.md#Contributing) -->
-
 ## Building
 
 Lots of files in this repository are generated in order to reduce duplication and ensure consistency. To build these

charts/feature-integrations/README.md

@@ -105,3 +105,50 @@ Actual integration testing in a live environment should be done in the main [k8s
 | global.alloyModules.source | string | `"git"` | The source of the Alloy modules. The valid options are "configMap" or "git" |
 | global.maxCacheSize | int | `100000` | Sets the max_cache_size for every prometheus.relabel component. ([docs](https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.relabel/#arguments)) This should be at least 2x-5x your largest scrape target or samples appended rate. |
 | global.scrapeInterval | string | `"60s"` | How frequently to scrape metrics. |
+
+### Integration: MySQL
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| mysql | object | `{"instances":[]}` | Scrape metrics from MySQL |
+
+## Contributing
+
+To contribute integrations to this feature, there are a few files that need to be created or modified:
+
+*   `values.yaml` - The main feature chart's values file. Add a section for your integration. It must contain an
+    `instance` array and any settings that apply to every instance of the integration. For example:
+
+    ```yaml
+    <slug>:
+      instances: []
+      globalSetting: value
+    ```
+
+*   `integrations/<slug>-values.yaml` - The values that will be used for each instance. This must include `name` to
+    differentiate it from other instances, and any other settings that are specific to that instance. For example:
+
+    ```yaml
+    name: ""
+    labelSelectors:
+      app.kubernetes.io/name: my-service
+    protocol: http
+    ...
+    ```
+
+*   `templates/_integration<=_<slug>.tpl` - The file that contains template functions that build the configuration to
+    discover, gather, process, and deliver the telemetry data. This file is required to implement the following template
+    functions:
+    *   `integrations.<slug>.type.metrics` - Returns true if this integration scrapes metrics.
+    *   `integrations.<slug>.type.logs` - Returns true if this integration gathers logs.
+    *   `integrations.<slug>.module` - Returns the configuration that is included once if this integration is used. This
+        is typically the module definition.
+    *   `integrations.<slug>.include.metrics` - Returns the configuration that is included for each instance of the
+        integration that scrapes metrics.
+    *   `integrations.<slug>.include.logs` - Returns the configuration that is included for each instance of the
+        integration that gathers logs.
+    *   `integrations.<slug>.exclude.logs` - Returns a rule that can be used by other Log-gathering features to ensure
+        that logs that are gathered from this integration are not collected twice. Typically the inverse of a rule in
+        the `integrations.<slug>.include.logs` function.
+    *   `default-allow-lists/<slug>.yaml` - If the integration scrapes metrics, a common pattern is to provide a list of
+        metrics that should be allowed. This minimizes the amount of metrics delivered to a useful minimal set.

charts/feature-integrations/README.md.gotmpl

@@ -66,3 +66,44 @@ Actual integration testing in a live environment should be done in the main [k8s
 <!-- markdownlint-enable no-bare-urls -->
 {{ template "chart.requirementsSection" . }}
 {{ template "chart.valuesSection" . }}
+
+## Contributing
+
+To contribute integrations to this feature, there are a few files that need to be created or modified:
+
+*   `values.yaml` - The main feature chart's values file. Add a section for your integration. It must contain an
+    `instance` array and any settings that apply to every instance of the integration. For example:
+
+    ```yaml
+    <slug>:
+      instances: []
+      globalSetting: value
+    ```
+
+*   `integrations/<slug>-values.yaml` - The values that will be used for each instance. This must include `name` to
+    differentiate it from other instances, and any other settings that are specific to that instance. For example:
+
+    ```yaml
+    name: ""
+    labelSelectors:
+      app.kubernetes.io/name: my-service
+    protocol: http
+    ...
+    ```
+
+*   `templates/_integration<=_<slug>.tpl` - The file that contains template functions that build the configuration to
+    discover, gather, process, and deliver the telemetry data. This file is required to implement the following template
+    functions:
+    *   `integrations.<slug>.type.metrics` - Returns true if this integration scrapes metrics.
+    *   `integrations.<slug>.type.logs` - Returns true if this integration gathers logs.
+    *   `integrations.<slug>.module` - Returns the configuration that is included once if this integration is used. This
+        is typically the module definition.
+    *   `integrations.<slug>.include.metrics` - Returns the configuration that is included for each instance of the
+        integration that scrapes metrics.
+    *   `integrations.<slug>.include.logs` - Returns the configuration that is included for each instance of the
+        integration that gathers logs.
+    *   `integrations.<slug>.exclude.logs` - Returns a rule that can be used by other Log-gathering features to ensure
+        that logs that are gathered from this integration are not collected twice. Typically the inverse of a rule in
+        the `integrations.<slug>.include.logs` function.
+    *   `default-allow-lists/<slug>.yaml` - If the integration scrapes metrics, a common pattern is to provide a list of
+        metrics that should be allowed. This minimizes the amount of metrics delivered to a useful minimal set.

charts/feature-integrations/docs/integrations/alloy.md

@@ -28,4 +28,4 @@
 
 | Key | Type | Default | Description |
 |-----|------|---------|-------------|
-| scrapeInterval | string | `60s` | How frequently to scrape metrics from Windows Exporter. |
+| scrapeInterval | string | `60s` | How frequently to scrape metrics from Alloy. |

charts/feature-integrations/docs/integrations/mysql.md

@@ -0,0 +1,55 @@
+# mysql
+
+## Values
+
+### Exporter Settings
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| exporter.collectors | list | `["heartbeat","mysql.user"]` | The list of collectors to enable for the MySQL Exporter ([Documentation](https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.exporter.mysql/#supported-collectors)). |
+| exporter.dataSource | object | `{"auth":{"password":"","passwordFrom":"","passwordKey":"password","username":"","usernameFrom":"","usernameKey":"username"},"host":"","port":3306}` | The data source to use for the MySQL Exporter. |
+| exporter.dataSource.auth.password | string | `""` | The password to use for the MySQL connection. |
+| exporter.dataSource.auth.passwordFrom | string | `""` | Raw config for accessing the password. |
+| exporter.dataSource.auth.passwordKey | string | `"password"` | The key for storing the password in the secret. |
+| exporter.dataSource.auth.username | string | `""` | The username to use for the MySQL connection. |
+| exporter.dataSource.auth.usernameFrom | string | `""` | Raw config for accessing the username. |
+| exporter.dataSource.auth.usernameKey | string | `"username"` | The key for storing the username in the secret. |
+| exporter.dataSource.host | string | `""` | The MySQL host to connect to. |
+| exporter.dataSource.port | int | `3306` | The MySQL port to connect to. |
+| exporter.dataSourceName | string | `""` | The data source string to use for the MySQL Exporter. |
+
+### Logs Settings
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| logs.enabled | bool | `true` | Whether to enable special processing of MySQL pod logs. |
+| logs.labelSelectors | object | `{}` | Label selectors to choose the set of pods to gather logs from for this MySQL Instance. Example: `app.kubernetes.io/name: mysql` |
+
+### Metric Processing Settings
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| maxCacheSize | string | `100000` | Sets the max_cache_size for prometheus.relabel component. This should be at least 2x-5x your largest scrape target or samples appended rate. ([docs](https://grafana.com/docs/alloy/latest/reference/components/prometheus.relabel/#arguments)) Overrides global.maxCacheSize |
+| metricsTuning.excludeMetrics | list | `[]` | Metrics to drop. Can use regular expressions. |
+| metricsTuning.includeMetrics | list | `[]` | Metrics to keep. Can use regular expressions. |
+
+### General Settings
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| name | string | `""` | Name for this MySQL instance. |
+
+### Scrape Settings
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| scrapeInterval | string | `60s` | How frequently to scrape metrics from MySQL Exporter. |
+
+### Secret
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| secret.create | bool | `true` | Whether to create a secret to store credentials for this MySQL integration instance. |
+| secret.embed | bool | `false` | If true, skip secret creation and embed the credentials directly into the configuration. |
+| secret.name | string | `""` | The name of the secret to create. |
+| secret.namespace | string | `""` | The namespace for the secret. |

charts/feature-integrations/integrations/alloy-values.yaml

@@ -8,7 +8,7 @@ name: ""
 labelSelectors:
   app.kubernetes.io/name: alloy
 
-# -- How frequently to scrape metrics from Windows Exporter.
+# -- How frequently to scrape metrics from Alloy.
 # @default -- `60s`
 # @section -- Scrape Settings
 scrapeInterval:

charts/feature-integrations/integrations/mysql-values.yaml

@@ -0,0 +1,94 @@
+---
+# -- Name for this MySQL instance.
+# @section -- General Settings
+name: ""
+
+# Settings for log gathering using the Pod Logs feature
+logs:
+  # -- Whether to enable special processing of MySQL pod logs.
+  # @section -- Logs Settings
+  enabled: true
+
+  # -- Label selectors to choose the set of pods to gather logs from for this MySQL Instance.
+  # Example:
+  # `app.kubernetes.io/name: mysql`
+  # @section -- Logs Settings
+  labelSelectors: {}
+
+# Settings for the Alloy embedded MySQL Exporter
+exporter:
+  # -- The data source string to use for the MySQL Exporter.
+  # @section -- Exporter Settings
+  dataSourceName: ""
+
+  # -- The data source to use for the MySQL Exporter.
+  # @section -- Exporter Settings
+  dataSource:
+    # -- The MySQL host to connect to.
+    # @section -- Exporter Settings
+    host: ""
+    # -- The MySQL port to connect to.
+    # @section -- Exporter Settings
+    port: 3306
+    auth:
+      # -- The username to use for the MySQL connection.
+      # @section -- Exporter Settings
+      username: ""
+      # -- The key for storing the username in the secret.
+      # @section -- Exporter Settings
+      usernameKey: "username"
+      # -- Raw config for accessing the username.
+      # @section -- Exporter Settings
+      usernameFrom: ""
+      # -- The password to use for the MySQL connection.
+      # @section -- Exporter Settings
+      password: ""
+      # -- The key for storing the password in the secret.
+      # @section -- Exporter Settings
+      passwordKey: "password"
+      # -- Raw config for accessing the password.
+      # @section -- Exporter Settings
+      passwordFrom: ""
+
+  # -- The list of collectors to enable for the MySQL Exporter ([Documentation](https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.exporter.mysql/#supported-collectors)).
+  # @section -- Exporter Settings
+  collectors:
+    - heartbeat
+    - mysql.user
+
+secret:
+  # -- (bool) Whether to create a secret to store credentials for this MySQL integration instance.
+  # @default -- `true`
+  # @section -- Secret
+  create:
+  # -- If true, skip secret creation and embed the credentials directly into the configuration.
+  # @section -- Secret
+  embed: false
+  # -- The name of the secret to create.
+  # @section -- Secret
+  name: ""
+  # -- The namespace for the secret.
+  # @section -- Secret
+  namespace: ""
+
+# -- How frequently to scrape metrics from MySQL Exporter.
+# @default -- `60s`
+# @section -- Scrape Settings
+scrapeInterval:
+
+# Adjustments to the scraped metrics to filter the amount of data sent to storage.
+metricsTuning:
+  # -- Metrics to keep. Can use regular expressions.
+  # @section -- Metric Processing Settings
+  includeMetrics: []
+  # -- Metrics to drop. Can use regular expressions.
+  # @section -- Metric Processing Settings
+  excludeMetrics: []
+
+# -- Sets the max_cache_size for prometheus.relabel component.
+# This should be at least 2x-5x your largest scrape target or samples appended rate.
+# ([docs](https://grafana.com/docs/alloy/latest/reference/components/prometheus.relabel/#arguments))
+# Overrides global.maxCacheSize
+# @default -- `100000`
+# @section -- Metric Processing Settings
+maxCacheSize:

charts/feature-integrations/schema-mods/definitions/mysql-integration.schema.json

@@ -0,0 +1,105 @@
+{
+    "type": "object",
+    "properties": {
+        "exporter": {
+            "type": "object",
+            "properties": {
+                "collectors": {
+                    "type": "array",
+                    "items": {
+                        "type": "string"
+                    }
+                },
+                "dataSource": {
+                    "type": "object",
+                    "properties": {
+                        "auth": {
+                            "type": "object",
+                            "properties": {
+                                "password": {
+                                    "type": "string"
+                                },
+                                "passwordFrom": {
+                                    "type": "string"
+                                },
+                                "passwordKey": {
+                                    "type": "string"
+                                },
+                                "username": {
+                                    "type": "string"
+                                },
+                                "usernameFrom": {
+                                    "type": "string"
+                                },
+                                "usernameKey": {
+                                    "type": "string"
+                                }
+                            }
+                        },
+                        "host": {
+                            "type": "string"
+                        },
+                        "port": {
+                            "type": "integer"
+                        }
+                    }
+                },
+                "dataSourceName": {
+                    "type": "string"
+                }
+            }
+        },
+        "logs": {
+            "type": "object",
+            "properties": {
+                "enabled": {
+                    "type": "boolean"
+                },
+                "labelSelectors": {
+                    "type": "object"
+                }
+            }
+        },
+        "maxCacheSize": {
+            "type": "null"
+        },
+        "metricsTuning": {
+            "type": "object",
+            "properties": {
+                "excludeMetrics": {
+                    "type": "array"
+                },
+                "includeMetrics": {
+                    "type": "array"
+                }
+            }
+        },
+        "name": {
+            "type": "string"
+        },
+        "scrapeInterval": {
+            "type": "null"
+        },
+        "secret": {
+            "type": "object",
+            "properties": {
+                "create": {
+                    "type": "null"
+                },
+                "embed": {
+                    "type": "boolean"
+                },
+                "name": {
+                    "type": "string"
+                },
+                "namespace": {
+                    "type": "string"
+                }
+            }
+        },
+        "type": {
+            "type": "string",
+            "const": "mysql"
+        }
+    }
+}

charts/feature-integrations/schema-mods/integration-list.json

@@ -6,7 +6,8 @@
         "anyOf": [
           { "$ref": "#/definitions/alloy-integration"},
           { "$ref": "#/definitions/cert-manager-integration"},
-          { "$ref": "#/definitions/etcd-integration"}
+          { "$ref": "#/definitions/etcd-integration"},
+          { "$ref": "#/definitions/mysql-integration"}
         ]
       }
     }