Merge pull request #45839 from brunobat/micrometer-to-otel-bridge-das…

…hboard

Grafana dashboard improvements - part 3

docs/src/main/asciidoc/observability-devservices-lgtm.adoc

@@ -4,14 +4,14 @@ and pull requests should be submitted there:
 https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
 ////
 = Observability Dev Services with Grafana OTel LGTM
-
 include::_attributes.adoc[]
 :categories: observability,devservices,telemetry,metrics,tracing,logging, opentelemetry, micrometer, prometheus, tempo, loki, grafana
 :summary: Instructions on how to use Grafana Otel LGTM
 :topics: observability,grafana,lgtm,otlp,opentelemetry,devservices,micrometer
 :extensions: io.quarkus:quarkus-observability-devservices
 
-https://github.com/grafana/docker-otel-lgtm[OTel-LGTM] is `all-in-one` Docker image containing OpenTelemetry's https://github.com/open-telemetry/opentelemetry-proto/blob/main/docs/README.md[OTLP] as the protocol to transport metrics, tracing and logging data to an https://opentelemetry.io/docs/collector[OpenTelemetry Collector] which then stores signals data into https://prometheus.io/[Prometheus] (metrics), https://github.com/grafana/tempo[Tempo] (traces) and https://github.com/grafana/loki[Loki] (logs), only to have it visualized by https://github.com/grafana/grafana[Grafana]. It's used by Quarkus Observability to provide the Grafana OTel LGTM Dev Resource.
+This Dev Service provides the https://github.com/grafana/docker-otel-lgtm[Grafana OTel-LGTM], an `all-in-one` Docker image containing an https://opentelemetry.io/docs/collector[OpenTelemetry Collector] receiving and then forwarding telemetry data to https://prometheus.io/[Prometheus] (metrics), https://github.com/grafana/tempo[Tempo] (traces) and https://github.com/grafana/loki[Loki] (logs).
+This data can then be visualized by https://github.com/grafana/grafana[Grafana].
 
 == Configuring your project
 
@@ -33,9 +33,35 @@ Add the Quarkus Grafana OTel LGTM sink (where data goes) extension to your build
 implementation("io.quarkus:quarkus-observability-devservices-lgtm")
 ----
 
-=== Metrics
+=== Micrometer
+
+The https://quarkus.io/guides/telemetry-micrometer[Micrometer Quarkus extension] provides metrics from automatic instrumentation implemented in Quarkus and its extensions.
+
+There are multiple ways to output Micrometer metrics. Next there are some examples:
+
+==== Using the Micrometer Prometheus registry
+
+This is the most common way to output metrics from Micrometer and the default way in Quarkus. The Micrometer Prometheus registry will publish data in the `/q/metrics` endpoint and a scraper inside the Grafana LGTM Dev Service will grab it (*pull* data from the service).
 
-If you need metrics, add the Micrometer OTLP registry to your build file:
+
+[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
+.pom.xml
+----
+<dependency>
+    <groupId>io.quarkiverse.micrometer.registry</groupId>
+    <artifactId>quarkus-micrometer-registry-prometheus</artifactId>
+</dependency>
+----
+
+[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
+.build.gradle
+----
+implementation("io.quarkiverse.micrometer.registry:quarkus-micrometer-registry-prometheus")
+----
+
+==== Using the Micrometer OTLP registry
+
+The https://docs.quarkiverse.io/quarkus-micrometer-registry/dev/micrometer-registry-otlp.html[Quarkiverse Micrometer OTLP registry] will output data using the OpenTelemetry OTLP protocol to the Grafana LGTM Dev Service. This will *push* data out of the service:
 
 [source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
 .pom.xml
@@ -52,11 +78,15 @@ If you need metrics, add the Micrometer OTLP registry to your build file:
 implementation("io.quarkiverse.micrometer.registry:quarkus-micrometer-registry-otlp")
 ----
 
-When using the https://micrometer.io/[MicroMeter's] Quarkiverse OTLP registry to push metrics to Grafana OTel LGTM, the `quarkus.micrometer.export.otlp.url` property is automatically set to OTel collector endpoint as seen from the outside of the docker container.
+When using the https://micrometer.io/[Micrometer's] Quarkiverse OTLP registry to push metrics to Grafana OTel LGTM, the `quarkus.micrometer.export.otlp.url` property is automatically set to OTel collector endpoint as seen from the outside of the Docker container.
+
+=== OpenTelemetry
 
-=== Tracing
+With OpenTelemetry, metrics, traces and logs can be created and sent to the Grafana LGTM Dev Service.
 
-For Tracing add the `quarkus-opentelemetry` extension to your build file:
+By default, the https://quarkus.io/guides/opentelemetry[OpenTelemetry extension] will produce https://quarkus.io/guides/opentelemetry-tracing[traces]. https://quarkus.io/guides/opentelemetry-metrics[Metrics] and https://quarkus.io/guides/opentelemetry-logging[logs] must be enabled separately.
+
+The `quarkus-opentelemetry` extension can be added to your build file like this:
 [source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
 .pom.xml
 ----
@@ -72,11 +102,36 @@ For Tracing add the `quarkus-opentelemetry` extension to your build file:
 implementation("io.quarkus:quarkus-opentelemetry")
 ----
 
-The `quarkus.otel.exporter.otlp.endpoint` property is automatically set to OTel collector endpoint as seen from the outside of the docker container.
+The `quarkus.otel.exporter.otlp.endpoint` property is automatically set to the OTel collector endpoint as seen from the outside of the Docker container.
 
 The `quarkus.otel.exporter.otlp.protocol` is set to `http/protobuf`.
 
-=== Access Grafana
+=== Micrometer to OpenTelemetry bridge
+
+This extension provides Micrometer metrics and OpenTelemetry metrics, traces and logs. All data is managed and sent out by the OpenTelemetry extension.
+
+**All signals are enabled by default.**
+
+The extension can be added to your build file like this:
+
+[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
+.pom.xml
+----
+<dependency>
+    <groupId>io.quarkus</groupId>
+    <artifactId>quarkus-micrometer-opentelemetry</artifactId>
+</dependency>
+----
+
+[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
+.build.gradle
+----
+implementation("io.quarkus:quarkus-micrometer-opentelemetry")
+----
+
+== Grafana
+
+=== Grafana UI access
 
 Once you start your app in dev mode:
 
@@ -87,19 +142,41 @@ You will see a log entry like this:
 [source, log]
 ----
 [io.qu.ob.de.ObservabilityDevServiceProcessor] (build-35) Dev Service Lgtm started, config: {grafana.endpoint=http://localhost:42797, quarkus.otel.exporter.otlp.endpoint=http://localhost:34711, otel-collector.url=localhost:34711, quarkus.micrometer.export.otlp.url=http://localhost:34711/v1/metrics, quarkus.otel.exporter.otlp.protocol=http/protobuf}
-
 ----
 Remember that Grafana is accessible in an ephemeral port, so you need to check the logs to see which port is being used. In this example, the grafana endpoint is `grafana.endpoint=http://localhost:42797`.
 
-If you miss the message you can always check the port with this Docker command:
-[source, bash]
-----
-docker ps | grep grafana
-----
+Another option is to use the *Dev UI* (http://localhost:8080/q/dev-ui/extensions) as the Grafana URL link will be available and if selected it will open a new browser tab directly to the running Grafana instance:
+
+image::dev-ui-observability-card.png[alt=Dev UI LGTM, align=center,width=50%]
+
+=== Explore
+
+In the explore section, you can query the data for all the data sources.
+
+To see traces, select the `tempo` data source and query for data:
+
+image::observability-grafana-tempo.png[alt=Dev UI LGTM, align=center,width=90%]
+
+For logs, select the `loki` data source and query for data:
+
+image::observability-grafana-loki.png[alt=Dev UI LGTM, align=center,width=90%]
+
+=== The dashboards
+
+The Dev Service includes a set of dashboards for metrics.
+
+image::observability-grafana-dashboards.png[alt=Dev UI LGTM, align=center,width=90%]
+
+Each dashboard is tuned for the specific application setup. The available dashboards are:
 
-Another option is to use the Dev UI as the Grafana URL link will be available and if selected will open a new browser tab directly to the running Grafana instance:
+* *Quarkus Micrometer Metrics - OTLP*: to be used with the Micrometer OTLP registry extension
+* *Quarkus Micrometer Metrics - Prometheus*: to be used with the Micrometer Prometheus registry extension.
+* *Quarkus Micrometer OpenTelemetry*: to be used with the Micrometer to OpenTelemetry bridge extension.
 
-image::dev-ui-observability-card.png[alt=Dev UI LGTM, align=center,width=80%]
+[NOTE]
+====
+Some panels in the dashboards might take a few minutes to show data when their values are calculated over a sliding window of time.
+====
 
 === Additional configuration
 

extensions/observability-devservices/testcontainers/src/main/java/io/quarkus/observability/testcontainers/LgtmContainer.java

@@ -71,6 +71,11 @@ public class LgtmContainer extends GrafanaContainer<LgtmContainer, LgtmConfig> {
                 options:
                   path: /otel-lgtm/grafana-dashboard-quarkus-micrometer-otlp.json
                   foldersFromFilesStructure: false
+              - name: "Quarkus Micrometer OpenTelemetry"
+                type: file
+                options:
+                  path: /otel-lgtm/grafana-dashboard-quarkus-micrometer-opentelemetry.json
+                  foldersFromFilesStructure: false
             """;
 
     public LgtmContainer() {
@@ -91,6 +96,9 @@ public LgtmContainer(LgtmConfig config) {
         withCopyFileToContainer(
                 MountableFile.forClasspathResource("/grafana-dashboard-quarkus-micrometer-otlp.json"),
                 "/otel-lgtm/grafana-dashboard-quarkus-micrometer-otlp.json");
+        withCopyFileToContainer(
+                MountableFile.forClasspathResource("/grafana-dashboard-quarkus-micrometer-opentelemetry.json"),
+                "/otel-lgtm/grafana-dashboard-quarkus-micrometer-opentelemetry.json");
         addFileToContainer(getPrometheusConfig().getBytes(), "/otel-lgtm/prometheus.yaml");
 
     }