Prometheus Golang Exporter from log file (with pre-configured grafana dashboard)

What it does

---

fetch and register metrics from configs from "env.yml" file. then set and update value of metrics based on logs line by line once new log is added.

Note: you see an .env file too. you can use it when you don't want to use yml based env file or just want to have a single ".env" file for the whole your project. so you can use .env file and change code to use ".env" based methods which are defined in "internal/env/env.go"

Note: by default it uses Guage metric type (more common) but you can add other metric types in "internal/logmetrics/logmetrics.go" as you wish

env structure:

• logs : define logs levels which should be pushed to prometheus as metric
• confs : name / log file address / regex pattern to fetch fields from logs

yaml file example:

logs:
  - "response"
confs:
  - name: "response"
    file: "../logs/responses.log"
    regex: 'path:(?P<L_responseTime_path>[^|]*).*response_time:(?P<M_responseTime_gauge>[^|]*).*status:(?P<L_responseTime_status>[^|]*)'

log example based on previous regex pattern:

2022-12-12T11:53:35|response_logger|INFO|request_id:1670846006290592061|m_responseTime_gauge|path:/api/v1/auth/otp/login/|body:"{\"detail\":\"There was an error parsing the body\"}"|response_time:60|status:200

generated metric example:

Usage

---

Prerequisite

1. clone the project

2. Build required docker image for exporter :

   run this command to build "golangprom-exporter:1.0" image from dockerfile

   docker build -t golangprom-exporter:1.0 .
   

   ! you can tag image to any name you would but you have to change image name of go-exporter service in compose file after that.

3. Install docker-compose

4. Prepare env yaml file acording to env structure

   regex groups definition role:

   • metrics: M_< metric name >
   • labels: L_< metric name >_< label name >

   NOTE: define regex patten inside '' to avoid getting unknown escape character error

NOTE : you can skip next configurations and use default configurations of project

5. Prepare prometheus configs yaml file (optional) :
   as example:

   global:
     scrape_interval: 1s
     evaluation_interval: 2s
     scrape_timeout: 0s900ms
   scrape_configs:
     # processing prometheus service metrics
     - job_name: prometheus
       static_configs:
         - targets: ["localhost:9090"] # address of prometheus service
     # processing golang exporter metrics
     - job_name: logging
       static_configs:
         - targets: ["go-exporter:3030"] # address of exporter service
   

if want to use pre-configured grafana dashboard continue these steps 6. Prepare grafana datasource config file (optional) :
define prometheus instance to fetch data. change parameters in grafana/datasources/datasource.yml to your desire config or just edit url of prometheus instance > url: http://prometheus-go-exporter:9090 7. Prepare grafana dashboard config file (optional) :

• grafana/datasources/dashboards/dashboard.yml : specify datasource which defined in previous step to read visualization data from it.
  as example:

apiVersion: 1
providers:
  - name: 'Prometheus' # datasource name
    orgId: 1
    folder: ''
    type: file
    disableDeletion: false
    editable: true
    options:
      path: /etc/grafana/provisioning/dashboards

• grafana/datasources/dashboards/*.json : json file of configured dashboard (can design dashboard in grafana panel and export it as json file).
  define this file as "GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH" compose environment variable to show it as home dashboard instead of grafana default home dashboard.

Run

---

place "logs" folder in outer directory from project directory ( or change mount path in compose file )

then run this command in project directory to services start working

docker-compose up -d