Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Allows saving of HTTP events into PostgreSQL #150

Merged
merged 5 commits into from
Oct 30, 2024

Conversation

andyquinterom
Copy link
Member

@andyquinterom andyquinterom commented Oct 26, 2024

This feature will allow us track the number of requests, response times and other data for plumber APIs and Shiny Applications.

The new feature

HTTP Telemetry

faucet now offers the option of saving HTTP events to a PostgreSQL database.
This can be very helpful for tracking latency, total API calls and other
important information.

In order to use this feature you will need a PostgreSQL database with a table
called faucet_http_events. You can create the table using this table with
the following SQL query:

CREATE TABLE faucet_http_events (
    request_uuid UUID,
    namespace TEXT,
    version TEXT,
    target TEXT,
    worker_route TEXT,
    worker_id INT,
    ip_addr INET,
    method TEXT,
    path TEXT,
    query_params TEXT,
    http_version TEXT,
    status SMALLINT,
    user_agent TEXT,
    elapsed BIGINT,
    time TIMESTAMPTZ
);

Connection Strings

In order to connect to the database you will need to pass the
FAUCET_TELEMETRY_POSTGRES_STRING environment variable or the
--pg-con-string CLI argument.

This should include either a connection string or a URL with the postgres://
protocol.

Example connection strings

FAUCET_TELEMETRY_POSTGRES_STRING="host=localhost user=postgres connect_timeout=10 keepalives=0"
FAUCET_TELEMETRY_POSTGRES_STRING="host=/var/lib/postgresql,localhost port=1234 user=postgres password='password with spaces'"
FAUCET_TELEMETRY_POSTGRES_STRING="postgresql://user@localhost"
FAUCET_TELEMETRY_POSTGRES_STRING="postgresql://user:[email protected]/mydb?connect_timeout=10"

Telemetry Namespaces

It is likely you want to track different services on the same database. You
can control the column namespace using the environment variable
FAUCET_TELEMETRY_NAMESPACE or cli argument --telemetry-namespace. By
default, this value is "faucet"

@andyquinterom
Copy link
Member Author

Video showcase: https://youtu.be/VpmVc6hI-F4

@andyquinterom
Copy link
Member Author

Example values in the table:

   namespace    |  target   | worker_route | worker_id |  ip_addr  | method |     path      |   query_params    | version  | status |       user_agent        | elapsed |             time
----------------+-----------+--------------+-----------+-----------+--------+---------------+-------------------+----------+--------+-------------------------+---------+-------------------------------
 simple-plumber | Worker::1 |              |         1 | 127.0.0.1 | GET    | /__docs__/    |                   | HTTP/1.1 |    200 | Mozilla/5.0 (X11; Li... |      49 | 2024-10-27 01:07:30.827733+00
 simple-plumber | Worker::1 |              |         1 | 127.0.0.1 | GET    | /openapi.json |                   | HTTP/1.1 |    200 | Mozilla/5.0 (X11; Li... |      93 | 2024-10-27 01:07:31.024664+00
 simple-plumber | Worker::1 |              |         1 | 127.0.0.1 | GET    | /echo         | msg=Hello%20World | HTTP/1.1 |    200 | Mozilla/5.0 (X11; Li... |       4 | 2024-10-27 01:07:38.186839+00
 simple-plumber | Worker::1 |              |         1 | 127.0.0.1 | GET    | /plot         | spec=setosa       | HTTP/1.1 |    200 | Mozilla/5.0 (X11; Li... |      45 | 2024-10-27 01:07:47.522051+00

@andyquinterom andyquinterom marked this pull request as draft October 27, 2024 02:05
@andyquinterom andyquinterom force-pushed the faucet_telemetry branch 2 times, most recently from 990a76d to ab38048 Compare October 28, 2024 05:01
@FvD
Copy link
Member

FvD commented Oct 30, 2024

This is great. I went 🤯 with the telemetry demo, and I am at least as impressed by how you implemented it. Please change the draft status at your earliest convenience so that I can merge the PR.

@andyquinterom andyquinterom marked this pull request as ready for review October 30, 2024 02:59
@FvD FvD merged commit b215aef into ixpantia:main Oct 30, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a way to save http events into postgresql
2 participants