Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(pubsub): additional buckets to consume_latency prom metrics #697

Merged
merged 1 commit into from
Mar 14, 2024
Merged

chore(pubsub): additional buckets to consume_latency prom metrics #697

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CONTRIBUTING.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ may need to keep this in mind while developing; all code will need to be
generateable into both module trees and runnable without asyncio (though there
are some conversion scripts we run to make things a bit more ergonomic --
you're generally safe to write standard asyncio code and rely on the
conversation script). Feel free to ping a maintainer if you're having issues
conversion script). Feel free to ping a maintainer if you're having issues
with this!

To build and test against the ``gcloud-rest`` projects, you can run:
Expand Down
2 changes: 2 additions & 0 deletions pubsub/gcloud/aio/pubsub/metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
namespace=_NAMESPACE,
subsystem=_SUBSYSTEM,
unit='seconds',
buckets=(.01, .1, .25, .5, 1.0, 2.5, 5.0, 7.5, 10.0, 20.0,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The defaullt buckets seem to be:

from prometheus_client import Histogram

# Create a Histogram without specifying buckets to see the default ones
histogram = Histogram('request_duration_seconds', 'Description of histogram', unit='seconds')

# Access the default buckets
default_buckets = histogram._upper_bounds
default_buckets

Results:

[0.005,
 0.01,
 0.025,
 0.05,
 0.075,
 0.1,
 0.25,
 0.5,
 0.75,
 1.0,
 2.5,
 5.0,
 7.5,
 10.0,
 inf]

Given the 2 plus minutes we're being reported, we might even need to have bigger buckets, say, 60, 90 and 120.

Copy link
Contributor

@shaundialpad shaundialpad Mar 14, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be fine going up to 2 minutes as well. I'm not sure the 90 second bucket is necessarily useful. For our purposes, if we're something like 30 seconds behind then we're definitely way too far behind already, I think we just wanted to have a better chance at capturing more accurate upper bounds (or perhaps more accurately, non-infinite upper bounds!). Also good to keep in mind this applies to all of our apps, plus the apps of our users who choose to make use of the metrics.

30.0, 60.0, 120.0, float('inf')),
)

BATCH_STATUS = prometheus_client.Counter(
Expand Down
2 changes: 1 addition & 1 deletion pubsub/poetry.rest.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# This file is automatically @generated by Poetry 1.8.2 and should not be changed by hand.
# This file is automatically @generated by Poetry 1.8.1 and should not be changed by hand.

[[package]]
name = "backoff"
Expand Down
2 changes: 1 addition & 1 deletion pubsub/pyproject.rest.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "gcloud-rest-pubsub"
version = "6.0.0"
version = "6.0.1"
description = "Python Client for Google Cloud Pub/Sub"
readme = "README.rst"

Expand Down
2 changes: 1 addition & 1 deletion pubsub/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "gcloud-aio-pubsub"
version = "6.0.0"
version = "6.0.1"
description = "Python Client for Google Cloud Pub/Sub"
readme = "README.rst"

Expand Down