Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregated stats for queries with most elapsed_time #1086

Merged
merged 2 commits into from
Mar 1, 2024
Merged

Conversation

athoscouto
Copy link
Contributor

@athoscouto athoscouto commented Feb 26, 2024

We need to be able to provide statistics about an individual query.
This is the first step, adding count, elapsed_ms, rows_written, and rows_read.


Those stats are not persisted on purpose, since we want them to be ephemeral and even reset from time to time.

@athoscouto athoscouto changed the title Athos/stats Aggregated stats for queries with most elapsed_time Feb 26, 2024
@athoscouto athoscouto marked this pull request as ready for review February 26, 2024 19:07
@athoscouto
Copy link
Contributor Author

elapsed_ms granularity may be too coarse to use as a sorting weight.
It may be a good idea to fall back to rows_read + rows_written.

libsql-server/src/stats.rs Outdated Show resolved Hide resolved
libsql-server/src/stats.rs Outdated Show resolved Hide resolved
libsql-server/src/stats.rs Outdated Show resolved Hide resolved
libsql-server/src/stats.rs Outdated Show resolved Hide resolved
libsql-server/src/stats.rs Outdated Show resolved Hide resolved
libsql-server/src/stats.rs Outdated Show resolved Hide resolved
libsql-server/src/stats.rs Outdated Show resolved Hide resolved
libsql-server/src/stats.rs Outdated Show resolved Hide resolved
@athoscouto
Copy link
Contributor Author

@LucioFranco it looks like my background worker is making tests hang.
Any idea why? Probably the blocking thread?

libsql-server/src/stats.rs Show resolved Hide resolved
* Record query latency percentiles with hdrhistogram

* Expose queries percentiles through stats response

* Simplify QueriesStats transformation to QueriesStatsResponse

* Reset query stats at the beginning of every hour (#1118)

* Add expires_at to control when QueriesStats should be reset

* Add query count and elapsed sum to query stats response

Also group all latency aggregations together under the elapsed key

* Remove option sprawl on QueriesStats fields

By making the whole struct optional where it is used

* Add created_at to stats queries responde object

* Set queries stats to none when stats is created

This will make the API response the same when no queries
have been recorded on the queries stats struct. This can
happen right after:
- the stats struct initialization
- the stats queries object expiration

Example of stats API response when no queries have been
recorded:

{
  ...
  "queries": null
}

* Use if else instead of early return from stats to response
@athoscouto athoscouto added this pull request to the merge queue Mar 1, 2024
Merged via the queue into main with commit 7f9e289 Mar 1, 2024
16 checks passed
@athoscouto athoscouto deleted the athos/stats branch March 1, 2024 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants