feat: measure external latency

[zepatrik]

This adds a middleware and utilities to measure external latency.

[alnr]

Does this have to be a middleware? This would work much more simply and be less tightly coupled if we defined a public Context key in the reqlog package which gives a *time.Duration, to which external calls can add the time they took.

[alnr]

Please let's not couple this functionality to the SSRF protection mechanism. We should allow users to opt into the MeasureExternalLatencyTransport if they wish. It's as easy as wrapping the transport at the site of res, err := client.Do(req).

[zepatrik]

This is still opt-in, because the transport is just a no-op when there is no "container" in the context. However, it does make sense to not do this per default for the httpx client.
I was at first thinking about only measuring the external latency on the "prohibitInternal*" variants of this transport, but as it is right now it does not make sense.

[zepatrik]

We will always need some kind of middleware to add the data container to the context. As is, the context key is not exposed but there are easy functions to add or get to the data container, I will also add a way to add the data container to a context without the middleware wrapping.

[zepatrik]

This is still opt-in, because the transport is just a no-op when there is no "container" in the context. However, it does make sense to not do this per default for the httpx client.
I was at first thinking about only measuring the external latency on the "prohibitInternal*" variants of this transport, but as it is right now it does not make sense.

[alnr]

We will always need some kind of middleware to add the data container to the context. As is, the context key is not exposed but there are easy functions to add or get to the data container, I will also add a way to add the data container to a context without the middleware wrapping.

Can we add the context key in the existing reqlog package?

[zepatrik]

Can we add the context key in the existing reqlog package?

What's the benefit of exposing the context key? Or do you mean add a context key for the overall latency as well?

[alnr]

This second iteration of the API is a lot better. 👍

IMO, this would be better with a such an API:

func SendWebhook() (...) {
  req, err := http.NewRequestWithContext(ctx, ...)

  defer reqlog.IncrExternalLatency(ctx, time.Now())
  res, err := client.Do(req)
  // maybe retries

  return res, err
}

Alternative usage:

func SendWebhook() (...) {
  req, err := http.NewRequestWithContext(ctx, ...)

  t0 := time.Now()
  res, err := client.Do(req)
  // maybe retries
  reqlog.IncrExternalLatency(ctx, t0)

  // time-consuming post-processing
  return res, err
}

[alnr]

Just re-read the code and this already exists in your API essentially 😅