Receiving response even after the timeoutSeconds is reached

[karanibm6]

For a knative revision, one can send the response headers and some data before the timeout. It is then seemingly possible to write data longer than the timeout, possibly forever. This is imo not how Knative defines the timeout seconds:

$ kubectl explain ksvc.spec.template.spec.timeoutSeconds
GROUP:      serving.knative.dev
KIND:       Service
VERSION:    v1

FIELD: timeoutSeconds <integer>

DESCRIPTION:
    TimeoutSeconds is the maximum duration in seconds that the request instance
    is allowed to respond to a request. If unspecified, a system default will be
    provided.

What version of Knative?

v1.15.2

Expected Behavior

The request should timeout when the timeoutSeconds is reached.

Actual Behavior

The request doesn't timeout within timeoutSeconds

Steps to Reproduce the Problem

1. Create a KSvc using the image ghcr.io/src2img/http-synthetics:latest (code is from https://github.com/src2img/http-synthetics), Use minScale and maxScale 1. and set timeoutSeconds to 30s
2. Wait for the Ksvc to become ready.
3. Call its endpoint with curl -X PUT http://endpoint/write-regularly?interval=10&count=5

You will observe that you will get the response after 50s. and the request doesn't times out in 30s.

[skonto]

Hi @karanibm6 the timeout is based on whether a write happened or not.
For example if a write does not happen within the timeout period the request expires:

curl -H "Host: http-synthetics.default.example.com" "http://192.168.39.242:31935/write-regularly?interval=40&count=1"
activator request timeout

That is the meaning of "... that the request instance is allowed to respond to a request."

[SaschaSchwarze0]

@skonto thanks for looking into this. Though, what you describe sounds rather like the responseStartTimeoutSeconds timeout:

$ kubectl explain ksvc.spec.template.spec.responseStartTimeoutSeconds
GROUP:      serving.knative.dev
KIND:       Service
VERSION:    v1

FIELD: responseStartTimeoutSeconds <integer>

DESCRIPTION:
    ResponseStartTimeoutSeconds is the maximum duration in seconds that the
    request
    routing layer will wait for a request delivered to a container to begin
    sending any network traffic.

What would be the purpose of timeout if it behaves like responseStartTimeoutSeconds ?

[skonto]

@SaschaSchwarze0 hi,

This is indeed confusing. I see that we have a conformance test:

{
		name:           "writes first byte before timeout",
		timeoutSeconds: 10,
		expectedStatus: http.StatusOK,
		sleep:          15 * time.Second,
		initialSleep:   0,
	}

The logic in the test image does the following:

	// Explicitly flush the already written data to trigger (or not)
	// the time-to-first-byte timeout.
	if f, ok := w.(http.Flusher); ok {
		f.Flush()
	}

	// Sleep for a set amount of time before sending response.
	timeout, _ := strconv.Atoi(r.URL.Query().Get("timeout"))
	time.Sleep(time.Duration(timeout) * time.Millisecond)

	fmt.Fprintf(w, "Slept for %d milliseconds", timeout)

This by design, allows to bypass the timeoutSeconds value as reported in the issue here.

However, in the past we changed the spec back to its original idea of using timeoutSeconds as maxDuration since it had deviated: see #12970 and knative/specs#100. So it seems we should just timeout when the response is not within the timeoutSeconds defined but we don't comply with that? 🤔 cc @dprotaso what is the status here?

[skonto]

I have a PR that tests enforcing the timeout. Note that once we write the header we can't re-write it, but behavior is enforced to kill the connection.

[dprotaso]

So it seems we should just timeout when the response is not within the timeoutSeconds defined but we don't comply with that

The request should be killed - I'm guessing we should track if we've written a header and not do it again