Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: thread autoscaling #1266

Open
wants to merge 182 commits into
base: main
Choose a base branch
from
Open

Conversation

Alliballibaba2
Copy link
Collaborator

I originally wanted to just create a PR that allows adding threads via the admin API, but after letting threads scale automatically, that PR kind of didn't make sense anymore by itself.

So here is what this PR does:

It adds 4 Caddy admin endpoints

POST     /frankenphp/workers/restart   # restarts workers (this can also be put into a smaller PR if necessary)
GET      /frankenphp/threads           # prints the current state of all threads (for debugging/caddytests)
PUT      /frankenphp/threads           # Adds a thread at runtime - accepts 'worker' and 'count' query parameters
DELETE   /frankenphp/threads           # Removes a thread at runtime - accepts 'worker' and 'count' query parameters

Additionally, the PR also introduces a new directive in the config: max_threads.

frankenphp {
    max_threads 200
    num_threads 40
}

If it's bigger than num_threads, worker and regular threads will attempt to autoscale after a request on a few different conditions:

  • no thread was available to immediately handle the request
  • the request was stalled for more than a few ms (15ms currently)
  • no other scaling is happening at that time
  • A CPU probe (50ms) successfully determines that PHP threads are consuming less than a predefined amount of CPU (80% currently)
  • we have not reached max_threads yet

This is all still a WIP. I'm not yet sure if max_threads is the best way to configure autoscaling or if it's even necessary to have the PUT/DELETE endpoints. Maybe it would also make sense to determine max_threads based on available memory.
I'll conduct some benchmarks showing that this approach performs better than default settings in a lot of different scenarios (and makes people worry less about thread configuration).

In regards to recent issues, spawning and destroying threads would also make the server more stable if we're experiencing timeouts (not sure yet how to safely destroy running threads).

@AlliBalliBaba
Copy link
Collaborator

Not yet sure if we should keep the admin endpoints to add and remove threads. While the are nice to have, I'm not sure how useful they actually are. Maybe they would make sense if we allow going above max_threads? That would require locking the phpThreads slice though.

@dunglas
Copy link
Owner

dunglas commented Jan 15, 2025

For the sake of simplicity, I would be to remove these endpoints for now, as long as we haven't a real-life you case for them.

WDYT?

caddy/caddy.go Outdated Show resolved Hide resolved
caddy/caddy_test.go Outdated Show resolved Hide resolved
caddy/watcher_test.go Show resolved Hide resolved
docs/config.md Outdated Show resolved Hide resolved
docs/config.md Outdated Show resolved Hide resolved
frankenphp.c Show resolved Hide resolved
internal/cpu/cpu_fallback.go Outdated Show resolved Hide resolved
internal/cpu/cpu_unix.go Outdated Show resolved Hide resolved
internal/cpu/cpu_unix.go Outdated Show resolved Hide resolved
phpmainthread.go Outdated Show resolved Hide resolved
@AlliBalliBaba
Copy link
Collaborator

I removed the POST and DELETE admin endpoints for now, we can add them back in if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants