Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: thread autoscaling #1266

Open
wants to merge 196 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 171 commits
Commits
Show all changes
196 commits
Select commit Hold shift + click to select a range
fe1158f
Decouple workers.
Alliballibaba2 Nov 1, 2024
ad34140
Moves code to separate file.
Alliballibaba2 Nov 1, 2024
89b211d
Cleans up the exponential backoff.
Alliballibaba2 Nov 2, 2024
7d2ab8c
Initial working implementation.
Alliballibaba2 Nov 2, 2024
f7e7d41
Refactors php threads to take callbacks.
Alliballibaba2 Nov 2, 2024
c03c59b
Cleanup.
Alliballibaba2 Nov 2, 2024
a9857dc
Cleanup.
Alliballibaba2 Nov 2, 2024
bac9555
Cleanup.
Alliballibaba2 Nov 2, 2024
a2f8d59
Cleanup.
Alliballibaba2 Nov 2, 2024
279924c
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Nov 3, 2024
0825453
Adjusts watcher logic.
Alliballibaba2 Nov 3, 2024
17d5cbe
Adjusts the watcher logic.
Alliballibaba2 Nov 3, 2024
09e0ca6
Fix opcache_reset race condition.
Alliballibaba2 Nov 4, 2024
a726a2c
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Nov 4, 2024
7f13ada
Fixing merge conflicts and formatting.
Alliballibaba2 Nov 4, 2024
13fb4bb
Prevents overlapping of TSRM reservation and script execution.
Alliballibaba2 Nov 5, 2024
a8a00c8
Adjustments as suggested by @dunglas.
Alliballibaba2 Nov 5, 2024
b4dd138
Adds error assertions.
Alliballibaba2 Nov 5, 2024
03f98fa
Adds comments.
Alliballibaba2 Nov 5, 2024
e52dd0f
Removes logs and explicitly compares to C.false.
Alliballibaba2 Nov 5, 2024
cd98e33
Resets check.
Alliballibaba2 Nov 5, 2024
4e2a2c6
Adds cast for safety.
Alliballibaba2 Nov 5, 2024
c51eb93
Fixes waitgroup overflow.
Alliballibaba2 Nov 5, 2024
89d8e26
Resolves waitgroup race condition on startup.
Alliballibaba2 Nov 6, 2024
3587243
Moves worker request logic to worker.go.
Alliballibaba2 Nov 7, 2024
ec32f0c
Removes defer.
Alliballibaba2 Nov 7, 2024
4e35698
Removes call from go to c.
Alliballibaba2 Nov 11, 2024
740fac7
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Nov 15, 2024
8a272cb
Fixes merge conflict.
Alliballibaba2 Nov 15, 2024
ecce5d5
Adds fibers test back in.
Alliballibaba2 Nov 15, 2024
06ebd67
Refactors new thread loop approach.
Alliballibaba2 Nov 15, 2024
c811f4a
Removes redundant check.
Alliballibaba2 Nov 16, 2024
6bd047a
Adds compareAndSwap.
Alliballibaba2 Nov 16, 2024
55ad8ba
Refactor: removes global waitgroups and uses a 'thread state' abstrac…
Alliballibaba2 Nov 17, 2024
3ffbe06
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Nov 17, 2024
01ed92b
Removes unnecessary method.
Alliballibaba2 Nov 17, 2024
790cccc
Updates comment.
Alliballibaba2 Nov 17, 2024
0dd2605
Removes unnecessary booleans.
Alliballibaba2 Nov 18, 2024
60a66b4
test
Alliballibaba2 Nov 24, 2024
4719fa8
First state machine steps.
Alliballibaba2 Nov 25, 2024
f72e8cb
Splits threads.
Alliballibaba2 Dec 6, 2024
d20e706
Minimal working implementation with broken tests.
Alliballibaba2 Dec 6, 2024
6747d15
Fixes tests.
Alliballibaba2 Dec 6, 2024
54dc267
Refactoring.
Alliballibaba2 Dec 7, 2024
660efcd
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Dec 7, 2024
6214754
Fixes merge conflicts.
Alliballibaba2 Dec 7, 2024
00eb834
Formatting
Alliballibaba2 Dec 7, 2024
02b73b1
C formatting.
Alliballibaba2 Dec 7, 2024
421904e
More cleanup.
Alliballibaba2 Dec 7, 2024
cca2a00
Allows for clean state transitions.
Alliballibaba2 Dec 7, 2024
ec8aeb7
Adds state tests.
Alliballibaba2 Dec 7, 2024
b598bd3
Adds support for thread transitioning.
Alliballibaba2 Dec 7, 2024
66cedbd
Fixes the testdata path.
Alliballibaba2 Dec 7, 2024
06af5d5
Formatting.
Alliballibaba2 Dec 7, 2024
71c16bc
Allows transitioning back to inactive state.
Alliballibaba2 Dec 7, 2024
5095342
Fixes go linting.
Alliballibaba2 Dec 7, 2024
4b18059
Formatting.
Alliballibaba2 Dec 7, 2024
15429d9
Removes duplication.
Alliballibaba2 Dec 7, 2024
c080608
Applies suggestions by @dunglas
Alliballibaba2 Dec 7, 2024
a166b87
Removes redundant check.
Alliballibaba2 Dec 7, 2024
9491e6b
Locks the handler on restart.
Alliballibaba2 Dec 7, 2024
e795c86
Removes unnecessary log.
Alliballibaba2 Dec 8, 2024
68fa124
Adds frankenphp admin api.
Alliballibaba2 Dec 8, 2024
b6cbfae
Allows booting threads at runtime.
Alliballibaba2 Dec 8, 2024
f185279
Adds proper admin status codes and tests.
Alliballibaba2 Dec 8, 2024
ea0a4fe
Makes config smaller.
Alliballibaba2 Dec 8, 2024
fcb5f8c
Adds max threads option and debug status.
Alliballibaba2 Dec 8, 2024
a43ecbe
Adds test with debug message.
Alliballibaba2 Dec 8, 2024
b117bff
Formatting and comments.
Alliballibaba2 Dec 8, 2024
ef1bd0d
Changes Unpin() logic as suggested by @withinboredom
Alliballibaba2 Dec 9, 2024
8cd9061
Allows scaling regular threads.
Alliballibaba2 Dec 9, 2024
9e8d8f0
Only allows POST requests.
Alliballibaba2 Dec 9, 2024
a8a4545
Adds suggestions by @dunglas and resolves TODO.
Alliballibaba2 Dec 10, 2024
23a6362
Makes restarts fully safe.
Alliballibaba2 Dec 10, 2024
62224bf
Merge branch 'refactor/start-worker-threads-directly' into feat/worke…
Alliballibaba2 Dec 10, 2024
18e3e58
Will make the initial startup fail even if the watcher is enabled (as…
Alliballibaba2 Dec 10, 2024
3672c60
Also adds compareAndSwap to the test.
Alliballibaba2 Dec 10, 2024
38f87b7
Adds comment.
Alliballibaba2 Dec 10, 2024
96265d3
Merge branch 'refactor/start-worker-threads-directly' into feat/worke…
Alliballibaba2 Dec 10, 2024
d97ebfe
Prevents panic on initial watcher startup.
Alliballibaba2 Dec 10, 2024
950a509
Merge branch 'refactor/start-worker-threads-directly' into feat/worke…
Alliballibaba2 Dec 11, 2024
5f1ec1f
Cleans up admin endpoints.
Alliballibaba2 Dec 11, 2024
7c61dfa
Fixes admin test.
Alliballibaba2 Dec 11, 2024
2af993e
Boots a thread in a test.
Alliballibaba2 Dec 11, 2024
547139f
Sets more explicit max_threads.
Alliballibaba2 Dec 11, 2024
c8bf1ec
Adjusts naming.
Alliballibaba2 Dec 12, 2024
7f2b94e
Adds docs.
Alliballibaba2 Dec 12, 2024
df78254
Changes logic to actually terminate the thread.
Alliballibaba2 Dec 12, 2024
ec0bc0f
Removes the test's randomness.
Alliballibaba2 Dec 13, 2024
8f10407
Adds comments.
Alliballibaba2 Dec 13, 2024
91c324d
Adds comments.
Alliballibaba2 Dec 13, 2024
ff06bd7
Scaling v1.
Alliballibaba2 Dec 15, 2024
50ba106
Scaling v2.
Alliballibaba2 Dec 15, 2024
bfe3de1
Allows regular thread scaling.
Alliballibaba2 Dec 15, 2024
e9f62b9
Refactors wait-time.
Alliballibaba2 Dec 15, 2024
21949dd
Explicitly requires setting max_threads.
Alliballibaba2 Dec 15, 2024
39a7fc9
Removes redundant check.
Alliballibaba2 Dec 15, 2024
442a558
Removes unnecessary import.
Alliballibaba2 Dec 15, 2024
c213fc9
Records clock time.
Alliballibaba2 Dec 17, 2024
58daaa5
Saves CPU metrics of last 100 requests.
Alliballibaba2 Dec 17, 2024
14925f6
Integrates CPU tracking.
Alliballibaba2 Dec 18, 2024
d408bdd
Replaces clock with probing.
Alliballibaba2 Dec 19, 2024
8fc3293
fmt.
Alliballibaba2 Dec 19, 2024
0314247
Adds autoscale tests.
Alliballibaba2 Dec 19, 2024
bd4af11
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Dec 19, 2024
dc10546
Merges main.
Alliballibaba2 Dec 19, 2024
3b9f577
Fixes alpine (probably)
Alliballibaba2 Dec 19, 2024
790ce4e
Fixes alpine (definitely)
Alliballibaba2 Dec 19, 2024
29de62a
go fmt
Alliballibaba2 Dec 19, 2024
b447412
Removes unnecessary 'isProtected'
Alliballibaba2 Dec 19, 2024
6fa90d6
Adds perf tests.
Alliballibaba2 Dec 20, 2024
3bd7c76
Adds request status message to thread debug status.
Alliballibaba2 Dec 21, 2024
45cd915
Adjusts performance tests.
Alliballibaba2 Dec 21, 2024
af40470
Adds an exponential backoff on request overflow.
Alliballibaba2 Dec 21, 2024
d5e8f86
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Dec 21, 2024
c7acb25
changes dir.
Alliballibaba2 Dec 21, 2024
8c22cbf
Linting and formatting.
Alliballibaba2 Dec 22, 2024
745b29b
Linting and formatting.
Alliballibaba2 Dec 22, 2024
68ae2e4
Adds explicit scaling tests.
Alliballibaba2 Dec 22, 2024
09a5caf
Adjusts perf tests.
Alliballibaba2 Dec 22, 2024
3cfcb11
Uses different worker in removal test.
Alliballibaba2 Dec 22, 2024
cbe45fc
More formatting fixes.
Alliballibaba2 Dec 22, 2024
1d8e973
Replaces inline errors and adjusts comments.
Alliballibaba2 Dec 22, 2024
bf48b14
Formatting.
Alliballibaba2 Dec 22, 2024
4f0cc8a
Formatting.
Alliballibaba2 Dec 22, 2024
d483baf
Formatting.
Alliballibaba2 Dec 22, 2024
d222520
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Dec 22, 2024
4807947
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Dec 23, 2024
eef7815
Implements suggestions by @dunglas.
Alliballibaba2 Dec 25, 2024
601a43a
Adds note.
Alliballibaba2 Dec 26, 2024
971e1dc
Formatting.
Alliballibaba2 Dec 26, 2024
f4f9576
suggestion: Refactor scaling strategy (#1289)
withinboredom Dec 29, 2024
6ec9ad9
Merge branch 'feat/auto-scale-clock-time' of https://github.com/dungl…
Alliballibaba2 Dec 29, 2024
5282e32
Resets strategy and fixes tests.
Alliballibaba2 Dec 29, 2024
83d8c11
Adds scaling strategies.
Alliballibaba2 Dec 29, 2024
c094680
Fixes test reloading.
Alliballibaba2 Dec 29, 2024
3ba080a
Prevents threads from respawning after shutdown.
Alliballibaba2 Dec 30, 2024
7934b50
Adjusts transition tests.
Alliballibaba2 Dec 30, 2024
eb5e76d
Simplifies scaling.
Alliballibaba2 Jan 9, 2025
f33aaad
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Jan 9, 2025
5f70afb
Shows the original request path in debug status.
Alliballibaba2 Jan 9, 2025
9dcc2cd
Adds automatic thread limit calculation.
Alliballibaba2 Jan 10, 2025
ed90b75
Properly forwards automatic max threads.
Alliballibaba2 Jan 10, 2025
d047149
Adds ini overrides.
Alliballibaba2 Jan 10, 2025
5bdd50c
Adds Caddy php ini directive (for testing)
Alliballibaba2 Jan 10, 2025
936556b
Adds explicit nil check
Alliballibaba2 Jan 10, 2025
6758903
Adds php.ini override test.
Alliballibaba2 Jan 10, 2025
548d095
Fixes linting and conversions.
Alliballibaba2 Jan 10, 2025
2761d9c
Fixes fuzzing tests.
Alliballibaba2 Jan 10, 2025
36d3f54
clang-format
Alliballibaba2 Jan 10, 2025
cd5c905
Checks for original request existence on debug status.
Alliballibaba2 Jan 10, 2025
98f2279
Changes the memory calculation implementation
Alliballibaba2 Jan 10, 2025
a16de66
Renames var.
Alliballibaba2 Jan 10, 2025
6a6d040
Reduces the stall-time.
Alliballibaba2 Jan 10, 2025
83e0c08
Adjusts scaling logic and comments.
Alliballibaba2 Jan 10, 2025
79447b8
Swaps to unix sys package.
Alliballibaba2 Jan 10, 2025
0545e1b
Only supports CPU probing on unix and SysMemory on Linux.
Alliballibaba2 Jan 11, 2025
615662e
Adjusts timers and passes the main thread directly to scaling.
Alliballibaba2 Jan 11, 2025
618345b
Properly overrides ini settings on startup.
Alliballibaba2 Jan 12, 2025
a8b2c7c
clang-format
Alliballibaba2 Jan 12, 2025
9e4a9ca
Adds php_ini error.
Alliballibaba2 Jan 12, 2025
ca20704
Adds ini config.
Alliballibaba2 Jan 12, 2025
3e20e8f
Moves ini overrides to #else.
Alliballibaba2 Jan 15, 2025
7ce4c09
Adds some max_threads docs.
Alliballibaba2 Jan 15, 2025
2b1427b
Shows special debug message if in-between worker requests.
Alliballibaba2 Jan 15, 2025
d3df9f9
Refactors ini overrides.
Alliballibaba2 Jan 15, 2025
1df0c9b
formatting.
Alliballibaba2 Jan 15, 2025
e51c07d
Changes TODO.
Alliballibaba2 Jan 15, 2025
d43f6f0
trigger
Alliballibaba2 Jan 15, 2025
9c1c8a8
linting
Alliballibaba2 Jan 15, 2025
825735a
Resets Caddyfile.
Alliballibaba2 Jan 15, 2025
7de0972
phpIniOverrides -> phpIni
Alliballibaba2 Jan 16, 2025
54b28ac
Removes admin endpoints to PUT and DELETE threads.
Alliballibaba2 Jan 16, 2025
0bba5f6
Puts MaxThreads for tests back in.
Alliballibaba2 Jan 16, 2025
bace48c
Uses gomaxprocs instead.
Alliballibaba2 Jan 16, 2025
a43d2c0
Adjusts docs.
Alliballibaba2 Jan 16, 2025
7614435
Uses go's monotonic clock instead.
Alliballibaba2 Jan 16, 2025
32a3129
Removes unused func.
Alliballibaba2 Jan 16, 2025
d189010
Removes old notations.
Alliballibaba2 Jan 16, 2025
32517fe
Locks the thread handler on debug status.
Alliballibaba2 Jan 16, 2025
1d899db
Locks the thread handler on debug status.
Alliballibaba2 Jan 16, 2025
9a83ab1
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Jan 22, 2025
b3fd756
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Jan 27, 2025
e3d38b5
go fmt
Alliballibaba2 Jan 27, 2025
755551a
Makes scale chan nil if scaling is not active.
Alliballibaba2 Jan 27, 2025
2135354
locks before modification.
Alliballibaba2 Jan 27, 2025
69597ef
Fixes race condition on restarts in tests.
Alliballibaba2 Jan 27, 2025
57a1343
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Jan 30, 2025
16e8169
Resets the opcache more safely.
Alliballibaba2 Feb 1, 2025
c038273
trigger build
Alliballibaba2 Feb 1, 2025
064a368
Starts all threads as inactive.
Alliballibaba2 Feb 2, 2025
60b437f
Properly removes autoscaled threads.
Alliballibaba2 Feb 2, 2025
4a5cfd1
Fix.
Alliballibaba2 Feb 2, 2025
954ac9e
Marks a single thread to call opcache_reset.
Alliballibaba2 Feb 2, 2025
3085a65
Marks a single thread to call opcache_reset.
Alliballibaba2 Feb 2, 2025
a1f89e3
Resets opcache_reset logic.
Alliballibaba2 Feb 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 132 additions & 0 deletions caddy/admin.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
package caddy

import (
"fmt"
"github.com/caddyserver/caddy/v2"
"github.com/dunglas/frankenphp"
"net/http"
"strconv"
"strings"
)

type FrankenPHPAdmin struct{}

// if the id starts with "admin.api" the module will register AdminRoutes via module.Routes()
func (FrankenPHPAdmin) CaddyModule() caddy.ModuleInfo {
return caddy.ModuleInfo{
ID: "admin.api.frankenphp",
New: func() caddy.Module { return new(FrankenPHPAdmin) },
}
}

func (admin FrankenPHPAdmin) Routes() []caddy.AdminRoute {
return []caddy.AdminRoute{
{
Pattern: "/frankenphp/workers/restart",
Handler: caddy.AdminHandlerFunc(admin.restartWorkers),
},
{
Pattern: "/frankenphp/threads",
Handler: caddy.AdminHandlerFunc(admin.threads),
},
}
}

func (admin *FrankenPHPAdmin) restartWorkers(w http.ResponseWriter, r *http.Request) error {
if r.Method != http.MethodPost {
return admin.error(http.StatusMethodNotAllowed, fmt.Errorf("method not allowed"))
}

frankenphp.RestartWorkers()
caddy.Log().Info("workers restarted from admin api")
admin.success(w, "workers restarted successfully\n")

return nil
}

func (admin *FrankenPHPAdmin) threads(w http.ResponseWriter, r *http.Request) error {
switch r.Method {
case http.MethodPut:
return admin.changeThreads(w, r, admin.getCountFromRequest(r))
case http.MethodDelete:
return admin.changeThreads(w, r, -admin.getCountFromRequest(r))
case http.MethodGet:
return admin.success(w, frankenphp.ThreadDebugStatus())
default:
return admin.error(http.StatusMethodNotAllowed, fmt.Errorf("method not allowed, try: GET,PUT,DELETE"))
}
}

func (admin *FrankenPHPAdmin) changeThreads(w http.ResponseWriter, r *http.Request, count int) error {
if !r.URL.Query().Has("worker") {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: you could store the result of Query() in a variable to prevent parsing the query two times.

You could even directly get the value and check if it is the zero value here.

Copy link
Collaborator

@AlliBalliBaba AlliBalliBaba Dec 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I'm explicitly checking for 'has' here is so something like this also works:
curl -X PUT http://localhost:2019/frankenphp/threads?worker

(most installations will probably only have 1 worker, so this is just for convenience)

return admin.changeRegularThreads(w, count)
}
workerFilename := admin.getWorkerBySuffix(r.URL.Query().Get("worker"))

return admin.changeWorkerThreads(w, count, workerFilename)
}

func (admin *FrankenPHPAdmin) changeWorkerThreads(w http.ResponseWriter, num int, workerFilename string) error {
method := frankenphp.AddWorkerThread
if num < 0 {
num = -num
method = frankenphp.RemoveWorkerThread
}
message := ""
for i := 0; i < num; i++ {
threadCount, err := method(workerFilename)
if err != nil {
return admin.error(http.StatusBadRequest, err)
}
message = fmt.Sprintf("New thread count: %d %s\n", threadCount, workerFilename)
}
return admin.success(w, message)
}

func (admin *FrankenPHPAdmin) changeRegularThreads(w http.ResponseWriter, num int) error {
method := frankenphp.AddRegularThread
if num < 0 {
num = -num
method = frankenphp.RemoveRegularThread
}
message := ""
for i := 0; i < num; i++ {
threadCount, err := method()
if err != nil {
return admin.error(http.StatusBadRequest, err)
}
message = fmt.Sprintf("New thread count: %d Regular Threads\n", threadCount)
}
return admin.success(w, message)
}

func (admin *FrankenPHPAdmin) success(w http.ResponseWriter, message string) error {
w.WriteHeader(http.StatusOK)
_, err := w.Write([]byte(message))
return err
}

func (admin *FrankenPHPAdmin) error(statusCode int, err error) error {
return caddy.APIError{HTTPStatus: statusCode, Err: err}
}

func (admin *FrankenPHPAdmin) getCountFromRequest(r *http.Request) int {
value := r.URL.Query().Get("count")
if value == "" {
return 1
}
i, err := strconv.Atoi(value)
if err != nil {
return 1
}
return i
}

func (admin *FrankenPHPAdmin) getWorkerBySuffix(pattern string) string {
for _, workerFilename := range frankenphp.WorkerFileNames() {
if strings.HasSuffix(workerFilename, pattern) {
return workerFilename
}
}
return ""
}
Loading
Loading