Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests failing with 1.34.1 #763

Open
txtsd opened this issue Jan 20, 2025 · 2 comments
Open

Tests failing with 1.34.1 #763

txtsd opened this issue Jan 20, 2025 · 2 comments

Comments

@txtsd
Copy link

txtsd commented Jan 20, 2025

Describe the bug
Tests are failing

To Reproduce
Steps to reproduce the behavior:

go test ./...

Expected behavior
Tests shouldn't fail

Logs

?   	github.com/tidwall/tile38/cmd/tile38-benchmark	[no test files]
?   	github.com/tidwall/tile38/cmd/tile38-benchmark/az	[no test files]
?   	github.com/tidwall/tile38/cmd/tile38-cli	[no test files]
?   	github.com/tidwall/tile38/cmd/tile38-luamemtest	[no test files]
?   	github.com/tidwall/tile38/cmd/tile38-server	[no test files]
ok  	github.com/tidwall/tile38/core	0.007s
ok  	github.com/tidwall/tile38/internal/bing	0.164s
ok  	github.com/tidwall/tile38/internal/buffer	0.008s
?   	github.com/tidwall/tile38/internal/deadline	[no test files]
?   	github.com/tidwall/tile38/internal/endpoint	[no test files]
ok  	github.com/tidwall/tile38/internal/clip	0.005s
?   	github.com/tidwall/tile38/internal/hservice	[no test files]
ok  	github.com/tidwall/tile38/internal/collection	0.679s
ok  	github.com/tidwall/tile38/internal/field	2.005s
ok  	github.com/tidwall/tile38/internal/glob	1.495s
ok  	github.com/tidwall/tile38/internal/log	0.006s
ok  	github.com/tidwall/tile38/internal/object	0.005s
?   	github.com/tidwall/tile38/tests/107	[no test files]
?   	github.com/tidwall/tile38/tests/616	[no test files]
ok  	github.com/tidwall/tile38/internal/server	0.013s
ok  	github.com/tidwall/tile38/internal/sstring	0.002s

�[1mTesting keys�[0m
BOUNDS ...                   [�[32mok�[0m]
DEL ...                      [�[32mok�[0m]
DROP ...                     [�[32mok�[0m]
RENAME ...                   [�[32mok�[0m]
RENAMENX ...                 [�[32mok�[0m]
EXPIRE ...                   [�[32mok�[0m]
FSET ...                     [�[32mok�[0m]
FGET ...                     [�[32mok�[0m]
GET ...                      [�[32mok�[0m]
KEYS ...                     [�[32mok�[0m]
PERSIST ...                  [�[32mok�[0m]
SET ...                      [�[32mok�[0m]
STATS ...                    [�[32mok�[0m]
TTL ...                      [�[32mok�[0m]
EXIST ...                    [�[32mok�[0m]
FEXIST ...                   [�[32mok�[0m]
SET EX ...                   [�[32mok�[0m]
PDEL ...                     [�[32mok�[0m]
FIELDS ...                   [�[32mok�[0m]
WHEREIN ...                  [�[32mok�[0m]
WHEREEVAL ...                [�[32mok�[0m]
TYPE ...                     [�[32mok�[0m]
FLUSHDB ...                  [�[32mok�[0m]
HEALTHZ ...                  [�[32mok�[0m]
SERVER ...                   [�[32mok�[0m]
INFO ...                     [�[32mok�[0m]

�[1mTesting json�[0m
basic ...                    [�[32mok�[0m]
geojson ...                  [�[32mok�[0m]
number ...                   [�[32mok�[0m]

�[1mTesting search�[0m
KNN_BASIC ...                [�[32mok�[0m]
KNN_RANDOM ...               [�[32mok�[0m]
KNN_CURSOR ...               [�[32mok�[0m]
NEARBY_SPARSE ...            [�[32mok�[0m]
WITHIN_CIRCLE ...            [�[32mok�[0m]
WITHIN_SECTOR ...            [�[32mok�[0m]
INTERSECTS_CIRCLE ...        [�[32mok�[0m]
INTERSECTS_SECTOR ...        [�[32mok�[0m]
WITHIN ...                   [�[32mok�[0m]
WITHIN_CURSOR ...            [�[32mok�[0m]
WITHIN_CLIPBY ...            [�[32mok�[0m]
INTERSECTS ...               [�[32mok�[0m]
INTERSECTS_CURSOR ...        [�[32mok�[0m]
INTERSECTS_CLIPBY ...        [�[32mok�[0m]
SCAN_CURSOR ...              [�[32mok�[0m]
SEARCH_CURSOR ...            [�[32mok�[0m]
MATCH ...                    [�[32mok�[0m]
FIELDS ...                   [�[32mok�[0m]
BUFFER ...                   [�[32mok�[0m]

�[1mTesting testcmd�[0m
WITHIN ...                   [�[32mok�[0m]
INTERSECTS ...               [�[32mok�[0m]
INTERSECTS_CLIP ...          [�[32mok�[0m]
ExpressionErrors ...         [�[32mok�[0m]
Expressions ...              [�[32mok�[0m]

�[1mTesting client�[0m
OUTPUT ...                   [�[32mok�[0m]
CLIENT ...                   [�[32mok�[0m]

�[1mTesting scripts�[0m
BASIC ...                    [�[32mok�[0m]
ATOMIC ...                   [�[32mok�[0m]
READONLY ...                 [�[32mok�[0m]
NONATOMIC ...                [�[32mok�[0m]
VULN ...                     [�[32mok�[0m]

�[1mTesting fence�[0m
basic ...                    [�[32mok�[0m]
channel message order ...    panic: test timed out after 10m0s
	running tests:
		TestIntegration (10m0s)

goroutine 21151 [running]:
testing.(*M).startAlarm.func1()
	testing/testing.go:2373 +0x385
created by time.goFunc
	time/sleep.go:215 +0x2d

goroutine 1 [chan receive, 10 minutes]:
testing.(*T).Run(0xc0000cd860, {0x5964fe76679e?, 0x0?}, 0x5964fef6deb8)
	testing/testing.go:1751 +0x3ab
testing.runTests.func1(0xc0000cd860)
	testing/testing.go:2168 +0x37
testing.tRunner(0xc0000cd860, 0xc0004b3c70)
	testing/testing.go:1690 +0xf4
testing.runTests(0xc000120690, {0x5964ff814780, 0x1, 0x1}, {0x5964fda7c1b0?, 0x5964fda7be1a?, 0x5964ff841ca0?})
	testing/testing.go:2166 +0x43d
testing.(*M).Run(0xc000504aa0)
	testing/testing.go:2034 +0x64a
main.main()
	_testmain.go:47 +0x9b

goroutine 6 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000203800)
	[email protected]/stats/view/worker.go:292 +0x9f
created by go.opencensus.io/stats/view.init.0 in goroutine 1
	[email protected]/stats/view/worker.go:34 +0x8d

goroutine 33 [chan receive, 9 minutes]:
github.com/tidwall/tile38/tests.runTestGroups(0xc0000cda00)
	github.com/tidwall/tile38/tests/tests_test.go:178 +0x1c5
github.com/tidwall/tile38/tests.TestIntegration(0xc0000cda00)
	github.com/tidwall/tile38/tests/tests_test.go:61 +0x13ee
testing.tRunner(0xc0000cda00, 0x5964fef6deb8)
	testing/testing.go:1690 +0xf4
created by testing.(*T).Run in goroutine 1
	testing/testing.go:1743 +0x390

goroutine 49 [syscall, 10 minutes]:
os/signal.signal_recv()
	runtime/sigqueue.go:152 +0x29
os/signal.loop()
	os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 33
	os/signal/signal.go:151 +0x1f

goroutine 50 [chan receive, 10 minutes]:
github.com/tidwall/tile38/tests.TestIntegration.func1()
	github.com/tidwall/tile38/tests/tests_test.go:42 +0x19
created by github.com/tidwall/tile38/tests.TestIntegration in goroutine 33
	github.com/tidwall/tile38/tests/tests_test.go:41 +0x112

goroutine 316 [sleep]:
time.Sleep(0xee6b280)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/tests.runTestGroups.func2()
	github.com/tidwall/tile38/tests/tests_test.go:175 +0x9e
created by github.com/tidwall/tile38/tests.runTestGroups in goroutine 33
	github.com/tidwall/tile38/tests/tests_test.go:115 +0x1b6

goroutine 1795 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0007a52c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1764
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 4960 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc002cfe9d0, 0x0)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0xc000389078?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Server).liveAOF(0xc002225888, 0x0, {0x5964fef92478, 0xc004be8400}, 0xc0038b2050, 0x4?)
	github.com/tidwall/tile38/internal/server/aof.go:527 +0x50f
github.com/tidwall/tile38/internal/server.(*Server).goLive(0xc002225888, {0x5964fef7c840, 0x5964ff7ce540}, {0x5964fef92478, 0xc004be8400}, 0xc0038b2050, 0xc0037a20a0, 0x0)
	github.com/tidwall/tile38/internal/server/live.go:103 +0x1f1
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2.2()
	github.com/tidwall/tile38/internal/server/server.go:620 +0x135
created by github.com/tidwall/tile38/internal/server.(*Server).netServe.func2 in goroutine 4928
	github.com/tidwall/tile38/internal/server/server.go:618 +0x15eb

goroutine 1643 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0005b0600)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1637
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 5100 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc002cfe9d0, 0x1)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0xc0049a8020?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Server).liveAOF(0xc002225888, 0x0, {0x5964fef92478, 0xc0034ec0e0}, 0xc000662050, 0x4?)
	github.com/tidwall/tile38/internal/server/aof.go:527 +0x50f
github.com/tidwall/tile38/internal/server.(*Server).goLive(0xc002225888, {0x5964fef7c840, 0x5964ff7ce540}, {0x5964fef92478, 0xc0034ec0e0}, 0xc000662050, 0xc000386280, 0x0)
	github.com/tidwall/tile38/internal/server/live.go:103 +0x1f1
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2.2()
	github.com/tidwall/tile38/internal/server/server.go:620 +0x135
created by github.com/tidwall/tile38/internal/server.(*Server).netServe.func2 in goroutine 5219
	github.com/tidwall/tile38/internal/server/server.go:618 +0x15eb

goroutine 2285 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000e8c000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2300
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2189 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000b58000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2245
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2165 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001d820c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2149
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2199 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0020ee0c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1947
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2357 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0010d2cc0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2272
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 453 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000000780)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 124
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 196 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0007ee000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 113
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1702 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x762af6c8e5f8, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0018dda00?, 0xc0044b0000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0018dda00, {0xc0044b0000, 0x1000, 0x1000})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0018dda00, {0xc0044b0000?, 0x5964fda1e165?, 0x5964fda1e165?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0007ea780, {0xc0044b0000?, 0xc0047f2f00?, 0xc0047f2ec0?})
	net/net.go:189 +0x45
bufio.(*Reader).fill(0xc0034a8ba0)
	bufio/bufio.go:110 +0x103
bufio.(*Reader).ReadSlice(0xc0034a8ba0, 0xa)
	bufio/bufio.go:376 +0x29
github.com/gomodule/redigo/redis.(*conn).readLine(0xc0004e4000)
	github.com/gomodule/[email protected]/redis/conn.go:518 +0x33
github.com/gomodule/redigo/redis.(*conn).readReply(0xc0004e4000)
	github.com/gomodule/[email protected]/redis/conn.go:598 +0x1c
github.com/gomodule/redigo/redis.(*conn).ReceiveWithTimeout(0xc0004e4000, 0x5964fed3fb20?)
	github.com/gomodule/[email protected]/redis/conn.go:688 +0x6c
github.com/gomodule/redigo/redis.(*conn).Receive(0x0?)
	github.com/gomodule/[email protected]/redis/conn.go:678 +0x17
github.com/gomodule/redigo/redis.PubSubConn.Receive({{0x5964fef8e6b8?, 0xc0004e4000?}})
	github.com/gomodule/[email protected]/redis/pubsub.go:100 +0x2b
github.com/tidwall/tile38/tests.fence_channel_message_order_test.func1()
	github.com/tidwall/tile38/tests/fence_test.go:200 +0x2fd
created by github.com/tidwall/tile38/tests.fence_channel_message_order_test in goroutine 243
	github.com/tidwall/tile38/tests/fence_test.go:179 +0xa6

goroutine 1424 [chan receive, 9 minutes]:
github.com/tidwall/tile38/internal/server.Serve.func4()
	github.com/tidwall/tile38/internal/server/server.go:319 +0x45
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:318 +0x1219

goroutine 1251 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001d672c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1211
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1068 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0031343c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1062
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1321 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001e54780)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1315
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 726 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000001c80)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 704
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2311 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc0003823d0, 0x0)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0xc000cb8600?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Server).startPublishQueue(0xc002225888, 0xc002f823c0?)
	github.com/tidwall/tile38/internal/server/pubqueue.go:59 +0x106
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 2296
	github.com/tidwall/tile38/internal/server/server.go:427 +0x19bf

goroutine 1529 [chan receive, 9 minutes]:
github.com/tidwall/tile38/internal/server.Serve({{0x5964fe75fc84, 0x9}, 0x278b, {0xc002dc2ab0, 0xf}, 0x1, {0xc002079130, 0x6}, {0x0, 0x0}, ...})
	github.com/tidwall/tile38/internal/server/server.go:442 +0x1a5e
github.com/tidwall/tile38/tests.mockOpenServer.func1()
	github.com/tidwall/tile38/tests/mock_test.go:126 +0x13e
created by github.com/tidwall/tile38/tests.mockOpenServer in goroutine 243
	github.com/tidwall/tile38/tests/mock_test.go:112 +0x455

goroutine 2103 [chan send, 9 minutes]:
github.com/tidwall/tile38/internal/server.Serve.func2()
	github.com/tidwall/tile38/internal/server/server.go:307 +0x3e
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 2091
	github.com/tidwall/tile38/internal/server/server.go:305 +0x1133

goroutine 3451 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc0000e4b40)
	net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 3307
	net/http/transport.go:1875 +0x15a5

goroutine 1593 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc00435a540)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1587
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 243 [chan receive, 9 minutes]:
github.com/tidwall/tile38/tests.fence_channel_message_order_test(0xc002c96500)
	github.com/tidwall/tile38/tests/fence_test.go:242 +0x325
github.com/tidwall/tile38/tests.(*testGroupSub).run.func2(0xc000694000)
	github.com/tidwall/tile38/tests/tests_test.go:237 +0x8f
github.com/tidwall/tile38/tests.(*testGroupSub).run(0x0?)
	github.com/tidwall/tile38/tests/tests_test.go:238 +0x53
github.com/tidwall/tile38/tests.runTestGroups.runTestGroups.func1.func3.1.1.1()
	github.com/tidwall/tile38/tests/tests_test.go:96 +0x85
created by github.com/tidwall/tile38/tests.runTestGroups.runTestGroups.func1.func3.1.1 in goroutine 242
	github.com/tidwall/tile38/tests/tests_test.go:90 +0xe5

goroutine 1038 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001ef3ec0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1138
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 185 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000390240)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 24
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2120 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001d82000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2031
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 189 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000390300)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 179
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1722 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc00071ecc0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1762
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2296 [chan receive, 9 minutes]:
github.com/tidwall/tile38/internal/server.Serve({{0x5964fe75fc84, 0x9}, 0x27bd, {0xc00434e300, 0xf}, 0x1, {0xc00438a5a8, 0x6}, {0x0, 0x0}, ...})
	github.com/tidwall/tile38/internal/server/server.go:442 +0x1a5e
github.com/tidwall/tile38/tests.mockOpenServer.func1()
	github.com/tidwall/tile38/tests/mock_test.go:126 +0x13e
created by github.com/tidwall/tile38/tests.mockOpenServer in goroutine 303
	github.com/tidwall/tile38/tests/mock_test.go:112 +0x455

goroutine 1776 [chan receive, 9 minutes]:
github.com/tidwall/tile38/internal/server.Serve({{0x5964fe75fc84, 0x9}, 0x27a9, {0xc00152f840, 0xf}, 0x1, {0xc0012f4380, 0x6}, {0x0, 0x0}, ...})
	github.com/tidwall/tile38/internal/server/server.go:442 +0x1a5e
github.com/tidwall/tile38/tests.mockOpenServer.func1()
	github.com/tidwall/tile38/tests/mock_test.go:126 +0x13e
created by github.com/tidwall/tile38/tests.mockOpenServer in goroutine 296
	github.com/tidwall/tile38/tests/mock_test.go:112 +0x455

goroutine 815 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0023ba840)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 888
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 280 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0007ee180)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 11
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 484 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0007ee0c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 418
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 568 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0007ee6c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 562
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1907 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002caef00)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1869
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1297 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0037cd140)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1214
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 338 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000a60000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 29
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 324 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0003903c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 129
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 715 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000a60300)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 871
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1571 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc00256fb50, 0x957)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0x5964fda82c34?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Server).processLives(0xc001f92388, 0xc0020791a0)
	github.com/tidwall/tile38/internal/server/live.go:61 +0x165
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:415 +0x1733

goroutine 369 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000d88000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 177
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1194 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001faf080)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1125
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1569 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002f823c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1529
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 650 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000e2ca80)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 644
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1707 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc000859f10, 0x1)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0x5964fee2e701?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Server).liveSubscription.func9()
	github.com/tidwall/tile38/internal/server/pubsub.go:346 +0xb4
created by github.com/tidwall/tile38/internal/server.(*Server).liveSubscription in goroutine 1706
	github.com/tidwall/tile38/internal/server/pubsub.go:327 +0x52d

goroutine 2033 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc00036a110, 0x0)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0xc000fe9f80?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Server).startPublishQueue(0xc003256a88, 0xc0007e1ea0?)
	github.com/tidwall/tile38/internal/server/pubqueue.go:59 +0x106
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1776
	github.com/tidwall/tile38/internal/server/server.go:427 +0x19bf

goroutine 262 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000e2c000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 26
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2295 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc003210240)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2289
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2573 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0007a5680)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2505
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1570 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x762af6fa66e0, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc002977a00?, 0x10?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc002977a00)
	internal/poll/fd_unix.go:620 +0x295
net.(*netFD).accept(0xc002977a00)
	net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc002fe9b40)
	net/tcpsock_posix.go:159 +0x1e
net.(*TCPListener).Accept(0xc002fe9b40)
	net/tcpsock.go:372 +0x30
net/http.(*Server).Serve(0xc000f114a0, {0x5964fef86a50, 0xc002fe9b40})
	net/http/server.go:3330 +0x30c
net/http.Serve(...)
	net/http/server.go:2858
github.com/tidwall/tile38/internal/server.Serve.func8()
	github.com/tidwall/tile38/internal/server/server.go:405 +0x157
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:400 +0x16ac

goroutine 410 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000a60840)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 127
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 428 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000390cc0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 422
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1978 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x762af6c8e4e0, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0012f3980?, 0x10?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0012f3980)
	internal/poll/fd_unix.go:620 +0x295
net.(*netFD).accept(0xc0012f3980)
	net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc00036a0c0)
	net/tcpsock_posix.go:159 +0x1e
net.(*TCPListener).Accept(0xc00036a0c0)
	net/tcpsock.go:372 +0x30
net/http.(*Server).Serve(0xc0000eed20, {0x5964fef86a50, 0xc00036a0c0})
	net/http/server.go:3330 +0x30c
net/http.Serve(...)
	net/http/server.go:2858
github.com/tidwall/tile38/internal/server.Serve.func8()
	github.com/tidwall/tile38/internal/server/server.go:405 +0x157
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1776
	github.com/tidwall/tile38/internal/server/server.go:400 +0x16ac

goroutine 362 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000cc2180)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 356
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 436 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000ac2540)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 318
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2098 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc001379010, 0x7)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0xc00053c800?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Server).liveAOF(0xc003256a88, 0x0, {0x5964fef92478, 0xc001e7c080}, 0xc001b02050, 0x4?)
	github.com/tidwall/tile38/internal/server/aof.go:527 +0x50f
github.com/tidwall/tile38/internal/server.(*Server).goLive(0xc003256a88, {0x5964fef7c840, 0x5964ff7ce540}, {0x5964fef92478, 0xc001e7c080}, 0xc001b02050, 0xc00384c8c0, 0x0)
	github.com/tidwall/tile38/internal/server/live.go:103 +0x1f1
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2.2()
	github.com/tidwall/tile38/internal/server/server.go:620 +0x135
created by github.com/tidwall/tile38/internal/server.(*Server).netServe.func2 in goroutine 2069
	github.com/tidwall/tile38/internal/server/server.go:618 +0x15eb

goroutine 2227 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc003210000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2143
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1486 [sleep]:
time.Sleep(0xee6b280)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/server.(*Server).processLives.func1()
	github.com/tidwall/tile38/internal/server/live.go:36 +0x6f
created by github.com/tidwall/tile38/internal/server.(*Server).processLives in goroutine 1571
	github.com/tidwall/tile38/internal/server/live.go:29 +0xf9

goroutine 2292 [chan send, 9 minutes]:
github.com/tidwall/tile38/internal/server.Serve.func2()
	github.com/tidwall/tile38/internal/server/server.go:307 +0x3e
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 2289
	github.com/tidwall/tile38/internal/server/server.go:305 +0x1133

goroutine 2232 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0032100c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2264
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 352 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000a60780)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 382
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1973 [sleep]:
time.Sleep(0x3b9aca00)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/endpoint.(*Manager).run(0xc0037baeb0)
	github.com/tidwall/tile38/internal/endpoint/endpoint.go:183 +0x55
created by github.com/tidwall/tile38/internal/endpoint.NewManager in goroutine 1776
	github.com/tidwall/tile38/internal/endpoint/endpoint.go:161 +0xc5

goroutine 1362 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001726840)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1276
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1398 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001ef2e40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1371
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 679 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000d88900)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 600
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 584 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000000a80)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 641
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 978 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001e54cc0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 953
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 504 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000e2dbc0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 388
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 963 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0024e4180)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 891
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1704 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x762af6c8e3c8, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc001d68c00?, 0xc000802000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc001d68c00, {0xc000802000, 0xffff, 0xffff})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc001d68c00, {0xc000802000?, 0x5964fe6491c5?, 0xc001f92538?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc002bf6168, {0xc000802000?, 0xc001f92538?, 0xc000575d30?})
	net/net.go:189 +0x45
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2({0x5964fef92070, 0xc002bf6168})
	github.com/tidwall/tile38/internal/server/server.go:555 +0x503
created by github.com/tidwall/tile38/internal/server.(*Server).netServe in goroutine 1422
	github.com/tidwall/tile38/internal/server/server.go:502 +0x4d2

goroutine 1574 [sleep]:
time.Sleep(0xbebc200)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/server.(*Server).loopUntilServerStops(0xc001f92388, 0x3b9aca00, 0xc0001a3f90)
	github.com/tidwall/tile38/internal/server/server.go:785 +0x45
github.com/tidwall/tile38/internal/server.(*Server).watchAutoGC(0xc001f92388, 0xc00049cb00?)
	github.com/tidwall/tile38/internal/server/server.go:731 +0x86
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:421 +0x1877

goroutine 1095 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc003276840)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1089
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1162 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001234b40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1156
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1230 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002b20e40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1224
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 550 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000e2dc80)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 474
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 937 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002980180)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 911
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 5219 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc002cfe1c0?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0xc001af6630?)
	sync/waitgroup.go:118 +0x48
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2({0x5964fef92070, 0xc004cc40f8})
	github.com/tidwall/tile38/internal/server/server.go:631 +0x15f8
created by github.com/tidwall/tile38/internal/server.(*Server).netServe in goroutine 2220
	github.com/tidwall/tile38/internal/server/server.go:502 +0x4d2

goroutine 611 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000e2dd40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 578
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 536 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0007ee900)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 476
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1705 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc000859ec0?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0xc002feb810?)
	sync/waitgroup.go:118 +0x48
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2({0x5964fef92070, 0xc002bf6170})
	github.com/tidwall/tile38/internal/server/server.go:631 +0x15f8
created by github.com/tidwall/tile38/internal/server.(*Server).netServe in goroutine 1422
	github.com/tidwall/tile38/internal/server/server.go:502 +0x4d2

goroutine 2169 [chan send, 9 minutes]:
github.com/tidwall/tile38/internal/server.Serve.func2()
	github.com/tidwall/tile38/internal/server/server.go:307 +0x3e
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 2141
	github.com/tidwall/tile38/internal/server/server.go:305 +0x1133

goroutine 2000 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc003276000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2029
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 4928 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc00036b900?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0xc002b93320?)
	sync/waitgroup.go:118 +0x48
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2({0x5964fef92070, 0xc000089190})
	github.com/tidwall/tile38/internal/server/server.go:631 +0x15f8
created by github.com/tidwall/tile38/internal/server.(*Server).netServe in goroutine 2220
	github.com/tidwall/tile38/internal/server/server.go:502 +0x4d2

goroutine 693 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000000b40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 660
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1423 [sleep]:
time.Sleep(0xee6b280)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/server.Serve.func3()
	github.com/tidwall/tile38/internal/server/server.go:314 +0x38
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:311 +0x11a5

goroutine 2106 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001d82180)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2091
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 745 [sleep]:
time.Sleep(0xbebc200)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/server.readMemStats.func1()
	github.com/tidwall/tile38/internal/server/stats.go:38 +0x45
created by github.com/tidwall/tile38/internal/server.readMemStats in goroutine 823
	github.com/tidwall/tile38/internal/server/stats.go:31 +0x91

goroutine 899 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0020ee480)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 634
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1491 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002a249c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1475
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 606 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000a61c80)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 736
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1113 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002a5c300)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1008
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2159 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0023ba000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2180
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 781 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001726780)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 743
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1613 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc00435ab40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1546
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1658 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0030b0840)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1652
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 998 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0032b1080)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 927
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 766 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001d82e40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 760
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1015 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0032118c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 976
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 827 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000cc2f00)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 628
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1706 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x762af6fa6398, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc001d68d00?, 0xc0007cc070?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc001d68d00, {0xc0007cc070, 0xffff, 0xffff})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc001d68d00, {0xc0007cc070?, 0x5964fda1e42b?, 0xc000b5d888?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc002bf6170, {0xc0007cc070?, 0x762af6c12f58?, 0x1a?})
	net/net.go:189 +0x45
github.com/tidwall/tile38/internal/server.(*PipelineReader).ReadMessages(0xc0007cc050)
	github.com/tidwall/tile38/internal/server/server.go:1577 +0x50
github.com/tidwall/tile38/internal/server.(*Server).liveSubscription(0xc001f92388, {0x5964fef92478, 0xc003104a80}, 0xc0007cc050, 0xc0041756d0, 0x0)
	github.com/tidwall/tile38/internal/server/pubsub.go:392 +0x570
github.com/tidwall/tile38/internal/server.(*Server).goLive(0xc001f92388, {0x5964fef7c7e0, 0x5964ff8657e0}, {0x5964fef92478, 0xc003104a80}, 0xc0007cc050, 0xc0041756d0, 0x0)
	github.com/tidwall/tile38/internal/server/live.go:105 +0x9c5
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2.2()
	github.com/tidwall/tile38/internal/server/server.go:620 +0x135
created by github.com/tidwall/tile38/internal/server.(*Server).netServe.func2 in goroutine 1705
	github.com/tidwall/tile38/internal/server/server.go:618 +0x15eb

goroutine 852 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0020ee540)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 622
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1181 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002e10a80)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1175
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 856 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0020ee600)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 803
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2433 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc000e4a190, 0x99)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0xc000e60240?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Hook).manager(0xc000e60240)
	github.com/tidwall/tile38/internal/server/hooks.go:623 +0xa7
created by github.com/tidwall/tile38/internal/server.(*Hook).Open in goroutine 2376
	github.com/tidwall/tile38/internal/server/hooks.go:564 +0x128

goroutine 1351 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001e54a80)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1392
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2156 [chan send, 9 minutes]:
github.com/tidwall/tile38/internal/server.Serve.func2()
	github.com/tidwall/tile38/internal/server/server.go:307 +0x3e
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 2180
	github.com/tidwall/tile38/internal/server/server.go:305 +0x1133

goroutine 840 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc000cc2000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 748
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1238 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0024e43c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1143
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1625 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x762af6c76470, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0018dd900?, 0xc000792000?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0018dd900, {0xc000792000, 0xffff, 0xffff})
	internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0018dd900, {0xc000792000?, 0x5964fe6491c5?, 0xc001f92538?})
	net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0007ea768, {0xc000792000?, 0xc001f92538?, 0xc001bafd30?})
	net/net.go:189 +0x45
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2({0x5964fef92070, 0xc0007ea768})
	github.com/tidwall/tile38/internal/server/server.go:555 +0x503
created by github.com/tidwall/tile38/internal/server.(*Server).netServe in goroutine 1422
	github.com/tidwall/tile38/internal/server/server.go:502 +0x4d2

goroutine 992 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002b20cc0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1009
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2069 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc00196db00?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0xc0015ba410?)
	sync/waitgroup.go:118 +0x48
github.com/tidwall/tile38/internal/server.(*Server).netServe.func2({0x5964fef92070, 0xc0003767c0})
	github.com/tidwall/tile38/internal/server/server.go:631 +0x15f8
created by github.com/tidwall/tile38/internal/server.(*Server).netServe in goroutine 1974
	github.com/tidwall/tile38/internal/server/server.go:502 +0x4d2

goroutine 1428 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002a24000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1389
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1576 [sleep]:
time.Sleep(0xbebc200)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/server.(*Server).loopUntilServerStops(0xc001f92388, 0x3b9aca00, 0xc004329f98)
	github.com/tidwall/tile38/internal/server/server.go:785 +0x45
github.com/tidwall/tile38/internal/server.(*Server).backgroundSyncAOF(0x0?, 0x0?)
	github.com/tidwall/tile38/internal/server/server.go:806 +0x5a
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:425 +0x1953

goroutine 1577 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc002fe9b90, 0x1)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0xc000197200?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Server).startPublishQueue(0xc001f92388, 0x0?)
	github.com/tidwall/tile38/internal/server/pubqueue.go:59 +0x106
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:427 +0x19bf

goroutine 1446 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002688600)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1410
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1535 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0030e8cc0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1603
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2375 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001d36780)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2317
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1575 [sleep]:
time.Sleep(0xbebc200)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/server.(*Server).loopUntilServerStops(0xc001f92388, 0x5f5e100, 0xc000a06f98)
	github.com/tidwall/tile38/internal/server/server.go:785 +0x45
github.com/tidwall/tile38/internal/server.(*Server).backgroundExpiring(0x0?, 0x0?)
	github.com/tidwall/tile38/internal/server/expire.go:18 +0x5a
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:423 +0x18e5

goroutine 1573 [sleep]:
time.Sleep(0xbebc200)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/server.(*Server).loopUntilServerStops(0xc001f92388, 0x2540be400, 0xc002192f98)
	github.com/tidwall/tile38/internal/server/server.go:785 +0x45
github.com/tidwall/tile38/internal/server.(*Server).watchLuaStatePool(0xc003257888?, 0xc002192fa8?)
	github.com/tidwall/tile38/internal/server/server.go:798 +0x5f
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:419 +0x180b

goroutine 1572 [sleep]:
time.Sleep(0xbebc200)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/server.(*Server).loopUntilServerStops(0xc001f92388, 0xee6b2800, 0xc00202df98)
	github.com/tidwall/tile38/internal/server/server.go:785 +0x45
github.com/tidwall/tile38/internal/server.(*Server).watchOutOfMemory(0x5964fef92070?, 0xc0007eaf70?)
	github.com/tidwall/tile38/internal/server/server.go:791 +0x5a
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:417 +0x179f

goroutine 1883 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002c2fe00)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1877
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1378 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001fae000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1337
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1977 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001a82180)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1776
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2224 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x762af6c8e080, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc000dc3b80?, 0x10?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000dc3b80)
	internal/poll/fd_unix.go:620 +0x295
net.(*netFD).accept(0xc000dc3b80)
	net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc000382380)
	net/tcpsock_posix.go:159 +0x1e
net.(*TCPListener).Accept(0xc000382380)
	net/tcpsock.go:372 +0x30
net/http.(*Server).Serve(0xc0001641e0, {0x5964fef86a50, 0xc000382380})
	net/http/server.go:3330 +0x30c
net/http.Serve(...)
	net/http/server.go:2858
github.com/tidwall/tile38/internal/server.Serve.func8()
	github.com/tidwall/tile38/internal/server/server.go:405 +0x157
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 2296
	github.com/tidwall/tile38/internal/server/server.go:400 +0x16ac

goroutine 2172 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002e10000)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2141
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1817 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0008783c0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1811
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1974 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc0019f8d60?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0xc0019f8e00?)
	sync/waitgroup.go:118 +0x48
github.com/tidwall/tile38/internal/server.(*Server).netServe.func1()
	github.com/tidwall/tile38/internal/server/server.go:484 +0x1b2
github.com/tidwall/tile38/internal/server.(*Server).netServe(0xc003256a88)
	github.com/tidwall/tile38/internal/server/server.go:495 +0x4ef
github.com/tidwall/tile38/internal/server.Serve.func2()
	github.com/tidwall/tile38/internal/server/server.go:307 +0x25
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1776
	github.com/tidwall/tile38/internal/server/server.go:305 +0x1133

goroutine 1422 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x762af6c77078, 0x72)
	runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc00387c980?, 0x900000036?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc00387c980)
	internal/poll/fd_unix.go:620 +0x295
net.(*netFD).accept(0xc00387c980)
	net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc0038429c0)
	net/tcpsock_posix.go:159 +0x1e
net.(*TCPListener).Accept(0xc0038429c0)
	net/tcpsock.go:372 +0x30
github.com/tidwall/tile38/internal/server.(*Server).netServe(0xc001f92388)
	github.com/tidwall/tile38/internal/server/server.go:492 +0x31c
github.com/tidwall/tile38/internal/server.Serve.func2()
	github.com/tidwall/tile38/internal/server/server.go:307 +0x25
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 1529
	github.com/tidwall/tile38/internal/server/server.go:305 +0x1133

goroutine 1421 [sleep]:
time.Sleep(0x3b9aca00)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/endpoint.(*Manager).run(0xc0020db450)
	github.com/tidwall/tile38/internal/endpoint/endpoint.go:183 +0x55
created by github.com/tidwall/tile38/internal/endpoint.NewManager in goroutine 1529
	github.com/tidwall/tile38/internal/endpoint/endpoint.go:161 +0xc5

goroutine 1556 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc001e54b40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1483
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1963 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0052d9bc0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1938
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1517 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc00416ba40)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1488
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2223 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0005b0240)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2296
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1923 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc002b09e00)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1757
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 2040 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc005501ec0)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 2034
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 1806 [sync.Cond.Wait, 9 minutes]:
sync.runtime_notifyListWait(0xc0031de050, 0x1)
	runtime/sema.go:587 +0x15a
sync.(*Cond).Wait(0xc0007a55c0?)
	sync/cond.go:71 +0x85
github.com/tidwall/tile38/internal/server.(*Hook).manager(0xc0007a55c0)
	github.com/tidwall/tile38/internal/server/hooks.go:623 +0xa7
created by github.com/tidwall/tile38/internal/server.(*Hook).Open in goroutine 1685
	github.com/tidwall/tile38/internal/server/hooks.go:564 +0x128

goroutine 2220 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc0001c8d60?)
	runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0xc0001c8e00?)
	sync/waitgroup.go:118 +0x48
github.com/tidwall/tile38/internal/server.(*Server).netServe.func1()
	github.com/tidwall/tile38/internal/server/server.go:484 +0x1b2
github.com/tidwall/tile38/internal/server.(*Server).netServe(0xc002225888)
	github.com/tidwall/tile38/internal/server/server.go:495 +0x4ef
github.com/tidwall/tile38/internal/server.Serve.func2()
	github.com/tidwall/tile38/internal/server/server.go:307 +0x25
created by github.com/tidwall/tile38/internal/server.Serve in goroutine 2296
	github.com/tidwall/tile38/internal/server/server.go:305 +0x1133

goroutine 1740 [chan receive]:
github.com/tidwall/buntdb.(*DB).backgroundManager(0xc0005b1380)
	github.com/tidwall/[email protected]/buntdb.go:547 +0xaa
created by github.com/tidwall/buntdb.Open in goroutine 1782
	github.com/tidwall/[email protected]/buntdb.go:169 +0x2ac

goroutine 3450 [select, 9 minutes]:
net/http.(*persistConn).readLoop(0xc0000e4b40)
	net/http/transport.go:2325 +0xca5
created by net/http.(*Transport).dialConn in goroutine 3307
	net/http/transport.go:1874 +0x154f

goroutine 2219 [sleep]:
time.Sleep(0x3b9aca00)
	runtime/time.go:300 +0xf4
github.com/tidwall/tile38/internal/endpoint.(*Manager).run(0xc002bb90e0)
	github.com/tidwall/tile38/internal/endpoint/endpoint.go:183 +0x55
created by github.com/tidwall/tile38/internal/endpoint.NewManager in goroutine 2296
	github.com/tidwall/tile38/internal/endpoint/endpoint.go:161 +0xc5
FAIL	github.com/tidwall/tile38/tests	600.033s
FAIL

Operating System (please complete the following information):

  • OS: Arch Linuux
  • CPU: Ryzen 2400G
  • Version: 1.34.1
  • Container: None

Additional context
Add any other context about the problem here.
@tidwall
Copy link
Owner

tidwall commented Jan 22, 2025

Looks like there may be a bug in one of the tests.
Did you re-run, and did it happen to work the second time?

@txtsd
Copy link
Author

txtsd commented Jan 23, 2025

Yes, I ran them several times in a clean chroot and they failed every time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@txtsd @tidwall