-
Notifications
You must be signed in to change notification settings - Fork 431
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Procfs improvements #4540
Open
geyslan
wants to merge
8
commits into
aquasecurity:main
Choose a base branch
from
geyslan:procfs-work
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Procfs improvements #4540
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
geyslan
force-pushed
the
procfs-work
branch
2 times, most recently
from
January 21, 2025 19:34
86fdd6c
to
ce212f0
Compare
geyslan
force-pushed
the
procfs-work
branch
2 times, most recently
from
January 29, 2025 22:06
b95c22d
to
093169d
Compare
Since the type of the converted primitive is already known, formatter helpers should be used to construct procfs file paths instead of relying on `fmt.Sprintf`. Using `fmt.Sprintf` is relatively costly due to its internal formatting logic, which is unnecessary for simple path construction.
`os.ReadFile` is not efficient for reading files in `/proc` because it attempts to determine the file size before reading. This step is unnecessary for `/proc` files, as they are virtual files with sizes that are often reported as unknown or `0`. `proc.ReadFile` is a new function designed specifically for reading files in `/proc`. It reads directly into a buffer and is more efficient than `os.ReadFile` because it allows tuning the initial buffer size to better suit the characteristics of `/proc` files. Running tool: /home/gg/.goenv/versions/1.22.4/bin/go test -benchmem -run=^$ -tags ebpf -bench ^BenchmarkReadFile$ github.com/aquasecurity/tracee/pkg/utils/proc -benchtime=10000000x goos: linux goarch: amd64 pkg: github.com/aquasecurity/tracee/pkg/utils/proc cpu: AMD Ryzen 9 7950X 16-Core Processor BenchmarkReadFile/ProcFSReadFile/Empty_File-32 10000000 3525 ns/op 408 B/op 4 allocs/op BenchmarkReadFile/OsReadFile/Empty_File-32 10000000 4070 ns/op 872 B/op 5 allocs/op BenchmarkReadFile/ProcFSReadFile/Small_File-32 10000000 3961 ns/op 408 B/op 4 allocs/op BenchmarkReadFile/OsReadFile/Small_File-32 10000000 4538 ns/op 872 B/op 5 allocs/op BenchmarkReadFile/ProcFSReadFile/Exact_Buffer_Size-32 10000000 4229 ns/op 920 B/op 5 allocs/op BenchmarkReadFile/OsReadFile/Exact_Buffer_Size-32 10000000 4523 ns/op 872 B/op 5 allocs/op BenchmarkReadFile/ProcFSReadFile_/proc/self/stat-32 10000000 4043 ns/op 408 B/op 4 allocs/op BenchmarkReadFile/OsReadFile_/proc/self/stat-32 10000000 4585 ns/op 872 B/op 5 allocs/op PASS ok github.com/aquasecurity/tracee/pkg/utils/proc 334.751s
Remove the use of library functions to parse the stat file and instead parse it manually (on the fly) to reduce the number of allocations and improve performance. chore(proc): align parsing of stat field with the formats size This also align parsing sizes with the formats to avoid wrong parsing of the stat file. The internal fields are represented aligned with the actual kernel fields to avoid any confusion (signed/unsigned).
Propagate values based on its real size which in most cases is smaller than int (64-bit). This change reduces the memory footprint or at least the stress on the stack/heap.
Remove the use of library functions to parse the status file and instead parse it manually (on the fly) to reduce the number of allocations and improve performance.
Reduce ProcNS memory footprint by using the right member type sizes - namespace id is an uint32, since it is the inode number in struct ns_common. This change also improves the performance of GetAllProcNS(), GetProcNS() and GetMountNSFirstProcesses().
- NewProcStatFields() - NewThreadStatFields() - NewProcStatusFields() - NewThreadStatusFields()
Calling stat on /proc/<pid> would only increase the window for process termination between the stat call and the read of the file. This also replaces fmt.Sprintf with string concatenation and strconv.FormatInt for better performance.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Close: #4547
1. Explain what the PR does
9a53e02 perf(proctree): remove stat call
31937e6 chore: introduce builders with specific fields
3d5afb5 perf(proc): improve ns
58bc552 perf(proc): improve status file parsing
4b64197 perf(proctree/proc): align fields to real size
bfe8fc8 perf(proc): improve stat file parsing
c5f8cb0 perf(proc): introduce ReadFile for /proc
1b6899d perf(proc): use formatters for procfs file paths
9a53e02 perf(proctree): remove stat call
31937e6 chore: introduce builders with specific fields
3d5afb5 perf(proc): improve ns
58bc552 perf(proc): improve status file parsing
4b64197 perf(proctree/proc): align fields to real size
bfe8fc8 perf(proc): improve stat file parsing
c5f8cb0 perf(proc): introduce ReadFile for /proc
1b6899d perf(proc): use formatters for procfs file paths
2. Explain how to test it
3. Other comments