[BUG] deepflow-agent内存占用高，导致OOM

[yujianweilai]

Search before asking

• I had searched in the issues and found no similar feature requirement.

DeepFlow Component

Agent

What you expected to happen

我们使用了deepflow后，发现deepflow-agent，后出现重启的情况。因为从监控平台上看，deepflow-agent每次启动后，内存都回随着时间的推移，持续升高，直至达到的内存limit值，然后被k8s kill 。
deepflow-agent所在的主机上，message文件，会打印如下信息：
Jan 2 20:12:01 pretycx-mid3 systemd: Started Session 59550 of user root.
Jan 2 20:12:02 pretycx-mid3 dockerd: time="2025-01-02T20:12:02.591231160+08:00" level=info msg="ignoring event" container=f576d3c3158a3
7dc6f6b286649337a9d9a2aadfc4139953aacf4eefad5096432 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Jan 2 20:12:02 pretycx-mid3 containerd: time="2025-01-02T20:12:02.591181993+08:00" level=info msg="shim disconnected" id=f576d3c3158a37
dc6f6b286649337a9d9a2aadfc4139953aacf4eefad5096432
Jan 2 20:12:02 pretycx-mid3 containerd: time="2025-01-02T20:12:02.591243602+08:00" level=warning msg="cleaning up after shim disconnect
ed" id=f576d3c3158a37dc6f6b286649337a9d9a2aadfc4139953aacf4eefad5096432 namespace=moby
Jan 2 20:12:02 pretycx-mid3 containerd: time="2025-01-02T20:12:02.591260698+08:00" level=info msg="cleaning up dead shim"
Jan 2 20:12:02 pretycx-mid3 containerd: time="2025-01-02T20:12:02.599756159+08:00" level=warning msg="cleanup warnings time="2025-01-0
2T20:12:02+08:00" level=info msg="starting signal loop" namespace=moby pid=31722 runtime=io.containerd.runc.v2\n"
Jan 2 20:12:03 pretycx-mid3 kubelet: I0102 20:12:03.499599 1169 topology_manager.go:221] [topologymanager] RemoveContainer - Contain
er ID: c744d181e910c03e9fc1ce7729b55f68111f9c47a591afa0d5c3e40d8991d088
Jan 2 20:12:03 pretycx-mid3 kubelet: I0102 20:12:03.499884 1169 topology_manager.go:221] [topologymanager] RemoveContainer - Contain
er ID: f576d3c3158a37dc6f6b286649337a9d9a2aadfc4139953aacf4eefad5096432
Jan 2 20:12:22 pretycx-mid3 containerd: time="2025-01-02T20:12:22.515647673+08:00" level=info msg="loading plugin "io.containerd.event
.v1.publisher"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Jan 2 20:12:22 pretycx-mid3 containerd: time="2025-01-02T20:12:22.515695299+08:00" level=info msg="loading plugin "io.containerd.inter
nal.v1.shutdown"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Jan 2 20:12:22 pretycx-mid3 containerd: time="2025-01-02T20:12:22.515708850+08:00" level=info msg="loading plugin "io.containerd.ttrpc
.v1.task"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Jan 2 20:12:22 pretycx-mid3 containerd: time="2025-01-02T20:12:22.515857931+08:00" level=info msg="starting signal loop" namespace=moby
path=/run/containerd/io.containerd.runtime.v2.task/moby/3f91bea85fbfd72bbdf5abb955e7f7b9d0b48bd76d63b1cce0ed2807adc8dc44 pid=32217 runt
ime=io.containerd.runc.v2
Jan 2 20:12:22 pretycx-mid3 systemd: Started libcontainer container 3f91bea85fbfd72bbdf5abb955e7f7b9d0b48bd76d63b1cce0ed2807adc8dc44.
Jan 2 20:12:22 pretycx-mid3 systemd: Couldn't stat device /dev/char/10:200
Jan 2 20:14:01 pretycx-mid3 systemd: Started Session 59551 of user root.

propromethus监控平台上看，agent的内存使用是持续上升的。

deepflow-agent的镜像版本 registry.cn-beijing.aliyuncs.com/deepflow-ce/deepflow-agent:v6.6 ，image_id=503eb0f61a40

How to reproduce

No response

DeepFlow version

[root@pretycx-master1 ~]# kubectl exec -it -n deepflow deploy/deepflow-server -- deepflow-server -v
Name: deepflow-server community edition
Branch: v6.6
CommitID: dde430f
RevCount: 11427
Compiler: go version go1.21.13 linux/amd64
CompileTime: 2024-12-18 10:19:42
[root@pretycx-master1 ~]#
[root@pretycx-master1 ~]# kubectl exec -it -n deepflow ds/deepflow-agent -- deepflow-agent -v
11445-b05bb95c57a0454ef7318726d930348c9e996b02
Name: deepflow-agent community edition
Branch: v6.6
CommitId: b05bb95
RevCount: 11445
Compiler: rustc 1.77.1 (7cf61ebde 2024-03-27)
CompileTime: 2024-12-27 07:36:28
[root@pretycx-master1 ~]#

DeepFlow agent list

[root@pretycx-master1 ~]# deepflow-ctl agent list --ip 192.168.4.81
ID NAME TYPE CTRL_IP CTRL_MAC STATE GROUP EXCEPTIONS REVISION UPGRADE_REVISION
3 pretycx-emqx1-V14 K8S_VM 192.168.4.52 fa:16:3e:73:91:be NORMAL default v6.6 11445
4 pretycx-docker05-V2 K8S_VM 192.168.4.81 fa:16:3e:93:ef:3c NORMAL default v6.6 11454
5 pretycx-docker06-V15 K8S_VM 192.168.4.21 fa:16:3e:93:ef:00 NORMAL default v6.6 11445
6 pretycx-docker3-V4 K8S_VM 192.168.4.219 fa:16:3e:86:b6:c5 NORMAL default v6.6 11458
7 pretycx-nginx1-V6 K8S_VM 192.168.1.82 fa:16:3e:9e:71:67 NORMAL default v6.6 11450
8 pretycx-docker1-V3 K8S_VM 192.168.4.127 fa:16:3e:f5:fd:07 NORMAL default v6.6 11458
9 pretycx-docker04-V19 K8S_VM 192.168.4.57 fa:16:3e:93:ef:24 NORMAL default v6.6 11450
10 pretycx-mid3-V8 K8S_VM 192.168.4.152 fa:16:3e:8b:07:62 NORMAL default v6.6 11458
11 pretycx-voip2-V10 K8S_VM 192.168.1.217 fa:16:3e:7e:e4:e6 NORMAL default v6.6 11450
12 pretycx-mid4-V17 K8S_VM 192.168.4.139 fa:16:3e:79:d4:5c NORMAL default v6.6 11458
13 pretycx-mid1-V1 K8S_VM 192.168.4.133 fa:16:3e:83:46:c2 NORMAL default v6.6 11454
14 pretycx-emqx3-V9 K8S_VM 192.168.4.226 fa:16:3e:93:ef:cd NORMAL default v6.6 11445
15 pretycx-docker2-V11 K8S_VM 192.168.4.210 fa:16:3e:89:00:e4 NORMAL default v6.6 11454
16 pretycx-nginx2-V18 K8S_VM 192.168.1.149 fa:16:3e:fb:3f:d1 NORMAL default v6.6 11454
17 pretycx-mid2-V12 K8S_VM 192.168.4.95 fa:16:3e:f8:50:4f NORMAL default v6.6 11458
18 pretycx-emqx2-V20 K8S_VM 192.168.4.35 fa:16:3e:97:31:2a NORMAL default v6.6 11450
19 pretycx-voip1-V7 K8S_VM 192.168.1.237 fa:16:3e:cd:fe:7b NORMAL default v6.6 11454
218 pretycx-redis1-W31 CHOST_VM 192.168.0.133 fa:16:3e:16:d7:34 NORMAL default update_ebpf_v6.6.5 11188
219 pretycx-es3-W42 CHOST_VM 192.168.0.51 fa:16:3e:57:b7:5d NORMAL default update_ebpf_v6.6.5 11188
220 pretycx-doris_BE3-W38 CHOST_VM 192.168.0.181 fa:16:3e:c6:1e:d3 NORMAL default update_ebpf_v6.6.5 11188
221 pretycx-doris_FE1-W41 CHOST_VM 192.168.0.112 fa:16:3e:c6:1e:8e NORMAL default update_ebpf_v6.6.5 11188
222 pretycx-lvs1-W33 CHOST_VM 192.168.1.241 fa:16:3e:f7:50:41 NORMAL default update_ebpf_v6.6.5 11188
223 pretycx-fdfs1-W24 CHOST_VM 192.168.0.34 fa:16:3e:cc:d6:9a NORMAL default update_ebpf_v6.6.5 11188
224 pretycx-cass1-W37 CHOST_VM 192.168.0.176 fa:16:3e:67:fd:f6 NORMAL default update_ebpf_v6.6.5 11188
225 pretycx-fdfs2-W26 CHOST_VM 192.168.0.54 fa:16:3e:0a:08:16 NORMAL default update_ebpf_v6.6.5 11188
226 pretycx-es1-W34 CHOST_VM 192.168.0.55 fa:16:3e:d0:49:b1 NORMAL default update_ebpf_v6.6.5 11188
227 pretycx-mysql2-W27 CHOST_VM 192.168.0.156 fa:16:3e:ae:bb:c8 NORMAL default update_ebpf_v6.6.5 11188
228 pretycx-doris_BE2-W23 CHOST_VM 192.168.0.65 fa:16:3e:c6:1e:5f NORMAL default update_ebpf_v6.6.5 11188
229 pretycx-es2-W32 CHOST_VM 192.168.0.149 fa:16:3e:a5:88:01 NORMAL default update_ebpf_v6.6.5 11188
230 pretycx-fdfs3-W22 CHOST_VM 192.168.0.139 fa:16:3e:29:b1:57 NORMAL default update_ebpf_v6.6.5 11188
231 pretycx-cass2-W36 CHOST_VM 192.168.0.60 fa:16:3e:da:39:51 NORMAL default update_ebpf_v6.6.5 11188
232 pretycx-redis2-W21 CHOST_VM 192.168.0.146 fa:16:3e:6a:4d:12 NORMAL default update_ebpf_v6.6.5 11188
233 pretycx-redis5-W29 CHOST_VM 192.168.0.31 fa:16:3e:34:b6:c3 NORMAL default update_ebpf_v6.6.5 11188
234 pretycx-cass3-W35 CHOST_VM 192.168.0.99 fa:16:3e:86:0a:7b NORMAL default update_ebpf_v6.6.5 11188
235 pretycx-mysql1-W39 CHOST_VM 192.168.0.52 fa:16:3e:75:51:49 NORMAL default update_ebpf_v6.6.5 11188
236 pretycx-redis3-W28 CHOST_VM 192.168.0.233 fa:16:3e:14:12:83 NORMAL default update_ebpf_v6.6.5 11188
237 pretycx-redis4-W25 CHOST_VM 192.168.0.119 fa:16:3e:de:fa:72 NORMAL default update_ebpf_v6.6.5 11188
238 pretycx-doris_BE1-W30 CHOST_VM 192.168.0.215 fa:16:3e:c6:1e:f5 NORMAL default update_ebpf_v6.6.5 11188
239 pretycx-lvs2-W40 CHOST_VM 192.168.1.38 fa:16:3e:f7:50:75 NORMAL default update_ebpf_v6.6.5 11188
[root@pretycx-master1 ~]#

Kubernetes CNI

k8s 1.19.16 +calico

Operation-System/Kernel version

[root@pretycx-master1 ~]# awk -F '=' '/PRETTY_NAME/ { print $2 }' /etc/os-release
"Anolis OS 7.9"
[root@pretycx-master1 ~]#
[root@pretycx-master1 ~]# uname -r
4.19.91-28.2.an7.x86_64
[root@pretycx-master1 ~]#

Anything else

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct