agamotto login: panic: runtime error: invalid memory address or nil pointer dereference

[roeyjobsyud]

Hi guys,
Trying to run agamotto on GCE instance.
Compiled and installed the host kernel (exactly the same one that you guys used), obviously with the patch and CONFIG_KVM_AGAMOTTO enabled.
Used the same guest kernel, QEMU and syzkaller, all of them patched just as in your installation tutorial.
We face a repetetive error, that occurs with all of the syzkaller configs.

Ran
roi@agamotto1nested:/opt/gopath/src/github.com/google/syzkaller$ ./bin/syz-manager -config /home/roi/agamotto/configs/syzkaller/generated/snapshot-usb.go7007.cfg -debug -vv 999999 > debugged.txt

1176 2020/08/17 13:06:40 executor already running - using existing in/out/err pipes
1177 2020/08/17 13:06:40 makeCommandWithPipes bin:[/usr/bin/ssh -p 1569 -F /dev/null -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o ConnectTimeout=10 -i /home/roi/agamotto/scripts/stretch.id_rsa root@localhost /syz-executor.wrapper]
1178 2020/08/17 13:06:40 performing handshake with an already running executor...
1179 periscope: syz-fuzzer receive handshake (magic=0xbadc0ffeebadface)
1180 periscope: paddr=0x12ce5000 for vaddr=0x7f75f8955000 l=0x18
1181 periscope: syz-fuzzer receive handshake (size=24)
1182 periscope: guest agent did not request shutdown 0
1183 [^[[0;32m  OK  ^[[0m] Started /etc/rc.local Compatibility.^M
1184 [^[[0;32m  OK  ^[[0m] Started Serial Getty on ttyS0.^M
1185 [^[[0;32m  OK  ^[[0m] Started Getty on tty1.^M
1186 [^[[0;32m  OK  ^[[0m] Reached target Login Prompts.^M
1187 [^[[0;32m  OK  ^[[0m] Reached target Multi-User System.^M
1188 [^[[0;32m  OK  ^[[0m] Reached target Graphical Interface.^M
1189          Starting Update UTMP about System Runlevel Changes...^M
1190 [^[[0;32m  OK  ^[[0m] Started Update UTMP about System Runlevel Changes.^M
1191 ^M^M
1192 Debian GNU/Linux 9 agamotto ttyS0^M
1193 ^M
1194 agamotto login: panic: runtime error: invalid memory address or nil pointer dereference
1195 [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8276dd]
1196
1197 goroutine 57 [running]:
1198 main.(*Proc).executeRaw(0xc00018ff40, 0xc00039ded8, 0xc000036040, 0x0, 0x0)
1199         /opt/gopath/src/github.com/google/syzkaller/syz-fuzzer/proc.go:349 +0xdd
1200 main.(*Proc).execute(0xc00018ff40, 0xc00039ded8, 0xc000036040, 0x0, 0x0, 0x2)
1201         /opt/gopath/src/github.com/google/syzkaller/syz-fuzzer/proc.go:299 +0x6a
1202 main.(*Proc).loop(0xc00018ff40)
1203         /opt/gopath/src/github.com/google/syzkaller/syz-fuzzer/proc.go:113 +0x3b6
1204 created by main.main
1205         /opt/gopath/src/github.com/google/syzkaller/syz-fuzzer/fuzzer.go:449 +0x166a

attaching debugged.txt

[zzoru]

Trying to run agamotto on GCE instance.

Agamotto relies on KVM virtualization, and maybe your GCE instance should support nested virtualization
Can you check that your GCE instance supports nested virtualization?
(https://cloud.google.com/compute/docs/instances/enable-nested-virtualization-vm-instances#restrictions)

[roeyjobsyud]

Hi, thanks for your response.
Yea it does, I enabled nested virtualization in advance, before installing Agamotto.

[shuidixu]

hello
@zzoru i have the same issue! can you help solve this? it is very important for me to solve the issue.

[dokyungs]

We only tested Agamotto in a non-nested, bare-metal virtualization environment. Would it be possible for you to use the same environment? @roeyjobsyud @shuidixu

[shiftre]

Hey all,

Seeing as this thread is a bit stuck I will try to emphasize our attempts and work.

First, we are thankful about your response. We value your academic work and opensource contribution.
Agamotto looks like a very good candidate for fuzzing and we would like to try it on ourselves.

As @roeyjobsyud pointed out we setup a nested vm environment and set on our luck with trying to fuzz
native syscalls, but as it looks like now, the syz-executor.wrap binary is not being executed on the target machine as it does not exist there, rather only syz-executor.debug exist which seem to wrap syz-executor.

I believe the tool is actually working on regular vanilla syscalls but I am not sure, it seems the original crash occurs due to an actual timeout that syz-manager generates a program but it never reaches the guest machine to get executed/fuzzed.

Moreover, I don't fully understand how snapshotting works as I can only see the agent-chkpt binary responsible for executing a snapshot request but nothing occurs on the host side to actually forward and execute another machine with a new state, can you explain how the process works?

To summary, my questions are:

1. What happens when a new program is being sent to syz-fuzzers.debug, (assuming this is the right binary to execute, as I did
   not find syz-fuzzer.wrap)?
2. Who is responsible to perform a checkpoint request upon each syscall? Is it possible that agent-chkpt needs to be called manually during critical points?
3. Who is responsible to spin up a new machine once a checkpoint has been created and continue execution?

Thanks for the help :)