fixup! Initial sections for profiling with perf and native debugging

manual/src/cmds/native-debugger.etex

@@ -3,30 +3,34 @@
 
 \section{s:native-debugger-preliminaries}{Preliminaries}
 
-This chapter describes the support for debugging executables built with the native-code compiler \texttt{ocamlopt} using GDB or LLDB on Linux, macOS and BSD. We will call this "native debugging", compared to bytecode debugging supported via \texttt{ocamldebug}.
+This chapter describes the support for debugging executables built with the native-code compiler \texttt{ocamlopt} using GDB or LLDB on Linux, macOS and BSD. We will call this \texttt{native debugging}, compared to bytecode debugging supported via \texttt{ocamldebug} (see chapter~\ref{c:debugger}).
 
 \subsection{ss:native-debugger-dwarf}{DWARF}
 
-The OCaml compiler uses the \href{http://dwarfstd.org/}{DWARF} debugging information file format to describe the debug information it generates. 
+The OCaml compiler uses the \href{http://dwarfstd.org/}{DWARF} debugging information file format to describe the debug information it generates. DWARF is a debugging information file format used by many compilers and debuggers to support source level debugging, and it is used by Linux ELF, macOS Mach-O and FreBSD ELF.
 
-\texttt{DWARF is a debugging information file format used by many compilers and debuggers to support source level debugging. It addresses the requirements of a number of procedural languages, such as C, C++, and Fortran, and is designed to be extensible to other languages. DWARF is architecture independent and applicable to any processor or operating system. It is widely used on Unix, Linux and other operating systems, as well as in stand-alone environments.}
+Within the DWARF standard the compiler specifically uses Call Frame Information (hereafter abbreviated as CFI) to describe a call stack for OCaml code, sections of the runtime written in C. e.g. Garbage Collector and across the Foreign Function Interface (FFI) if the language provides CFI information. (If the language has been compiled to include CFI information).
 
-Within the DWARF standard the compiler specifically uses Call Frame Information (hereafter abbreviated as CFI) to describe a call stack for OCaml generated assembly and sections of the runtime written in C. e.g. Garbage Collector.
+OCaml defines its own calling convention (which is architecture specific and differs from the C calling convention) for how arguments are passed to functions, how values are returned and how registers are used. See [proc.ml] for specific details for your architecture.
 
-OCaml provides basic support for CFI to describe how one function calls another function. It also defines a calling convention which is architecture specific for how arguments are passed to functions and values are returned. Finally it identifies an area of memory that is allocated on the stack called a "call frame". See [proc.ml] for specific details for your architecture.
+The compiler also generates line information that maps instructions back to their location in the source program (e.g. the instruction at address x originated from myprogram.ml line 42). This allows native debuggers to display the OCaml source code for the program being debugged and enables stepping through OCaml source code.
 
 Debuggers often need to view or modify the state of any function that is on the call stack, CFI allows the native debugger to find and inspect this state. CFI information is preserved across the OCaml / C boundary so the OCaml runtime can be debugged.
 
 \subsection{ss:native-debugger-name-mangling}{Name Mangling}
 
 Name mangling is the process for describing how the OCaml compiler generates symbol names for OCaml language constructs. The format of these symbols is important for debuggers, performance and observability tools, to uniquely identify the source function for a symbol and to do so without resource to the original source code. In the absensce of source mappings, you often need to use mangled names to set breakpoints or they will appear in information the native debugger will display. As such knowing how OCaml performs name mangling is important when debugging OCaml programs. OCaml 5.1.1 uses the a name mangling scheme of \texttt{caml<MODULE_NAME>.<FUNCTION_NAME>_<NNN>} where \texttt{NNN} is a randomly generated number. Before 5.1.1 the scheme is uses two underscores as the separator e.g. \texttt{caml<MODULE_NAME>__<FUNCTION_NAME>_<NNN>}. 
 
+\subsection{ss:native-debugger-frame-pointers}{Frame Pointers}
+
+The OCaml native compiler also supports maintaining Frame Pointers, which can be used by a debugger to walk the stack of function calls in a program. The Frame Pointer (also known as the base pointer) is a register (e.g. %rbp on x86_64 or x29 on ARM64) that points to the base of the current stack frame. The stack frame (also known as the activation frame or the activation record) refers to the portion of the stack allocated to a single function call. By saving the frame pointer along with the return address, between stack frames the call stack for OCaml can be maintained. It should be possible to use just frame pointers to debug OCaml programs, similar to debugging plain assembly code.
+
 \section{s:native-debugger-compilation}{Compiling for debugging}
 
-Before debugging OCaml programs, first the native compiler \texttt{ocamlopt} must be installed with CFI emission enabled. CFI emission is controlled by the \texttt{--enable-cfi} flag which is enabled by default. This is sufficient to allow debugging the assembly code generated by \texttt{ocamlopt}. To perform source level debugging code need to be compiled with \texttt{-g} flag that record debugging information for exception backtraces, and generates mappings between assembly and source locations in OCaml. (Note only GDB and LLDB on Linux reliably support this feature).
+Before debugging OCaml programs, first the native compiler \texttt{ocamlopt} must be installed with CFI emission enabled. CFI emission is controlled by the \texttt{--enable-cfi} flag and is enabled by default. This is sufficient to allow debugging the assembly code generated by \texttt{ocamlopt}. To perform source level debugging code need to be compiled with \texttt{-g} flag that records debugging information for exception backtraces, and generates mappings between assembly and source locations in OCaml. (Note only GDB and LLDB on Linux reliably support source locations).
 
 \section{s:native-debugger-gdb}{Using GDB}
-Here we will walk through debugging a simple OCaml program using GDB on Linux, showing the commands to use and what features are expected to work.
+Here we will walk through debugging a simple OCaml program using GDB on Linux, showing the commands to use and the expected outputs.
 
 Consider the following program:
 \begin{caml_example*}{verbatim}
@@ -43,7 +47,7 @@ let main () =
 let _ = main ()
 \end{caml_example*}
 
-Compile this program with ocamlopt:
+Compile this program with ocamlopt like so:
 
 \begin{verbatim}
 $ ocamlopt --version
@@ -53,13 +57,13 @@ $ ./fib.exe 20
 fib(20) = 6765
 \end{verbatim}
 
-When run this program prints the 20th Fibonnaci number, using recursion which allows an opportunity to inspect the call stack. Now startup GDB:
+Then when run this program prints the 20th Fibonnaci number, using recursion allows an opportunity to inspect the call stack. Startup a GDB session for this program:
 
 \begin{verbatim}
 $ gdb ./fib.exe
 \end{verbatim}
 
-Break points can be set either using the mangled names produced by the compiler or using a combination of file name and line number. 
+Break points can be set either using the mangled names produced by the compiler or using a combination of file name and line number. For example:
 
 \begin{verbatim}
 (gdb) break camlFib.fib_     # press tab
@@ -98,9 +102,11 @@ Breakpoint 1, camlFib.fib_270 () at fib.ml:2
 #8  caml_startup (argv=<optimised out>) at runtime/startup_nat.c:144
 #9  caml_main (argv=<optimised out>) at runtime/startup_nat.c:151
 #10 0x000055555558e892 in main (argc=<optimised out>, argv=<optimised out>) at runtime/main.c:37
+\end{verbatim}
 
-# Printing values
+There is basic support for printing OCaml values using \href{https://github.com/ocaml/ocaml/blob/5.3.0/tools/gdb.py}{tools/gdb.py} and the built in Python scripting in GDB. Download that file and load it into GDB like so:
 
+\begin{verbatim}
 (gdb) source ~/ocaml/tools/gdb.py
 OCaml support module loaded. Values of type 'value' will now
 print as OCaml values, there is a $Array() convenience function,
@@ -112,7 +118,8 @@ $1 = caml:14
 
 \end{verbatim}
 
-We can also print other kinds of OCaml values. In order to illustrate this, consider the following program:
+We can also print other kinds of OCaml values.
+In order to illustrate this, consider the following program:
 \begin{caml_example*}{verbatim}
 (* test_blocks.ml *)
 type t = {s : string; i : int}
@@ -125,7 +132,7 @@ let main a b =
 let _ = main "foo" {s = "bar"; i = 42}
 \end{caml_example*}
 
-https://github.com/ocaml/ocaml/blob/5.3.0/tools/gdb.py
+Compile this program with ocamlopt like so and load it into GDB:
 
 \begin{verbatim}
 $ ocamlopt -g -o test_blocks.exe test_blocks.ml
@@ -135,52 +142,147 @@ $ gdb ./test_blocks.exe
 (gdb) break camlTest_blocks.main_273 
 Breakpoint 1 at 0x16db0: file test_blocks.ml, line 4.
 (gdb) run
-Starting program: /home/tsmc/test_blocks.exe 
-[Thread debugging using libthread_db enabled]
-Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
-
+...
 Breakpoint 1, camlTest_blocks.main_273 () at test_blocks.ml:4
 4	let main a b =
-
+(gdb) p (value)$rax      # Print out the first argument to main
+$1 = caml(-):'foo'<3>
+(gdb) p (value)$rbx      # Then print the second argument
+$2 = caml(-):('bar', 42) = {caml(-):'bar'<3>, caml:42}
+(gdb) p *(value*)$rbx@2  # Examine the second field
+$3 = {caml(-):'bar'<3>, caml:42}
 \end{verbatim}
 
-\subsection{ss:native-debugger-gdb-commands}{GDB Commands}
+Note the use of x86_64 register names. We can print values as their OCaml representations (note The (m) or (u) (or (g) or (-)) is the GC color). 
 
+\subsection{ss:native-debugger-gdb-commands}{GDB Commands}
+Summary of interesting OCaml specific GDB commands:
 \begin{options}
 \item["break "\var{locspec}]
 Set a breakpoint at all of the code locations matching \var{locspec}. e.g. Using the mangled OCaml names or specifying the linenum in the source file as \texttt{filename:linenum}.
 
 \item["backtrace"]
-Print the backtrace of the entire stack, will include OCaml source references identifying which stack frame maps to a source location. e.g. fib.ml:4
+Print the backtrace of the entire stack, this will include OCaml source references identifying which stack frame maps to a source location. e.g. fib.ml:4
 
-\item["disassemble"]
-Display a range of addresses as machine instructions. Typically used with the mangled OCaml names to display the assembly for a function. 
+\item["disassemble "\var{addresses}]
+Display a range of \var{addresses} as machine instructions. Typically used with the mangled OCaml names to display the assembly for a function. 
 
-\item["info frame"]
-This command prints a verbose description of the selected stack frame.
+\item["info "\var{frame}]
+This command prints a verbose description of the selected stack \var{frame}.
 
 \item["list "\var{linenum}]
-Print lines centered around line number linenum in the current source file. This will print the source code for OCaml and the OCaml runtime written in C.
+Print lines centered around line number \var{linenum} in the current source file. This will print the source code for OCaml and the OCaml runtime written in C.
 
 \end{options}
 
 See the \href{https://sourceware.org/gdb/current/onlinedocs/gdb.html/}{Debugging with GDB} documentation for more details. In general only the features described above are expected to work in GDB and otherwise users will need to fall back to assembly debugging. GDB is expected to work on all supported Linux architectures.
 
-\section{s:native-debugger-gdb}{Using LLDB}
+\section{s:native-debugger-lldb}{Using LLDB}
 
-https://github.com/ocaml/ocaml/blob/5.3.0/tools/lldb.py
+Here we will walk through debugging the earlier fib example using LLDB on Linux. Startup an LLDB session using the `fib.exe` from earlier:
 
-Tutorial for LLDB - including commands
-\subsection{ss:native-debugger-lldb-commands}{LLDB Commands}
+\begin{verbatim}
+$ lldb ./fib.exe
+Current executable set to 'fib.exe' (aarch64).
+(lldb)
+\end{verbatim}
 
-Printing Values
+Breakpoints can be set using the OCaml mangled names or using a combination of file name and line number. For example:
 
-What should work?
+\begin{verbatim}
+(lldb) breakpoint set -n camlFib.main        # press tab for autocomplete
+(lldb) breakpoint set -n camlFib.main_272 
+Breakpoint 3: where = fib.exe`camlFib.main_272 + 40, address = 0x00000000000510b0
+(lldb) breakpoint set -f fib.ml -l 7         # breakpoint for line 7 in fib.ml
+Breakpoint 2: where = fib.exe`camlFib.main_272, address = 0x0000000000051088
+(lldb)
+\end{verbatim}
 
-\subsection{ss:native-debugger-gdb-commands}{GDB Commands}
+Now we can run the program.
+\begin{verbatim}
+(lldb) run
+Process 11391 launched: '/home/tsmc/fib.exe' (aarch64)
+Process 11391 stopped
+* thread #1, name = 'fib.exe', stop reason = breakpoint 2.1
+    frame #0: 0x0000aaaaaaaf1088 fib.exe`camlFib.main_272 at fib.ml:7
+   4   	  else if n = 1 then 1
+   5   	  else fib (n-1) + fib (n-2)
+   6   	
+-> 7   	let main () = 
+   8   	  let r = fib 20 in 
+   9   	  Printf.printf "fib(20) = %d" r
+   10  	
+warning: This version of LLDB has no plugin for the language "assembler". Inspection of frame variables will be limited.
+(lldb) bt                # Print the backtrace
+* thread #1, name = 'fib.exe', stop reason = breakpoint 2.1
+  * frame #0: 0x0000aaaaaaaf1088 fib.exe`camlFib.main_272 at fib.ml:7
+    frame #1: 0x0000aaaaaaaf117c fib.exe`camlFib.entry at fib.ml:11
+    frame #2: 0x0000aaaaaaaee644 fib.exe`caml_program + 476
+    frame #3: 0x0000aaaaaab45b08 fib.exe`caml_start_program + 132
+    frame #4: 0x0000aaaaaab45600 fib.exe`caml_main [inlined] caml_startup(argv=<unavailable>) at startup_nat.c:145:7
+    frame #5: 0x0000aaaaaab455fc fib.exe`caml_main(argv=<unavailable>) at startup_nat.c:151:3
+    frame #6: 0x0000aaaaaaaee2d0 fib.exe`main(argc=<unavailable>, argv=<unavailable>) at main.c:37:3
+    frame #7: 0x0000fffff7d784c4 libc.so.6`__libc_start_call_main(main=(fib.exe`main at main.c:31:1), argc=1, argv=0x0000fffffffffb78) at libc_start_call_main.h:58:16
+    frame #8: 0x0000fffff7d78598 libc.so.6`__libc_start_main_impl(main=0x0000aaaaaaba0dc8, argc=16, argv=0x000000000000000f, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=<unavailable>) at libc-start.c:360:3
+    frame #9: 0x0000aaaaaaaee370 fib.exe`_start + 48
+(lldb) 
+\end{verbatim}
+
+There is basic support for printing OCaml values using \href{https://github.com/ocaml/ocaml/blob/5.3.0/tools/lldb.py}{tools/lldb.py} and the built in Python scripting in LLDB. Download that file and load it into LLDB like so:
+
+\begin{verbatim}
+(lldb) command script import ~/ocaml/tools/lldb.py
+OCaml support module loaded. Values of type 'value' will now
+print as OCaml values, and an 'ocaml' command is available for
+heap exploration (see 'help ocaml' for more information).
+(lldb)  p (value)$x0
+(value) 41 caml:20
+(lldb) 
+\end{verbatim}
+
+Note we are using an ARM64 Linux machine so our first argument is in the first register x0
+
+We can also print out all kinds of OCaml values. Reusing the `test_blocks.exe` startup a new LLDB session:
+
+\begin{verbatim}
+$ lldb ./test_blocks.exe
+...
+(lldb) command script import ~/ocaml/tools/lldb.py
+...
+(lldb) br s -n camlTest_blocks.main_273 
+Breakpoint 1: where = test_blocks.exe`camlTest_blocks.main_273 + 40, address = 0x0000000000019ab0
+(lldb) run
+...
+(lldb) p (value)$x0
+(value) 187649984891864 caml(-):'Hello, world!'<13>
+(lldb) p (value)$x1
+(value) 187649984891808 caml(-):('bar', 42)
+\end{verbatim}
+
+\subsection{ss:native-debugger-lldb-commands}{LLDB Commands}
+
+Summary of interesting OCaml specific LLDB commands:
 
 \begin{options}
-\item["break "\var{locspec}]
-Set a breakpoint at all of the code locations matching \var{locspec}. e.g. Using the mangled OCaml names or specifying the linenum in the source file as \texttt{filename:linenum}.
+\item["breakpoint set -n "\var{symbol}]
+Set a breakpoint at code location matching \var{symbol}. e.g. Using the mangled OCaml name.
+
+\item["breakpoint set -f "\var{filename}" -l"\var{linenum}]
+Set a breakpoint at \var{linenum} in \var{filename}. e.g fib.ml:7
+
+\item["breakpoint set -a "\var{address}]
+Set a breakpoint on a memory \var{address}.
+
+\item["backtrace"]
+Print the backtrace of the entire stack, will include OCaml source references identifying which stack frame maps to a source location. e.g. fib.ml:4
+
+\item["disassemble"]
+Disassemble specified instructions in the current target. Useful options include \texttt{-n} plus mangled OCaml name to disassemble a specific function and \texttt{-a} plus an address to disassemble function containing this address.
+
+\item["frame info"]
+List information about the current stack frame in the current thread.
+
+\item["source"]
+Commands for examining source code described by debug information for the current target process.
 
 \end{options}