Skip to content

Commit

Permalink
Merge pull request #2842 from boutproject/remove-backtrace-example
Browse files Browse the repository at this point in the history
Remove `backtrace` example, move into docs instead
  • Loading branch information
ZedThree authored Feb 14, 2024
2 parents a630622 + de8b50d commit 7be2be9
Show file tree
Hide file tree
Showing 9 changed files with 174 additions and 130 deletions.
1 change: 0 additions & 1 deletion examples/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ add_subdirectory(IMEX/drift-wave)
add_subdirectory(IMEX/drift-wave-constraint)
add_subdirectory(advdiff)
add_subdirectory(advdiff2)
add_subdirectory(backtrace)
add_subdirectory(blob2d)
add_subdirectory(blob2d-outerloop)
add_subdirectory(blob2d-laplacexz)
Expand Down
1 change: 0 additions & 1 deletion examples/backtrace/.gitignore

This file was deleted.

9 changes: 0 additions & 9 deletions examples/backtrace/CMakeLists.txt

This file was deleted.

40 changes: 0 additions & 40 deletions examples/backtrace/README.md

This file was deleted.

32 changes: 0 additions & 32 deletions examples/backtrace/backtrace.cxx

This file was deleted.

5 changes: 0 additions & 5 deletions examples/backtrace/data/BOUT.inp

This file was deleted.

146 changes: 146 additions & 0 deletions manual/sphinx/developer_docs/debugging.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
.. _sec-debugging:

==================
Debugging Models
==================

When developing a new physics model, or using an existing one in new
regimes, it's expected that things will occasionally go wrong and
you'll need to debug the program. While debuggers like ``gdb`` are
very powerful, using them in parallel can be difficult unless you have
access to a dedicated parallel debugger. BOUT++ has some utilities
that make it easier to debug issues that only arise in parallel and/or
long-running simulations.

Loggers
=======

The first of these is the standard "write to screen" with the
``output.write`` :ref:`family of logging functions <sec-logging>`. If
you have a bug which is easily reproducible and occurs almost
immediately every time you run the code, then this is probably the
easiest way to hunt it down.

The main downside of (most of) these loggers is that if you have a lot of
output they will slow down simulations. Even if you use the
``--quiet`` command line option to turn them off, they will still add
some overhead. The `output_debug` logger can be disabled entirely at
compile-time (so there will be no overhead at all), which means it's
well suited to adding in-depth diagnostic or debugging information
that can be kept permanently in the code and only enabled if needed.

To enable the ``output_debug`` messages, configure BOUT++ with a
``CHECK`` level ``>= 3``. To enable it at lower check levels,
configure BOUT++ with ``-DENABLE_OUTPUT_DEBUG``. When running BOUT++
add a ``-v -v`` flag to see ``output_debug`` messages.

Message Stack
=============

The second utility BOUT++ has to help debugging is the message stack
using the `TRACE` (and related `AUTO_TRACE`) macro. These are very
useful for when a bug only occurs after a long time of running, and/or
only occasionally. The ``TRACE`` macro can simply be dropped in
anywhere in the code::

{
TRACE("Some message here"); // message pushed

} // Scope ends, message popped

This will push the message, then pop the message when the current
scope ends. If an error occurs or BOUT++ crashes, any un-popped
messages will be printed, along with the file name and line number, to
help find where an error occurred. For example, given this snippet::

{
TRACE("1. Outer-most scope");
{
TRACE("2. Middle scope");
{
TRACE("3. Scope not appearing in the output");
}
{
TRACE("4. Inner-most scope");
throw BoutException("Something went wrong");
}
}
}

we would see something like the following output:

.. code:: text
====== Back trace ======
-> 4. Inner-most scope on line 58 of '/path/to/model.cxx'
-> 2. Middle scope on line 53 of '/path/to/model.cxx'
-> 1. Outer-most scope on line 51 of '/path/to/model.cxx'
====== Exception thrown ======
Something went wrong
The third ``TRACE`` message doesn't appear in the output because we've
left its scope and it's no longer relevant.

The run-time overhead of this should be small, but can be removed
entirely if the compile-time flag ``-DCHECK`` is not defined or set to
``0``. This turns off checking, and ``TRACE`` becomes an empty
macro. This means that ``TRACE`` macros can be left in your code
permanently, providing some simple diagnostics without compromising
performance, as well as demarcating separate sections with
user-friendly names.

If you need to capture runtime information in the message, you can use
the ``fmt`` syntax also used by the loggers::

TRACE("Value of i={}, some arbitrary {}", i, "string");

There is also an ``AUTO_TRACE`` macro that automatically captures the
name of the function it's used in. This is used throughout the main
library, especially in functions where numerical issues are likely to
arise.

Backtrace
=========

Lastly, BOUT++ can also automatically print a backtrace in the event
of a crash. This is a compile-time option in the BOUT++ library
(``-DBOUT_ENABLE_BACKTRACE=ON``, the default, requires the
``addr2line`` program to be installed), and debug symbols to be turned
on (``-DCMAKE_BUILD_TYPE=Debug`` or ``=RelWithDebInfo``) in BOUT++
_and_ the physics model. If debug symbols are only present in part, the
backtrace will be missing names for the other part.

The output looks something like this:

.. code:: text
...
Error encountered
====== Exception path ======
[bt] #10 ./backtrace() [0x40a27e]
_start at /home/abuild/rpmbuild/BUILD/glibc-2.33/csu/../sysdeps/x86_64/start.S:122
[bt] #9 /lib64/libc.so.6(__libc_start_main+0xd5) [0x7fecbfa28b25]
__libc_start_main at /usr/src/debug/glibc-2.33-4.1.x86_64/csu/../csu/libc-start.c:332
[bt] #8 ./backtrace() [0x40a467]
main at /path/to/BOUT-dev/build/../examples/backtrace/backtrace.cxx:32 (discriminator 9)
[bt] #7 /path/to/BOUT-dev/build/libbout++.so(_ZN6Solver8setModelEP12PhysicsModel+0xb5) [0x7fecc0ca2e93]
Solver::setModel(PhysicsModel*) at /path/to/BOUT-dev/build/../src/solver/solver.cxx:94
[bt] #6 /path/to/BOUT-dev/build/libbout++.so(_ZN12PhysicsModel10initialiseEP6Solver+0xc0) [0x7fecc0cad594]
PhysicsModel::initialise(Solver*) at /path/to/BOUT-dev/build/../include/bout/physicsmodel.hxx:93 (discriminator 5)
[bt] #5 ./backtrace() [0x40a986]
Backtrace::init(bool) at /path/to/BOUT-dev/build/../examples/backtrace/backtrace.cxx:27
[bt] #4 ./backtrace() [0x40a3cf]
f3() at /path/to/BOUT-dev/build/../examples/backtrace/backtrace.cxx:19
[bt] #3 ./backtrace() [0x40a3be]
f2(int) at /path/to/BOUT-dev/build/../examples/backtrace/backtrace.cxx:15
[bt] #2 ./backtrace() [0x40a386]
f1() at /path/to/BOUT-dev/build/../examples/backtrace/backtrace.cxx:13 (discriminator 2)
[bt] #1 ./backtrace(_ZN13BoutExceptionC1IA19_cJEEERKT_DpRKT0_+0xba) [0x40ae16]
BoutException::BoutException<char [19]>(char const (&) [19]) at /path/to/BOUT-dev/build/../include/bout/../boutexception.hxx:28 (discriminator 2)
This output tends to be much harder to read than the message stack
from ``TRACE`` macros, but the advantage is that it doesn't require
any modifications to the code to use, and can give you more precise
location information.
1 change: 1 addition & 0 deletions manual/sphinx/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ The documentation is divided into the following sections:
developer_docs/petsc_interface
developer_docs/mesh
developer_docs/file_io
developer_docs/debugging
developer_docs/natural_language
developer_docs/performance_profiling

Expand Down
69 changes: 27 additions & 42 deletions manual/sphinx/user_docs/physics_models.rst
Original file line number Diff line number Diff line change
Expand Up @@ -636,43 +636,28 @@ Finding where bugs have occurred in a (fairly large) parallel code is
a difficult problem. This is more of a concern for developers of
BOUT++ (see the developers manual), but it is still useful for the
user to be able to hunt down bug in their own code, or help narrow
down where a bug could be occurring.
down where a bug could be occurring. BOUT++ comes with a `TRACE` macro
that can be used to easily identify specific regions in a model when
an error occurs.

If you have a bug which is easily reproduceable i.e. it occurs almost
immediately every time you run the code, then the easiest way to hunt
down the bug is to insert lots of ``output.write`` statements (see
:ref:`sec-logging`). Things get harder when a bug only occurs after a
long time of running, and/or only occasionally. For this type of
problem, a useful tool can be the message stack. An easy way to use
this message stack is to use the `TRACE` macro::

{
TRACE("Some message here"); // message pushed

} // Scope ends, message popped

This will push the message, then pop the message when the current
scope ends (except when an exception occurs). The error message will
also have the file name and line number appended, to help find where
an error occurred. The run-time overhead of this should be small, but
can be removed entirely if the compile-time flag ``-DCHECK`` is not
defined or set to ``0``. This turns off checking, and ``TRACE``
becomes an empty macro. It is possible to use standard ``printf``
like formatting with the trace macro, for example::

{
TRACE("The value of i is %d and this is an arbitrary %s", i, "string"); // message pushed
} // Scope ends, message popped

In the ``mhd.cxx`` example each part of the ``rhs`` function is
trace'd. If an error occurs then at least the equation where it
happened will be printed::
In the ``mhd.cxx`` example each part of the ``rhs`` function has a
separate ``TRACE`` macro::

{
TRACE("ddt(rho)");
ddt(rho) = -V_dot_Grad(v, rho) - rho*Div(v);
}

If there's a problem here that causes the model to crash, BOUT++ will
print something like:

.. code:: text
====== Back trace ======
-> ddt(rho) on line 83 of 'examples/orszag-tang/mhd.cxx'
For more details on what you can do with ``TRACE`` macros, see
:ref:`sec-debugging`.

.. _sec-physicsmodel-boundary-conditions:

Expand Down Expand Up @@ -1069,7 +1054,7 @@ styles. For example::

output.write("This is an integer: {}, and this a real: {}\n", 5, 2.0)

output << "This is an integer: " << 5 << ", and this a real: " << 2.0 << endl;
output << "This is an integer: " << 5 << ", and this a real: " << 2.0 << '\n';


Formatting in the ``output.write`` function is done using the `{fmt}
Expand All @@ -1096,7 +1081,7 @@ For finer control over which messages are printed, several outputs are
available, listed in the table below.

=================== =================================================================
Name Useage
Name Usage
=================== =================================================================
``output_debug`` For highly verbose output messages, that are normally not needed.
Needs to be enabled with a compile switch
Expand All @@ -1110,20 +1095,20 @@ Name Useage
Controlling logging level
~~~~~~~~~~~~~~~~~~~~~~~~~

By default all of the outputs except ``output_debug`` are saved to log
and printed to console (processor 0 only).
By default all of the outputs (except ``output_debug``) are saved to
log and printed to console (processor 0 only).

To reduce the volume of outputs the command line argument ``-q``
(quiet) reduces the output level by one, and ``-v`` (verbose)
increases it by one. Running with ``-q`` in the command line arguments
suppresses the ``output_info`` messages, so that they will not appear
in the console or log file. Running with ``-q -q`` suppresses
everything except ``output_warn`` and ``output_error``.
To reduce the volume of outputs the command line argument ``--quiet``
(``-q`` for short) reduces the output level by one, and ``--verbose``
(``-v`` for short) increases it by one. Running with ``-q`` in the
command line arguments suppresses the ``output_info`` messages, so
that they will not appear in the console or log file. Running with
``-q -q`` suppresses everything except ``output_warn`` and
``output_error``.

To enable the ``output_debug`` messages, configure BOUT++ with a
``CHECK`` level ``>= 3``. To enable it at lower check levels,
configure BOUT++ with ``-DENABLE_OUTPUT_DEBUG`` (for ``CMake``;
``--enable-debug-output`` for ``./configure``)). When running BOUT++
configure BOUT++ with ``-DBOUT_ENABLE_OUTPUT_DEBUG=ON``. When running BOUT++
add a ``-v -v`` flag to see ``output_debug`` messages.

.. _sec-3to4:
Expand Down

0 comments on commit 7be2be9

Please sign in to comment.