Merge branch 'doc' into develop

README.md

@@ -18,20 +18,6 @@ problems with a production-quality code that is extensible and maintainable,
 using hardware resources efficiently, even for problems with _a priori_
 unknown, heterogeneous, and dynamic load distribution.
 
-## Directory layout
-
-    quinoa
-      ├── cmake/ - CMake code, shared between github.com/quinoacomputing/quinoa-tpl and src/
-      ├── doc/ - Documentation, rendered at quinoacomputing.org
-      ├── external/ - External packages from github.com/quinoacomputing/quinoa-tpl
-      ├── src/ - Compilable sources, see quinoacomputing.org/files.html
-      ├── tests/ - Unit-, and regression tests
-      ├── tools/ - Development utilities and docker files
-      ├── LICENSE - Copyright and license
-      └── README.md - This file, rendered at github.com/quinoacomputing/quinoa
-
-## More info
-
 For more details on philosophy, documentation, software design, journal papers,
 license, and contributing see the [documentation](https://quinoacomputing.org).
 

doc/pages/build_system.dox

@@ -0,0 +1,113 @@
+/*!
+  @page      build_system Internals
+
+@tableofcontents{xml}
+
+This page discusses the internals of the build system: how it works, how the
+different pieces fit together, and the various ways it can be configured. This
+page is intended for those who would like to understand how the build system
+works and how it can be modified or extended.
+
+@note This page is _not_ a description of how to build the third-party
+libraries or Quinoa. For various ways of building Quinoa, including
+requirements, optional libraries, defaults, and build system options, see the
+page on @ref build "building".
+
+@section build_system_overview Overview
+
+As section _@ref build_stages_ on the @ref build "build page" describes, Quinoa
+is built in two stages:
+
+ 1. Build third-party libraries (TPLs)
+ 2. Build Quinoa
+
+Both stages consist of the following two steps
+
+ - Use cmake to configure the build
+ - Perform the build
+
+As the section on @ref build_nonrecursive "non-recursive clones" discusses,
+some of the [cmake code](https://github.com/quinoacomputing/cmake-modules) is
+abstracted away into its own git repository so that it can be effectively
+shared between the two repositories:
+[quinoa](https://github.com/quinoacomputing/quinoa) and
+[quinoa-tpl](https://github.com/quinoacomputing/quinoa-tpl). For more details,
+see also the page on @ref git_submodules_subtrees "modules".
+
+@section build_system_tpl Third-party-libraries build
+
+In a @ref build_nonrecursive "recursive clone" the main entry point of the TPL-build is [external/CMakeLists.txt](https://github.com/quinoacomputing/quinoa-tpl/blob/master/CMakeLists.txt). The TPL-build uses cmake's [external project](https://cmake.org/cmake/help/latest/module/ExternalProject.html) functionality to build the third-party libraries.
+
+The following main steps are taken by the TPL-cmake script to configure the build of the third-party libraries:
+
+ - In-source builds are disallowed in [cmake/DisallowInSourceBuilds.cmake](https://github.com/quinoacomputing/cmake-modules/blob/master/DisallowInSourceBuilds.cmake).
+
+ - The default build type is set in [cmake/BuildType.cmake](https://github.com/quinoacomputing/cmake-modules/blob/master/BuildType.cmake). This can be configured by the cmake variable `CMAKE_BUILD_TYPE`. The default is _Release_. Other valid options are described at https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html.
+
+ - The operating system is detected in [cmake/DetectOS.cmake](https://github.com/quinoacomputing/cmake-modules/blob/master/DetectOS.cmake). This is mostly used for working around some quirks for some specific operating systems, e.g., Alpine Linux.
+
+ - Whether a dynamically or statically linked build is performed is decided in [cmake/BuildSharedS.cmake](https://github.com/quinoacomputing/cmake-modules/blob/master/BuildShared.cmake). This can be configured by the cmake variable `BUILD_SHARED_LIBS` (on/off). The default is _off_, yielding a dynamic build (except on Crays, where the default is static).
+
+   @warning If `BUILD_SHARED_LIBS` is configured one way for the TPL-build, it is important to set the same for the Quinoa build as well. The default is _off_ for both, yielding shared objects for the third party libraries as well as for the Quinoa build, allowing dynamic linking, which, in general is less error-prone for a small runtime cost (which is negligible compared to the number crunching cost).
+
+ - The target build architecture is detected and configured in [cmake/TargetArch.cmake](https://github.com/quinoacomputing/cmake-modules/blob/master/TargetArch.cmake). This is used for setting the correct architecture-specific configuration for, e.g., the runtime system, Charm++.
+
+ - The install directory in which to install all third-party libraries is configured by the `CMAKE_INSTALL_PREFIX` cmake variable. For a @ref build_nonrecursive "non-recursive clone" the default is `quinoa/external/install/<compiler>-<architecture>/`, e.g., `quinoa/external/install/clang-x86_64/`. If a custom location is set, the subsequent Quinoa build can be told by setting the `TPL_DIR` cmake variable to the path used as `CMAKE_INSTALL_PREFIX` for the TPL build.
+
+ - Next the TPL cmake run detects the number of CPUs available on the build machine. This is stored in the cmake variable `PROCESSOR_COUNT` and is used to configure longer-running regression tests that need more than a few CPUs, e.g., the whole build machine.
+
+ - Next we set a number of cmake boolean variables, named as `ENABLE_<SOME_TPL>`, e.g., `ENABLE_CHARM`, `ENABLE_HDF5`, etc. These are then selectively set to _true_ if a requested executable requires them. This enables configuring the build system to only build (and only build third-partly libraries for) selected executables. On how to build only a single executable, see @ref build_specific_executables.
+
+  @note The the default is to build all executables and their required TPLs. If you only care about a single executable, e.g., because you are looking for shorter build-types or less disk space, configuring the TPL-build using the `<executable>_ONLY` is your option.
+
+ - After this we configure the Charm++ build: architecture, communication sub-system, SMP mode, etc.
+
+ - Next the compilers are configured. At this time, the TPL build requires Fortran, C, and a C++ compiler. Also required is MPI, e.g., OpenMPI. The Quinoa build only requires C and C++ compilers. The cmake build echoes a detailed output to screen about the MPI wrappers and the underlying compilers configured/found. Also echoed are extra Charm++ arguments, which can be provided on top of the default ones by the cmake variable `CHARM_EXTRA_ARGS`.
+
+  @note While Charm++ does not require MPI, since it provides its own communication layer, at this time, Quinoa _requires_ Charm++ to be built using one the MPI-backends of Charm++. This is because we use MPI-only third-party libraries (which are _not_ aware that they are running on top of Charm++), and thus MPI libraries are required. We are currently exploring how to relax this requirement, by, e.g., providing the MPI layer using Charm++'s adaptive MPI (AMPI), but this is not fully functional at this time.
+
+ - Next are compiler-specific configuration blocks: default standard library, default compiler flags, etc. This section is also a good place to look for to find out what compiler we test, tested or at least tried to use in past.
+
+ - This is followed by multiple configuration blocks that are specific to individual third-party libraries if enabled. In each of these block, first we try to find the library on the system by invoking cmake code reused between the TPL and the Quinoa builds. These are given in the `Find_<TPL>.cmake` files that live under [cmake/BuildType.cmake](https://github.com/quinoacomputing/cmake-modules].
+
+ - After these TPL-specific configuration blocks, there are the calls to cmake's `ExternalProject_Add()` functions, parameterized to the given TPL. These are the sections to look at to see how an individual TPL is built, e.g., whether it uses autotools or cmake, or what it takes to install a given TPL to make suitable for Quinoa. These are also the sections that potentially apply patches that are not (yet) communicated upstream to the given TPL.
+
+@section build_system_quinoa Quinoa build
+
+In the Quinoa repository the main entry point of the Quinoa-build is [src/CMakeLists.txt](https://github.com/quinoacomputing/quinoa/blob/master/src/CMakeLists.txt). The following main steps are taken by the Quinoa-cmake script to configure the build. Many of the steps are the same (or very similar) to configuring the TPL build. See the @ref build_system_tpl for more details.
+
+ - In-source builds are disallowed. See the @ref build_system_tpl for more details.
+
+ - The default build type is set. This can be configured by the cmake variable `CMAKE_BUILD_TYPE`. The default is _Release_. See the @ref build_system_tpl for more details.
+
+ - The operating system is detected. See the @ref build_system_tpl for more details.
+
+ - Whether a dynamically or statically linked build is performed is decided. The default is dynamic linking. See the @ref build_system_tpl for more details.
+
+ - The target build architecture is detected and configured. See the @ref build_system_tpl for more details.
+
+ - The default TPL directory, in the cmake variable `TPL_DIR`, is set to the install directory that is the default in the cmake variable `CMAKE_INSTALL_PREFIX` in the TPL build. See the @ref build_system_tpl for more details.
+
+ - There is an option to disable tests. The default is to enable both unit-, and regression tests. This is controlled by the cmake variable `ENABLE_TESTS`.
+
+ - We then detect all TPLs and enable all executable targets for which TPLs are found.
+
+   @warning At this point no TPLs are _required_. If a TPL is not found, the executable target whose requriements are not found will be disabled. This enables building only specific executables.
+
+ - Similar to the TPL-cmake configuration, there is a detailed information to screen on the compilers: MPI wrappers and their underlying compilers.
+
+ - There is way to pass extra arguments to the linker, configured by the cmake variable `EXTRA_LINK_ARGS`.
+
+ - Extra compiler flags can be set by the cmake variables, `CMAKE_C_FLAGS` and `CMAKE_CXX_FLAGS`.
+
+   @note All of Quinoa is built using a C++ compiler. At this time the C compiler is only required for detecting the system architecture and some details on the HDF5 library.
+
+ - This is followed by compiler-specific settings and flags, e.g., enabling and disabling certain warnings for all of Quinoa. Some warnings are only locally disabled under directory `src/NoWarning/`.
+
+@section build_system_reated Related pages
+- @ref mainpage_build "Quickstart on how to build"
+- @ref git_submodules_subtrees "Third-party libraries and cmake code"
+- @ref licenses "A list of all required and optional third-party libraries"
+- @ref build_system "Build system internals"
+
+*/

doc/pages/contributing.dox

@@ -48,8 +48,10 @@ This can be done by relying on the existing software infrastructure, including
 - Distributed parallel file I/O, with partitioners, converters, and
   solution-adaptive refinement for tetrahedron meshes,
 
-- Automated testing infrastructure using multiple compilers, operating systems,
-  libraries, container technology, and code quality analysis,
+- Automated testing infrastructure, capable of testing serial, synchronous
+  (e.g., MPI) parallel, and asynchronous (e.g., Charm++) parallel functions and
+  executables, using multiple compilers, operating systems, libraries, container
+  technology, and code quality analysis,
 
 - Beautiful, professional, and no-nonsense documentation, featuring an
   expertly-designed, as-you-type search-engine, whose result looks great on any
@@ -97,7 +99,7 @@ Guidelines for bug reports:
    HEAD of the `develop` branch.
 
 3. Try to see if the bug can be reproduced by any of the regression tests
-   — see the @ref build "build page" on how to run the regression tests.
+   — see the @ref build "build page" on how to run them.
 
 4. Try to reproduce the bug in `Debug` mode — this can be configured by
    setting `CMAKE_BUILD_TYPE=Debug`.
@@ -113,7 +115,7 @@ Example:
 > Short and descriptive example bug report title
 >
 > A summary of the issue and the OS environment in which it occurs. If suitable,
-> include the steps required to reproduce the bug.
+> include the steps required to reproduce the bug:
 >
 > 1. This is the first step.
 > 2. This is the second step.
@@ -124,11 +126,12 @@ Example:
 > _Quinoa-cmake_ build configuration steps. See @ref build_stages.
 >
 > Attach the input file and, if relevant, the input mesh file. If the mesh is
-> large, use our [file transfer service](https://transfer.lanl.gov) and address
-> the file to [email protected].
+> large, try to reproduce it with a smaller mesh. If you cannot, use our [file
+> transfer service](https://transfer.lanl.gov) to send us the larger mesh and
+> address the file to [email protected].
 >
 > Attach the full screen output of the run reproducing the problem, including
-> those of the Charm++ runtime system as well as the _call and stack traces_.
+> those of the Charm++ runtime system as well as the call and stack traces.
 >
 > Include any other information you want to share that is relevant to the issue
 > being reported. This might include the lines of code that you have identified
@@ -142,10 +145,10 @@ output. If this happens, don't panic! ;-) Such bugs are usually due to the fact
 that in Charm++ (and thus in Quinoa) execution is asynchronous by default and
 that the runtime system may (and likely will) schedule messages and tasks (and
 thus execution) in a non-deterministic fashion (while still adhering to
-correctness as specified by the programmer).  However, if the programmer misses
-a single path of many possible paths of execution, that is incorrect, that can
-lead to data races and other bugs and only appear randomly. (Which we do all
-the time — that's why we have an extensive test suite.)
+correctness as specified by the programmer). However, if the programmer misses
+a single path of many possible paths of execution, that is incorrect, which may
+lead to data races and other bugs and thus may only appear randomly. (Which we
+do all the time — that's why we have an extensive test suite.)
 
 Such non-deterministic bugs are a bit a harder to reproduce. Here are some
 techniques that have helped us in the past trigger such bugs sooner rather than
@@ -154,15 +157,17 @@ later:
 - If the bug is reproduced in `Release` mode, see if it can also be reproduced
   in `Debug` mode.
 
-- Build the Charm++ runtime system using @ref build_rndq "randomized message queues".
+- Build the Charm++ runtime system using @ref build_rndq "randomized message
+  queues". This mode increases the chances of appearance of such
+  non-deterministic errors.
 
 - Run the case (or subset of regression tests) you believe will trigger the
   problem in an infinite shell loop _and at the same time_ load all CPUs of the
-  machine with something else. For example:
+  machine with something else. For example, do
 
        while ctest -j36 -R asynclogic -L migration --output-on-failure; do :; done | c++filt
 
-   While the above is running, in another terminal window:
+   While the above is running, in another terminal window do
 
        make clean && make -sj36
 
@@ -225,28 +230,28 @@ error messages. Using multiple compilers are not required but recommended since
 our automated testing exercises all of these compilers, and it is more
 time-effective to catch errors and warnings earlier than later.
 
-Additionally, by default we build @ref build "build Quinoa" turning on most (if
-not all) compiler warnings. (Also, our continuous integration testing turns all
+Additionally, by default we @ref build "build Quinoa" turning on most (if not
+all) compiler warnings. (Also, our continuous integration testing turns all
 warnings to errors.) There are some exceptions to this rule:
 
 - some warnings are turned off _globally_ (these are in `src/CMakeListst.txt`)
   under `compiler-specific settings` — a different list for each compiler,
   and
 
-- we must turn off some warnings _locally_, when headers of third-party
-  libraries and Charm++-generated code are included — these are collected
-  under `src/NoWarning/`.
+- we turn off some warnings _locally_, when headers of third-party libraries
+  and Charm++-generated code are included — these are collected under
+  `src/NoWarning/`.
 
 To ensure good code quality we do not deviate from these settings, and we ask
 all contributors to do the same.
 
 @subsection contributing_buildall Build and test all tools before a push
 
 While it is possible to clone, build the dependencies for, build, and work on
-@ref build_specific_executables "only on a single tool", doing so may run the
+@ref build_specific_executables "only a single tool", doing so may run the
 risk of breaking some of the code that such a single-tool build does not use
 without noticing it locally. Thus, we recommend, at least before pushing to a
-GitHub branch (e.g., before a @ref contributing_pr "pull requests", building
+GitHub branch (e.g., before a @ref contributing_pr "pull request", building
 and running the tests for the _full_ Quinoa build. If something breaks, our
 continuous integration will most likely catch it because they test _all_ the
 tools, thus testing the full build _locally_ (before publishing your changes)
@@ -274,7 +279,7 @@ build Charm++ in non-SMP and SMP mode.
 We don't really have coding guidelines. Please try not to diverge significantly
 from the style of the existing code. This includes file and function definition
 headers, comments, and API documentation. Also, please keep line lengths max 80
-characters. See also @ref priorities.
+characters. See also @ref priorities "our priorities for writing code".
 
 @subsection contributing_review Use reviewable.io for discussing pull requests
 
@@ -322,13 +327,14 @@ The basic steps for those who are _not_ members of GitHub's [QuinoaComputing](ht
 
        git checkout -b <topic-branch-name>
 
-4. At this point, you are ready to make your changes! Feel free to @ref
-   resources_contact "ask for help" &mdash; everyone is a beginner at first ;-).
-   If a developer asks you to _rebase_ your PR (or _merge in `develop`_), they're
-   saying that a lot of code has changed, and that you need to update your branch
-   so it's easier to merge. Make sure to update, or add to the tests when
-   appropriate. Patches and features will not be accepted without passing the
-   existing tests. See also the section on @ref contributing_test "how to run the tests".
+4. At this point, you are ready to make your changes or develop your feature!
+   Feel free to @ref resources_contact "ask for help" &mdash; everyone is a
+   beginner at first ;-).  If a developer asks you to _rebase_ your PR (or _merge
+   in `develop`_), they're saying that a lot of code has changed, and that you
+   need to update your branch so it's easier to merge. Make sure to update, or add
+   to the tests when appropriate. Patches and features will not be accepted
+   without passing the existing tests. See also the section on @ref
+   contributing_test "how to run the tests".
 
 5. If you added or changed a feature, make sure to document it accordingly so
    doxygen can generate the correct documentation.
@@ -364,6 +370,7 @@ The basic steps for those who are _not_ members of GitHub's [QuinoaComputing](ht
 
 5. Open a pull request with a clear title and description.
 
-@note If any of the above is unclear, @ref resources_contact "contact us".
+@note If any of the above is unclear, please do not hesitate to @ref
+resources_contact "contact us".
 
 */