CIVET Cases to Run

[pbauman]

@roystgnr

I'm setting up femputer with a CIVET server and client (server is up and running, working on client and software stack).

What do we want to have CIVET run for GRINS? I'm hoping to setup PR and nightly testing (this remains to be seen how to do this with CIVET).

Here were my initial thoughts.

PR:
make check:
dbg (with GLIBCXX_DEBUG, serial, with Cantera+Antioch)
opt (serial, with Cantera+Antioch)
opt (parallel, 4 processors, with Cantera+Antioch)
dbg (with GLIBCXX_DEBUG, serial, without Cantera+Antioch)

Nightly:
make check: dbg (with GLIBCXX_DEBUG, serial, with Cantera+Antioch)
dbg (with GLIBCXX_DEBUG, serial, without Cantera+Antioch)
devel (serial, with Cantera+Antioch)
opt (serial, with Cantera+Antioch)
opt (parallel, 2-16 (24?) processors ala the run parallel script we have, with Cantera+Antioch))
valgrind dbg (I'm not quite sure how to do this yet)

make run_examples (this would need to be set up in the GRINS build system):
dbg (with GLIBCXX_DEBUG, serial, with Cantera+Antioch)
devel (serial, with Cantera+Antioch)
opt (serial, with Cantera+Antioch)
opt (parallel, 4 processors, with Cantera+Antioch)

The question is against what libMesh? Do we take a special hash and only update when needed, e.g. when a GRINS PR uses a new feature in libMesh? Do we have CIVET pull and update libMesh before each PR and build against that? My thinking is for GRINS PRs we have a special hash of libMesh we build against and only update when needed and then nightly tests can pull the latest tip of libMesh master before starting.

Similarly for Antioch?

Also, what versions of dependencies for libMesh (PETSc, Boost, HDF5)? How many GCC versions? Other thoughts?

[pbauman]

I should've mentioned - femputer doesn't have a graphics card, which means no OpenGL which means no VTK.

[pbauman]

Nevermind, I just googled and it should be possible to build VTK without OpenGL, it will just take some massaging of the build script. Will get to that later.

[roystgnr]

We definitely want a dbg parallel case going; better speed for the slowest tests plus better code coverage is just win-win. Just running on a fixed N processors would be sufficient; no need for the "for N in 1 to 24" loop we'll want for solver option testing.

Valgrind checks are a bit redundant with dbg checks, aren't they? Rather than slow-squared we might want to run valgrind on devel mode. This plus dbg would probably make the normal devel run superfluous.

IMHO we ought to save a "minimum libMesh hash" in GRINS somewhere. Version number testing isn't enough, unless I get out of the habit of using GRINS as a frequent test bed for new libMesh features. We could then run tests against both the saved hash and the HEAD of libMesh/master. Using the saved hash for PRs and the HEAD for nightlies sounds like the best way to go.

Saving a "minimum compatible GCC version" then testing against both that and the latest would be a good idea. John's C++11 tests have saved me from accidentally breaking libMesh C++98 compatibility once or twice, and unless the future GRINS requirements look like "C++2x-prerelease or bugger off" we'll probably have similar concerns. As far as other dependencies go, I think PETSc is the only one that's really given us compatibility hell, and if we're doing our jobs right in libMesh then the messy details of that ought to be hidden away by the time GRINS is in the picture.

[dmcdougall]

no need for the "for N in 1 to 24" loop we'll want for solver option testing.

I definitely see use in testing a few values of N. Just to build confidence bugs aren't due to mesh partitioning or such.

[roystgnr]

Hmm... the point here is to catch bugs with Civet. Once the bug is found on dbg in parallel, we can manually run on dbg in serial to figure out what the problem is.

FYI, though, we currently have a problem with dbg: if we're building against a recent version of CPPUnit, then we're not ABI compatible when we compile our own code with _GLIBCXX_DEBUG defined, and unit_driver crashes at runtime. (not sure why this isn't caught at link time!) It would be nice to just autodetect and respect the libMesh configure --enable-glibcxx-debugging-cppunit settings, but I don't think that gets exported in any way.

[pbauman]

FYI, though, we currently have a problem with dbg: if we're building against a recent version of CPPUnit, then we're not ABI compatible when we compile our own code with _GLIBCXX_DEBUG defined, and unit_driver crashes at runtime.

Yeah, I've locally had a cppunit-debug module for this case and I plan to do the same on femputer. So, for dbg builds, all the relevant C++ modules will be debug versions.

[pbauman]

What about distcheck? I was thinking "nightly", although it's looking like "nightly" is going to "when anything is merged to master" (which I think I'm OK with since we can similarly trigger the Antioch repo to build against latest GRINS when something gets pushed to its master).

[roystgnr]

If "nightly" gets redefined as "post merge" then yeah, throwing in a distcheck too sounds good.

[pbauman]

OK, I have the first recipe (just serial grins devel +Antioch +Cantera, GCC 6.2) up, running, and passing through the whole cycle. It looks like, given libMesh/libmesh#1194, we'll want to make 1bb0b5 the current minimum hash for libMesh to test PRs against. (We should probably put that in the README...). I'll try to get more CIVET recipes setup tonight, but at the very least, we have one up and running now.

@roystgnr had mentioned echoing the output from the failed test. Need to not forget that. Should be easy enough to grep for the failed test name and just cat the log file for each of the failed tests. Will probably take a couple of iterations to get the output pretty though.

[pbauman]

We need to add a make target for running all the examples and add that step to the CIVET recipes. An error crept into one of the input files from awhile back that I missed.