Adding an example targeting pure fuzzing purposes ?

[debrouxl]

I recently fuzzed the old libdumb from SF using afl-fuzz and the following simple client:

#include "dumb.h"
#include <stdio.h>

int main(int argc, char * argv[])
{
    DUH * duh;
    dumb_register_stdfiles();

    duh = dumb_load_it(argv[1]);
    if (!duh) {
        duh = dumb_load_xm(argv[1]);
        if (!duh) {
            duh = dumb_load_s3m(argv[1]);
            if (!duh) {
                duh = dumb_load_mod(argv[1]);
                if (!duh) {
                    fprintf(stderr, "Failed to load %s!\n", argv[1]);
                    return 1;
                }
            }
        }
    }

    unload_duh(duh);
    return 0;
}

I haven't rebuilt this strain of libdumb with afl instrumentation yet, or adjusted the above example to use the function which attempts to load any file, to see whether this strain of libdumb fares better than the original one. But the point of this issue is, maybe such a trivial client with zero sound setup and output capability (they only take time for a fuzzing workload aimed at exercising the file loading code) belongs upstream ?

[kode54]

There is now a dumb_load_any / dumb_read_any that attempts to use a little detection logic and supports all identifiable formats.

[debrouxl]

Yup, as I wrote, I saw the "any" functions, and they're a major improvement over the original libdumb :)
In order to reduce the fuzzing cost while improving fuzzer coverage, I'll have to modify the code of dumb_read_any_quick() to get rid of strcmp() / memcmp()... or probably better, use a dictionary.
dumbfile_open_memory() was already there in the original libdumb, it's exactly the primitive a libfuzzer-based client needs (the signedness of the size argument to dumbfile_open_memory() isn't an issue, given that overly large inputs are not interesting).

FTR, this libdumb version already contains a fix for a frequent division by zero crash found by afl-fuzz in the original libdumb in mere seconds.

With at least one fuzzing client in upstream source code, libdumb could even be integrated into e.g. OSS-Fuzz :)

[kode54]

Thanks for the test material, I look forward to a pull request with any useful changes. You could also check out my dumb test tool on gitlab.kode54.net/kode54/dumbtest, where I crafted a lazy macOS project that iterates over argv[1] using opendir and open (not quick, so length probe if successful, then release) each file. Goes over most of your test vectors in a minute or less per directory, though most delays are due to length probing files that don’t outright fail, and because it’s running a debug build under the debugger. I also configure most memory aids, like Mallon scribbling and edge protection. I even had to make use of both leading and trailing edge protect, which are apparently mutually exclusive and configured by an environment variable. Oh well, keep up the good work. :)

[debrouxl]

In addition to the recursive directory enumeration mode, which is indeed great for sifting through a large set of testcases, I'd suggest making it possible to load a single file without rendering it - basically, calling the sole module_test() - because that's what afl-fuzz, honggfuzz and friends need the most :)
Double bonus points for adding a quick open mode, in order to be able to exercise the file loading code paths more independently from the length probing code.
The CLI might look like
load <directory|file>
run <directory|file>
play

Also, the indentation in dumbtest's main() looks funny ?

BTW, here's the input dictionary, based on the contents of dumb_read_any_quick(), which I fed into at least one of the afl-fuzz instances:
IT="IMPM"
XM="Extended Module: "
S3M="SCRM"
STM1="!Scream!"
STM2="BMOD2STM"
STM3="WUZAMOD!"
6691="if"
6692="JN"
PTM="PTMF"
PSM="PSM"
MTM="MTM"
RIFF="RIFF"
AMF1="ASYLUM Music Format"
AMF2=" V1.0"
AMF3="AMF"
OKT="OKTASONG"

[kode54]

Do note that I managed to find some crashers from a lot of these songs actually loading completely, but causing the player/parser to crash due to some invalid values that hadn't been noticed yet.

E: Fixed the tab-vs-spaces indentation, and added load/run/play modes, all of which can take single files.