ci: add freebsd workflow

[joshuasing]

Add a GitHub Actions workflow to run make check inside a FreeBSD VM, which runs daily at 00:00 or when manually requested.
This is similar to the Solaris VM workflow, and uses an action from the same vmactions org.

Successful CI run:
https://github.com/joshuasing/libressl-portable/actions/runs/11341234538/job/31539126298

Should this also run gmake test or similar?

[botovq]

Should this also run gmake test or similar?

gmake -j2 check is already part of the script, so no need for this.

What's missing is the cmake stanza. Something like this

$ cmake -G Ninja -B build
$ ninja -C build
$ ninja -C build test

[botovq]

Looks good to me now. Do you have a successful run somewhere given that it doesn't run in the per-PR CI?

[joshuasing]

@botovq Here's a run (cmake has test failures, which are probably fixed by #1108): https://github.com/joshuasing/libressl-portable/actions/runs/11341798906

Failure: https://github.com/joshuasing/libressl-portable/actions/runs/11341798906/job/31540817927#step:4:18520

Would it be worth setting --output-on-failure or similar to log more information when a test fails?
I was also going to pin the FreeBSD version and disable "copyback" since we don't need the action to copy files back from the VM, is that okay or should I leave it as-is?

[botovq]

The process '/usr/bin/bash' failed with exit code 1

Where does that come from? Bash isn't part of FreeBSD. You'd need to pkg install it and use /usr/local/bin/bash if it's really needed. Better use a posix shell and invoke it as /bin/sh.

Would it be worth setting --output-on-failure or similar to log more information when a test fails?

Yeah, that would make sense. Not sure what the right incantation is, though.

I was also going to pin the FreeBSD version and disable "copyback" since we don't need the action to copy files back from the VM, is that okay or should I leave it as-is?

Agreed, I don't think we need that.

[botovq]

  	 62 - err_test (SEGFAULT)
  	102 - signertest (Failed)
  	117 - tlstest (Failed)
  	122 - verifytest (Failed)

The three plain failures are what I saw. Should be fixed by rebasing your branch on top of HEAD. I think I know what the issue with the err_test is, will look.

[botovq]

I can't repro the segfault. I'll need to instrument this CI VM to figure it out. Will take a while.

Ah. I made a mistake. The underlying issue is the same as with the failing libtls tests. The initialization code in crypto/err/err.c doesn't work because of FreeBSD's questionable idea of providing a completely broken pthread_once() in libc.

That said, I think the err_test should be a bit more defensive.

So @joshuasing should I merge or do you want to make the changes you mentioned?

[joshuasing]

@botovq I will make the changes I mentioned and will rebase on master.

Where does that come from? Bash isn't part of FreeBSD. You'd need to pkg install it and use /usr/local/bin/bash if it's really needed. Better use a posix shell and invoke it as /bin/sh.

I am not sure. Where are you seeing that? 🤔
I didn't see it exit with either of those

[botovq]

I think I simply got confused about that bash thing. It's in the annotations "panel" when I click on https://github.com/joshuasing/libressl-portable/actions/runs/11341798906

[joshuasing]

Oh, I see. I believe that comes from the action running bash to start the VM 🤔

[botovq]

Agreed.

[joshuasing]

I have pinned the version, set copyback: false and added an environment variable which should effectively set --output-on-failure when ctest runs.

Run: https://github.com/joshuasing/libressl-portable/actions/runs/11343332671

[joshuasing]

Oops, forgot to rebase. I have now rebased, here is a run with the latest commits on master: https://github.com/joshuasing/libressl-portable/actions/runs/11343486719

[botovq]

Thanks. I'll merge if green.