rename possibly identical modules to prevent test collisions

[mattip]

Fixes #134 by renaming the test module, if not already specifically named. I needed two randint invocations to reduce the chances of collision.

[arigo]

I don't know what to do with the failures shown here on macosx. I'm not sure if they are relevant, and if not, I'm not sure how to say "merge anyway please".

[arigo]

Also, not sure why you use randint(32000); I would replace that with randint(1000000000) and still call it twice. It costs nothing and reduces the risks more.

[mattip]

OK. I can reproduce the failures locally. That test seems specifically designed to reuse the module, note how the lib2 code is exactly the same as the original one.

[arigo]

Still fails. Maybe differently?

[mattip]

All of test_verify.py, test_vgen.py, and test_vgen2.py call cffi.verifier.cleanup_tmpdir() at module setup. If another thread/process is running these in parallel, the cleanup may happen when another test is running. But that is a second-order problem, it should only cause slower test runs 🤞

[arigo]

Still failing...

[mattip]

CI is passing. I think randomizing module names now makes the test_vgen2.py test irrelevant. Thoughts?

[arigo]

Yes, that's bad. I don't remember any failure caught specifically by test_vgen2, but its point was precisely to reuse the modules.

[arigo]

This dooms the whole approach, I'm afraid. We need either to say pytest-xtest is not supported, or decide test_vgen2 is not very useful and kill it, or maybe find a proper refactoring.

[mattip]

Is it important that all the test_vgen tests be rerun without *.c sources, or can I add a specific test for a single case of removing the *.c source and running verify again, and actually check that the source is not regenerated? As far as I can tell, test_vgen2 does not actually check that the source is unneeded, so I am not sure what it is designed to check.

[arigo]

I may originally have decided that I would notice if test_verify2 took as long to execute as test_verify instead of being very fast, just because I was running the tests locally.

If you want to kill the test_v*2.py files and replace them with a single test that checks no C code is regenerated, then I'm fine with it.

[mattip]

The latest commit

• removes the test_v*2.py files and adds a test_no_regen that checks the C file is not regenerated
• moves the global verifier._TMPDIR and verifier._FORCE_GENERIC_ENGINE into a threading.local() to make the global state more thread-safe
• refactors compilation to lessen the likelyhood of creating a filename that is longer than MAXPATH on windows.

[mattip]

The latest commit

• uses filelock to make test_parse_c_type build lib only once
• restores verifier.sourcefilename after shortening it for compilation
• documents the need for filelock
• runs tests with pytest-xdist -n 4 and add back windows testing

[mattip]

Adding back windows testing almost worked, I had to skip embedded tests. I will open an issue so if someone is so inclined they can fix it. The tests are failing on the PyPy windows buildbot as well.

[mattip]

These days pip install is to be preferred

[mattip]

CI is passing.

[arigo]

I can approve this without looking in details as it mostly touches tests. Can you confirm the underlying reason, though: why do we want to run pytest-xdist? The negative points are that it creates issues and adds extra installation dependencies; what is the positive point?

[mattip]

On windows, tests now run in CI in ~11 minutes, where before they were 45 minutes.

[arigo]

It would be very nice, if cffi was actively being developed. Bu as it is not, my point stands..?

[mattip]

these tests are run by pypy (in its extra tests) for every nightly buildbot run. The windows runs are very slow. So these changes are more for the PyPy test than for cffi itself, trying to minimize the diff between the two copies. But I see your point and will close this.

[arigo]

Oh, if it makes the nightly test runs of pypy more than 30 minutes faster, that's an interesting outcome.

[arigo]

Should I review this? Or wait for #137 (which apparently needs some fixing)?