Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Firefox E2E CI gate #2363

Open
vytas7 opened this issue Oct 9, 2024 · 8 comments
Open

Fix Firefox E2E CI gate #2363

vytas7 opened this issue Oct 9, 2024 · 8 comments
Labels
good first issue Comment on this issue if you'd like to volunteer to work on this. Thanks! maintenance needs contributor Comment on this issue if you'd like to volunteer to work on this. Thanks!
Milestone

Comments

@vytas7
Copy link
Member

vytas7 commented Oct 9, 2024

It seems that something has got broken recently, so we had to disable these E2E tests on Firefox.
Was it Firefox 131.0, or anything in SeleniumBase? Or simply an issue with a new default GitHub Actions Ubuntu image?

You can find an example of a failed CI run here: https://github.com/falconry/falcon/actions/runs/11254279330/job/31291767229

__________________________________ test_chat ___________________________________

browser = <seleniumbase.plugins.pytest_plugin.sb.<locals>.BaseClass testMethod=base_method>
clear_log = <function clear_log.<locals>._impl at 0x7f7f0a27bc40>

    def test_chat(browser, clear_log):
        browser.slow_click('#button1')
>       browser.assert_text('Hello, WS1!', 'div.ws1', timeout=5)

e2e-tests/test_e2e.py:41: 

<...>

exception = <class 'selenium.common.exceptions.NoSuchElementException'>
message = 'Element {div.ws1} was not present after 5 seconds!'

    def timeout_exception(exception, message):
        exc, msg = shared_utils.format_exc(exception, message)
>       raise exc(msg)
E       seleniumbase.common.exceptions.NoSuchElementException: Message: 
E        Element {div.ws1} was not present after 5 seconds!
@vytas7 vytas7 added good first issue Comment on this issue if you'd like to volunteer to work on this. Thanks! needs contributor Comment on this issue if you'd like to volunteer to work on this. Thanks! maintenance labels Oct 9, 2024
@vytas7 vytas7 added this to the Version 4.x milestone Nov 11, 2024
@Cycloctane
Copy link

It seems that uvicorn subprocess somehow crashes with SIGSEGV when setting up firefox for second test... It only happens on github action vm with python3.12+. Cannot figured out the root cause, for it is super hard to debug in action vm.

[ 100.362871] traps: uvicorn[2029] general protection fault ip:7fdc603476f5 sp:7ffd6f6f7d70 error:0 in libpython3.12.so.1.0[7fdc60306000+234000]

Possible workaround may be using python3.11 for e2e test or disabling session scope setting of base_url fixture to setup new uvicorn process for every test.

@CaselIT
Copy link
Member

CaselIT commented Jan 4, 2025

I'm having issues on another project using ubuntu 24.04 pythons, that fails some test for unknown reasons, while the same test and python version have worked for years without any issue in older version of ubuntu.

All this is to suggest that maybe pinning ubuntu to 22.04 fixes this issue too?

@vytas7
Copy link
Member Author

vytas7 commented Jan 4, 2025

Thanks for looking into this @Cycloctane, that's very valuable feedback 💯

Maybe we should report this to Uvicorn and/or Cython too? It shouldn't be segfaulting like that... 🤔

@Cycloctane out of curiosity, could you try installing Uvicorn with pure Python deps only, i.e., uvicorn, wsproto instead of the current uvicorn[standard] in the E2E requirements?

@vytas7
Copy link
Member Author

vytas7 commented Jan 4, 2025

Note that in the original failure that I reported though, Uvicorn did not segfault. (Different CPython and package versions were in use.)

I'll check if there are any issues or discussions on Uvicorn's side, maybe @Kludex knows more.

@Cycloctane
Copy link

Note that in the original failure that I reported though, Uvicorn did not segfault. (Different CPython and package versions were in use.)

I'll check if there are any issues or discussions on Uvicorn's side, maybe @Kludex knows more.

Maybe the original was actually segfault failing, but it does not show details by default.

ci log with faulthandler enabled in uvicorn subprocess: https://github.com/Cycloctane/falcon/actions/runs/12612463548/job/35149212673

@vytas7
Copy link
Member Author

vytas7 commented Jan 4, 2025

Hm, you may be right @Cycloctane, we might have missed that. OTOH, we do assert the retval to fall within the expected values, so we should have captured that, even if we missed the traceback 🤔

@Cycloctane
Copy link

Hm, you may be right @Cycloctane, we might have missed that. OTOH, we do assert the retval to fall within the expected values, so we should have captured that, even if we missed the traceback 🤔

I cannot see what original conftest does to check the retval of subprocess. It is not able to detect if uvicorn crashed during the test. I added if uvicorn.poll() ... statement to handle this situation in my fork.

@vytas7
Copy link
Member Author

vytas7 commented Jan 4, 2025

Yes, sorry, you're dead right. I had our main ASGI server tests in mind (tests/asgi/test_asgi_servers.py), apparently we had a somewhat simpler fixture in the E2E tests.

Edit: and thanks once again for tracing this to be a segfault in Uvicorn. For reference, I created a discussion on Uvicorn's side (encode/uvicorn#2557), referencing your CI runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Comment on this issue if you'd like to volunteer to work on this. Thanks! maintenance needs contributor Comment on this issue if you'd like to volunteer to work on this. Thanks!
Projects
None yet
Development

No branches or pull requests

3 participants