pytest hangs on waiter.acquire() - (FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME)

[86Ilya]

• Pytest hangs forever on

Traceback (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312, in wait
    waiter.acquire()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 435, in result
    self._condition.wait(timeout)

• pip list

-i https://pypi.org/simple
anyio==3.6.2
attrs==22.2.0
certifi==2022.12.7
dependency-injector==4.41.0
exceptiongroup==1.1.0
fastapi==0.89.1
greenlet==2.0.1
h11==0.14.0
httpcore==0.16.3
httpx==0.23.3
idna==3.4
iniconfig==2.0.0
packaging==23.0
pluggy==1.0.0
pydantic==1.10.4
pytest==7.2.1
rfc3986[idna2008]==1.5.0
six==1.16.0
sniffio==1.3.0
sqlalchemy==1.4.46
starlette==0.22.0
tomli==2.0.1
typing-extensions==4.4.0

• pytest version is 7.2.1, Linux r2d2 5.10.0-12-amd64 #1 SMP Debian 5.10.103-1 (2022-03-07) x86_64 GNU/Linux

Example with dockerfile

docker build -t check-pytest .
docker run -it check-pytest

I'm not sure but it's looks like python wait for coroutine ending in wrong event loop.

[The-Compiler]

Here is the full trace from -o faulthandler_timeout=5:

Thread 0x00007fa889bdf700 (most recent call first):
  File "/usr/local/lib/python3.9/selectors.py", line 469 in select
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1869 in _run_once
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 601 in run_forever
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 634 in run_until_complete
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 44 in run
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 292 in run
  File "/usr/local/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 70 in run
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58 in run
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 83 in _worker
  File "/usr/local/lib/python3.9/threading.py", line 917 in run
  File "/usr/local/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/usr/local/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007fa88d8d7740 (most recent call first):
  File "/usr/local/lib/python3.9/threading.py", line 312 in wait
  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 441 in result
  File "/usr/local/lib/python3.9/site-packages/anyio/from_thread.py", line 283 in call
  File "/usr/local/lib/python3.9/site-packages/starlette/testclient.py", line 334 in handle_request
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1009 in _send_single_request
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 973 in _send_handling_redirects
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 936 in _send_handling_auth
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 908 in send
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 821 in request
  File "/usr/local/lib/python3.9/site-packages/starlette/testclient.py", line 454 in request
  File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1136 in post
  File "/usr/local/lib/python3.9/site-packages/starlette/testclient.py", line 579 in post
  File "/app/tests/test_views.py", line 12 in test_credentials_is_correct
  File "/usr/local/lib/python3.9/site-packages/_pytest/python.py", line 195 in pytest_pyfunc_call
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/local/lib/python3.9/site-packages/_pytest/python.py", line 1789 in runtest
  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 167 in pytest_runtest_call
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 260 in <lambda>
  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 339 in from_call
  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 259 in call_runtest_hook
  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 220 in call_and_report
  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 131 in runtestprotocol
  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 112 in pytest_runtest_protocol
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 324 in _main
  File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 270 in wrap_session
  File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py", line 167 in main
  File "/usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py", line 190 in console_main
  File "/usr/local/lib/python3.9/site-packages/pytest/__main__.py", line 5 in <module>
  File "/usr/local/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/local/lib/python3.9/runpy.py", line 197 in _run_module_as_main

pytest itself doesn't deal with threading or asyncio, and as you can see, the pytest main thread hangs while calling into starlette.testclient / httpx.client / anyio - so this doesn't seem like a pytest issue. I'm converting this into a Q&A discussion, maybe someone has an idea what's going on.

[86Ilya]

I've tried to use another http client - same result.

[zacps]

I can't speak to what's happening, but I also ran into this and found that it worked if we removed:

nest_asyncio.apply()

from our main fastapi file.

[mihirsamdarshi]

I'm running into this in two of my tests (and only tests specifically).

I did have nest_asyncio.apply() in one of my tests, and removing that helped for that test, but the other one is still stuck for some reason.