Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ptpython on Windows 11 in asyncio mode does not work #582

Open
sockduct opened this issue Jun 7, 2024 · 4 comments
Open

ptpython on Windows 11 in asyncio mode does not work #582

sockduct opened this issue Jun 7, 2024 · 4 comments

Comments

@sockduct
Copy link

sockduct commented Jun 7, 2024

I have Windows 11 and Python 3.12.3 and wish to experiment with the asyncio REPL. It works fine with the built-in REPL:

PS C:\> python -m asyncio
asyncio REPL 3.12.3 (tags/v3.12.3:f6650f9, Apr  9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)] on win32
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> await asyncio.sleep(1, 'Done!')
'Done!'
>>> exit()

However, I cannot get it to work with ptpython:

PS C:\> ptpython --asyncio

Starting ptpython asyncio REPL
Use "await" directly instead of "asyncio.run()".
In [1]: await asyncio.sleep(1, 'Done!')
Traceback (most recent call last):
  File "C:\Users\james\AppData\Roaming\Python\Python312\site-packages\ptpython\repl.py", line 183, in run_and_show_expression_async
    loop.add_signal_handler(signal.SIGINT, lambda *_: task.cancel())
  File "C:\Program Files\Python312\Lib\asyncio\events.py", line 582, in add_signal_handler
    raise NotImplementedError
NotImplementedError

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\james\AppData\Roaming\Python\Python312\Scripts\ptpython.exe\__main__.py", line 7, in <module>

  File "C:\Users\james\AppData\Roaming\Python\Python312\site-packages\ptpython\entry_points\run_ptpython.py", line 231, in run

    asyncio.run(embed_result)
  File "C:\Program Files\Python312\Lib\asyncio\runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete

    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\james\AppData\Roaming\Python\Python312\site-packages\ptpython\repl.py", line 528, in coroutine

    await repl.run_async()
  File "C:\Users\james\AppData\Roaming\Python\Python312\site-packages\ptpython\repl.py", line 252, in run_async

    await self.run_and_show_expression_async(text)
  File "C:\Users\james\AppData\Roaming\Python\Python312\site-packages\ptpython\repl.py", line 206, in run_and_show_expression_async

    loop.remove_signal_handler(signal.SIGINT)
  File "C:\Program Files\Python312\Lib\asyncio\events.py", line 585, in remove_signal_handler

    raise NotImplementedError
NotImplementedError
Task exception was never retrieved
future: <Task finished name='Task-314' coro=<PythonRepl.run_and_show_expression_async.<locals>.eval() done, defined at C:\Users\james\AppData\Roaming\Python\Python312\site-packages\ptpython\repl.py:172> exception=NameError("name 'asyncio' is not defined")>


Traceback (most recent call last):
  File "C:\Users\james\AppData\Roaming\Python\Python312\site-packages\ptpython\repl.py", line 175, in eval

    return await self.eval_async(text)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\james\AppData\Roaming\Python\Python312\site-packages\ptpython\repl.py", line 329, in eval_async

    result = await result
             ^^^^^^^^^^^^
  File "<stdin>", line 1, in <module>
NameError: name 'asyncio' is not defined. Did you forget to import 'asyncio'

Note it also doesn't work with Python 3.8.10:

PS C:\> python -m asyncio
asyncio REPL 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:34:34) [MSC v.1928 32 bit (Intel)] on win32
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> await asyncio.sleep(1, 'Done!')
'Done!'
>>> exit()

C:\> ptpython --asyncio
Starting ptpython asyncio REPL
Use "await" directly instead of "asyncio.run()".
In [1]: await asyncio.sleep(1, 'Done!')
Traceback (most recent call last):
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\site-packages\ptpython\repl.py", line 183, in run_
and_show_expression_async
    loop.add_signal_handler(signal.SIGINT, lambda *_: task.cancel())
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\asyncio\events.py", line 536, in add_signal_handler
    raise NotImplementedError
NotImplementedError

Traceback (most recent call last):
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\users\james\appdata\local\Programs\Python\Python38-32\Scripts\ptpython.exe\__main__.py", line 7, in <module>
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\site-packages\ptpython\entry_points\run_ptpython.py", line 231, in run
    asyncio.run(embed_result)
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\asyncio\base_events.py", line 616, in run_until_complete
    return future.result()
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\site-packages\ptpython\repl.py", line 528, in coroutine
    await repl.run_async()
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\site-packages\ptpython\repl.py", line 252, in run_async
    await self.run_and_show_expression_async(text)
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\site-packages\ptpython\repl.py", line 206, in run_and_show_expression_async
    loop.remove_signal_handler(signal.SIGINT)
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\asyncio\events.py", line 539, in remove_signal_handler
    raise NotImplementedError
NotImplementedError
Task exception was never retrieved
future: <Task finished name='Task-61' coro=<PythonRepl.run_and_show_expression_async.<locals>.eval() done, defined at c:\users\james\appdata\local\programs\python\python38-32\lib\site-packages\ptpython\repl.py:172> exception=NameError("name 'asyncio' is not defined")>
Traceback (most recent call last):
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\site-packages\ptpython\repl.py", line 175, in eval
    return await self.eval_async(text)
  File "c:\users\james\appdata\local\programs\python\python38-32\lib\site-packages\ptpython\repl.py", line 329, in eval_async
    result = await result
  File "<stdin>", line 1, in <module>
NameError: name 'asyncio' is not defined

Is it possible to get this working on Windows? I see on Stack Overflow, in some cases the event loop can be changed to support various use cases. Is there a configurable option to get this to work?

@jupiterbjy
Copy link

The other loop you probably meant is set by this:

import asyncio
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

But adding these at config.py doesn't make this work, nor that use of WindowsSelectorEventLoop is recommended neither, as it limit the functionality of asyncio.

@sockduct
Copy link
Author

OK - thanks for the feedback. Any suggestions on approaching debugging this to see what's missing? Maybe I could at least document next steps???

@jupiterbjy
Copy link

jupiterbjy commented Jun 12, 2024

I tried to figure out the internals but no luck so far, I'd need more time to look thru their source code, wish I had more time!

So far it seems to be already using asyncio internally:

# run_ptpython.py

def run() -> None:
    a = create_parser().parse_args()
    ...
    # When a file has been given, run that, otherwise start the shell.
    if a.args and not a.interactive:
        ...
    # Run interactive shell.
    else:
        enable_deprecation_warnings()

        # Apply config file
        def configure(repl: PythonRepl) -> None:
            ...

        embed_result = embed(  # type: ignore
            vi_mode=a.vi,
            history_filename=history_file,
            configure=configure,
            locals=__main__.__dict__,
            globals=__main__.__dict__,
            startup_paths=startup_paths,
            title="Python REPL (ptpython)",
            return_asyncio_coroutine=a.asyncio,
        )

        if a.asyncio:
            print("Starting ptpython asyncio REPL")
            print('Use "await" directly instead of "asyncio.run()".')
            asyncio.run(embed_result)

Yet can't debug further around line asyncio.run(), I'd try Pycharm later when I have time. I can only use vscode in company and it's so annoying to debug thru the builtin modules.

But so far feels like this section seems problematic:

    async def run_and_show_expression_async(self, text: str) -> Any:
        loop = asyncio.get_running_loop()
        system_exit: SystemExit | None = None

        try:
            try:
                # Create `eval` task. Ensure that control-c will cancel this
                # task.
                async def eval() -> Any:
                    nonlocal system_exit
                    try:
                        return await self.eval_async(text)
                    except SystemExit as e:
                        # Don't propagate SystemExit in `create_task()`. That
                        # will kill the event loop. We want to handle it
                        # gracefully.
                        system_exit = e

                task = asyncio.create_task(eval())
                loop.add_signal_handler(signal.SIGINT, lambda *_: task.cancel())
                # ^^^^^ here!
         ...

...as adding singal handler won't work on windows, potential hindsight of ptpython devs.

import signal
import asyncio


async def main():
    loop = asyncio.get_event_loop()
    loop.add_signal_handler(signal.SIGINT, lambda: print("SIGINT"))

    await asyncio.sleep(10)


asyncio.run(main())
Exception has occurred: NotImplementedError
exception: no description
  File "...\asyncio_task_count.py", line 7, in main
    loop.add_signal_handler(signal.SIGINT, lambda: print("SIGINT"))
  File ...\asyncio_task_count.py", line 12, in <module>
    asyncio.run(main())
NotImplementedError: 

@jupiterbjy
Copy link

Yeah I think speculation was right, so this section tries to handle ctrl+c but since it's not implemented in ProactorEventLoop it's failing. Windows triggers on KeyboardInterrupt instead as there's no signal.

(Line 183, 206)

ptpython/ptpython/repl.py

Lines 164 to 206 in fb9bed1

async def run_and_show_expression_async(self, text: str) -> Any:
loop = asyncio.get_running_loop()
system_exit: SystemExit | None = None
try:
try:
# Create `eval` task. Ensure that control-c will cancel this
# task.
async def eval() -> Any:
nonlocal system_exit
try:
return await self.eval_async(text)
except SystemExit as e:
# Don't propagate SystemExit in `create_task()`. That
# will kill the event loop. We want to handle it
# gracefully.
system_exit = e
task = asyncio.create_task(eval())
loop.add_signal_handler(signal.SIGINT, lambda *_: task.cancel())
result = await task
if system_exit is not None:
raise system_exit
except KeyboardInterrupt:
# KeyboardInterrupt doesn't inherit from Exception.
raise
except SystemExit:
raise
except BaseException as e:
self._handle_exception(e)
else:
# Print.
if result is not None:
await loop.run_in_executor(None, lambda: self._show_result(result))
# Loop.
self.current_statement_index += 1
self.signatures = []
# Return the result for future consumers.
return result
finally:
loop.remove_signal_handler(signal.SIGINT)

However as linked on TL;DR section of this SO answer it's almost impossibile to catch KeyboardInterrupt without singal handler by asyncio.Task side.

Looking at how asyncio/__main__.py handles the same work flawlessly - depending on structure of ptpython this might be plain impossible without overhaul, will need to look into it.

TL;DR: asyncio's REPL runs loop from non-async context, thus can resume loop after handling. Input handling is done by other thread, so this can be done.

https://github.com/python/cpython/blob/030b452e34bbb0096acacb70a31915b9590c8186/Lib/asyncio/__main__.py#L78-L137

https://github.com/python/cpython/blob/030b452e34bbb0096acacb70a31915b9590c8186/Lib/asyncio/__main__.py#L140-L195

https://github.com/python/cpython/blob/030b452e34bbb0096acacb70a31915b9590c8186/Lib/_pyrepl/simple_interact.py#L93-L171

But in ptpython seems like input handling is tied within async context. Would put some guard here and test if it help

if a.asyncio:
print("Starting ptpython asyncio REPL")
print('Use "await" directly instead of "asyncio.run()".')
asyncio.run(embed_result)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants