Breaking change in v4.48.0 and Python 3.9

[davidmezzetti]

System Info

Python 3.9 and Transformers v4.48.0

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

v4.48.0 introduced a breaking change that happens on import of TextIteratorStreamer

Steps to reproduce

1. Install Python 3.9
2. Install Transformers >= v4.48.0

Run following:

from transformers import TextIteratorStreamer

Receive the following stack

Traceback (most recent call last):
  File "/tmp/test/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1817, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/tmp/test/lib/python3.9/site-packages/transformers/generation/streamers.py", line 231, in <module>
    class AsyncTextIteratorStreamer(TextStreamer):
  File "/tmp/test/lib/python3.9/site-packages/transformers/generation/streamers.py", line 285, in AsyncTextIteratorStreamer
    self, tokenizer: "AutoTokenizer", skip_prompt: bool = False, timeout: float | None = None, **decode_kwargs
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/tmp/test/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1806, in __getattr__
    value = getattr(module, name)
  File "/tmp/test/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1805, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/tmp/test/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1819, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.generation.streamers because of the following error (look up to see its traceback):
unsupported operand type(s) for |: 'type' and 'NoneType'

Expected behavior

Work in Python 3.9

[Rocketknight1]

The cause here is the use of X | None types without a from __future__ import annotations line at the top of the file. Opening a PR now

[davidmezzetti]

Thank you @Rocketknight1! I really appreciate it!

[Rocketknight1]

@davidmezzetti the fix should be merged now, but you'll need to install from main or wait until the next release to use it!

[davidmezzetti]

@Rocketknight1 @gante Thank you for the updates.

Do you plan to release a 4.48.1 release or is this waiting until 4.49? As you can see here, there are a couple other projects in the same situation.

While Python 3.9 isn't as common and there is a workaround, the bug is pretty bad in that simply importing the module raises the error.

[davidmezzetti]

Pinging @ArthurZucker too.

It looks like there is another 3.9 compatibility related bug: #35843

[ArthurZucker]

Yes, we'll make sure it is included in the next patch before this weekend!

[davidmezzetti]

@ArthurZucker I appreciate it, thank you!

[ArthurZucker]

Oups slipped my mind on friday, sorry! ETA is today / tomorrow!

[davidmezzetti]

Thank you for following up!