ModuleNotFoundError: No module named 'distutils._modified'

[vhiremath]

setuptools version

setuptools (>=65)

Python version

Python 3.11

OS

Linux

Additional environment information

No response

Description

I see this error at the time of deploying to AWS lambda function.
This was working in my previous deploys. and suddenly started failing.
I have not added any new packages or deps in my pyproject.toml
I am using zappa for aws deploying

How to Reproduce

I am unable to repro this with local venv creation.
See it only when attempting to deploy via zappa
Unfortunately I dont have a trimmed repo ^^;

Other detail

Traceback (most recent call last):
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 3048, in handle
    sys.exit(cli.handle())
             ^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 521, in handle
    self.dispatch_command(self.command, stage)
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 574, in dispatch_command
    self.update(
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 980, in update
    self.create_package()
  File "/code/.venv/lib/python3.11/site-packages/zappa/cli.py", line 2453, in create_package
    self.zip_path = self.zappa.create_lambda_zip(  # type: ignore[attr-defined]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/zappa/core.py", line 690, in create_lambda_zip
    copy_tree(temp_package_path, temp_project_path, update=True)
==============
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/dir_util.py", line 155, in copy_tree
    return list(itertools.chain.from_iterable(map(copy_one, names)))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/dir_util.py", line 186, in _copy_one
    yield from copy_tree(
               ^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/dir_util.py", line 155, in copy_tree
    return list(itertools.chain.from_iterable(map(copy_one, names)))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/dir_util.py", line 197, in _copy_one
    file_util.copy_file(
  File "/code/.venv/lib/python3.11/site-packages/setuptools/_distutils/file_util.py", line 104, in copy_file
    from distutils._modified import newer
ModuleNotFoundError: No module named 'distutils._modified'

[abravalheri]

I am unable to repro this with local venv creation.
See it only when attempting to deploy via zappa
Unfortunately I dont have a trimmed repo ^^;

Hi @vhiremath, ideally we need a simplified reproducer to be provided before we start to investigate this problem. I am afraid it is very complicated to find out what is happening without one.

On top of that could you please also provide:

• What is the value for the SETUPTOOLS_USE_DISTUTILS environment variable, as seen by the Python runtime? (please note that setting this env var with the "stdlib" value is deprecated, therefore I recommend not setting this env var at all).
• What is the value of sys.path and sys.meta_path during runtime for your program?

Finally, could you please try to run your project using the latest version available of setuptools and without setting any SETUPTOOLS_* env var, to see if this particular problem is already solved in the latest version?

[abravalheri]

Another thing that may be related: before importing distutils, zappa must guarantee that setuptools is imported, as advised by this runtime warning: https://github.com/pypa/setuptools/blob/v75.1.0/_distutils_hack/__init__.py#L16-L23

[vhiremath]

This is fixed on zappa repo with issue zappa/Zappa#1349
I think we can close this issue.