[BUG] `AttributeError: 'PathDistribution' object has no attribute '_normalized_name'` with setuptools 60.9.0+

See original GitHub issue

setuptools version

setuptools==60.9.0

Python version

Python 3.10

OS

Ubuntu 20.04

Additional environment information

Also affects latest setuptools (62.2.0) + Python 3.8 and 3.9.

Description

With setuptools 60.9.0+ this requirements file (reduced from a user-provided report), installs successfully initially, however on subsequent installs (ie: when site-packages is already populated), results in an error. With setuptools 60.8.2 and older, this error did not occur.

importlib-metadata==2.0.0
git+https://github.com/izdi/django-slack-oauth.git@1.5.2
jira==2.0.0

Updating to newer importlib-metadata or downgrading setuptools resolves the issue.

Expected behavior

Either the existing requirements file continues to work, or else a clearer error message is given about what needs to be changed to fix it.

For example, by outputting an error message mentioning that the importlib-metadata version should be updated.

How to Reproduce

docker run --rm -it python:3.10.5 bash -c 'pip install -q setuptools==60.9.0 && pip install -q importlib-metadata==2.0.0 git+https://github.com/izdi/django-slack-oauth.git@1.5.2 jira==2.0.0 && pip install importlib-metadata==2.0.0 git+https://github.com/izdi/django-slack-oauth.git@1.5.2 jira==2.0.0'

Output

$ docker run --rm -it python:3.10.5 bash -c 'pip install -q setuptools==60.9.0 && pip install importlib-metadata==2.0.0 git+https://github.com/izdi/django-slack-oauth.git@1.5.2 jira==2.0.0 && pip install importlib-metadata==2.0.0 git+https://github.com/izdi/django-slack-oauth.git@1.5.2 jira==2.0.0'
...
Collecting git+https://github.com/izdi/django-slack-oauth.git@1.5.2
  Cloning https://github.com/izdi/django-slack-oauth.git (to revision 1.5.2) to /tmp/pip-req-build-yzzrjnd1
  Running command git clone --filter=blob:none --quiet https://github.com/izdi/django-slack-oauth.git /tmp/pip-req-build-yzzrjnd1
  Resolved https://github.com/izdi/django-slack-oauth.git to commit 0d72013198de94ca38e8d5ccbe7dd11dd484fc64
  Preparing metadata (setup.py) ... done
Collecting importlib-metadata==2.0.0
  Downloading importlib_metadata-2.0.0-py2.py3-none-any.whl (31 kB)
Collecting jira==2.0.0
  Downloading jira-2.0.0-py2.py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.6/57.6 KB 28.4 MB/s eta 0:00:00
Collecting zipp>=0.5
  Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB)
Collecting six>=1.10.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting requests-oauthlib>=0.6.1
  Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting requests-toolbelt
  Downloading requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.3/54.3 KB 29.1 MB/s eta 0:00:00
Requirement already satisfied: setuptools>=20.10.1 in /usr/local/lib/python3.10/site-packages (from jira==2.0.0) (60.9.0)
Collecting pbr>=3.0.0
  Downloading pbr-5.9.0-py2.py3-none-any.whl (112 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 112.3/112.3 KB 65.1 MB/s eta 0:00:00
Collecting requests>=2.10.0
  Downloading requests-2.28.1-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 KB 36.8 MB/s eta 0:00:00
Collecting defusedxml
  Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting oauthlib[signedtoken]>=1.0.0
  Downloading oauthlib-3.2.0-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.5/151.5 KB 80.1 MB/s eta 0:00:00
Collecting Django>=1.8
  Downloading Django-4.0.6-py3-none-any.whl (8.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.0/8.0 MB 63.8 MB/s eta 0:00:00
Collecting jsonfield
  Downloading jsonfield-3.1.0-py3-none-any.whl (8.0 kB)
Collecting sqlparse>=0.2.2
  Downloading sqlparse-0.4.2-py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.3/42.3 KB 25.9 MB/s eta 0:00:00
Collecting asgiref<4,>=3.4.1
  Downloading asgiref-3.5.2-py3-none-any.whl (22 kB)
Collecting pyjwt<3,>=2.0.0
  Downloading PyJWT-2.4.0-py3-none-any.whl (18 kB)
Collecting cryptography>=3.0.0
  Downloading cryptography-37.0.4-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (3.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 64.7 MB/s eta 0:00:00
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.2/139.2 KB 63.5 MB/s eta 0:00:00
Collecting idna<4,>=2.5
  Downloading idna-3.3-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 KB 40.4 MB/s eta 0:00:00
Collecting charset-normalizer<3,>=2
  Downloading charset_normalizer-2.1.0-py3-none-any.whl (39 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2022.6.15-py3-none-any.whl (160 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 160.2/160.2 KB 56.0 MB/s eta 0:00:00
Collecting cffi>=1.12
  Downloading cffi-1.15.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (449 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 449.9/449.9 KB 73.9 MB/s eta 0:00:00
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 KB 50.1 MB/s eta 0:00:00
Building wheels for collected packages: django-slack-oauth
  Building wheel for django-slack-oauth (setup.py) ... done
  Created wheel for django-slack-oauth: filename=django_slack_oauth-1.5.1-py3-none-any.whl size=11142 sha256=ad24b187de24b8ce040b5abc925e76e5f53a9252a489b1fe673eebba7bbd7b09
  Stored in directory: /tmp/pip-ephem-wheel-cache-1occgfey/wheels/57/60/f4/4ef8708f018860cd3487fa19bf1b4125810444c29e98591fcb
Successfully built django-slack-oauth
Installing collected packages: zipp, urllib3, sqlparse, six, pyjwt, pycparser, pbr, oauthlib, idna, defusedxml, charset-normalizer, certifi, asgiref, requests, importlib-metadata, Django, cffi, requests-toolbelt, requests-oauthlib, jsonfield, cryptography, django-slack-oauth, jira
Successfully installed Django-4.0.6 asgiref-3.5.2 certifi-2022.6.15 cffi-1.15.1 charset-normalizer-2.1.0 cryptography-37.0.4 defusedxml-0.7.1 django-slack-oauth-1.5.1 idna-3.3 importlib-metadata-2.0.0 jira-2.0.0 jsonfield-3.1.0 oauthlib-3.2.0 pbr-5.9.0 pycparser-2.21 pyjwt-2.4.0 requests-2.28.1 requests-oauthlib-1.3.1 requests-toolbelt-0.9.1 six-1.16.0 sqlparse-0.4.2 urllib3-1.26.10 zipp-3.8.1
...
Collecting git+https://github.com/izdi/django-slack-oauth.git@1.5.2
  Cloning https://github.com/izdi/django-slack-oauth.git (to revision 1.5.2) to /tmp/pip-req-build-fcax2atz
  Running command git clone --filter=blob:none --quiet https://github.com/izdi/django-slack-oauth.git /tmp/pip-req-build-fcax2atz
  Resolved https://github.com/izdi/django-slack-oauth.git to commit 0d72013198de94ca38e8d5ccbe7dd11dd484fc64
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [47 lines of output]
      running egg_info
      writing /tmp/pip-pip-egg-info-oz9bkwiy/django_slack_oauth.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-pip-egg-info-oz9bkwiy/django_slack_oauth.egg-info/dependency_links.txt
      writing requirements to /tmp/pip-pip-egg-info-oz9bkwiy/django_slack_oauth.egg-info/requires.txt
      writing top-level names to /tmp/pip-pip-egg-info-oz9bkwiy/django_slack_oauth.egg-info/top_level.txt
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-req-build-fcax2atz/setup.py", line 16, in <module>
          setup(
        File "/usr/local/lib/python3.10/site-packages/setuptools/__init__.py", line 155, in setup
          return distutils.core.setup(**attrs)
        File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 148, in setup
          return run_commands(dist)
        File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
          dist.run_commands()
        File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/local/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 298, in run
          self.find_sources()
        File "/usr/local/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 305, in find_sources
          mm.run()
        File "/usr/local/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 540, in run
          self.add_defaults()
        File "/usr/local/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 577, in add_defaults
          sdist.add_defaults(self)
        File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 226, in add_defaults
          self._add_defaults_python()
        File "/usr/local/lib/python3.10/site-packages/setuptools/command/sdist.py", line 111, in _add_defaults_python
          build_py = self.get_finalized_command('build_py')
        File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 298, in get_finalized_command
          cmd_obj = self.distribution.get_command_obj(command, create)
        File "/usr/local/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 858, in get_command_obj
          klass = self.get_command_class(command)
        File "/usr/local/lib/python3.10/site-packages/setuptools/dist.py", line 928, in get_command_class
          eps = metadata.entry_points(group='distutils.commands', name=command)
        File "/usr/local/lib/python3.10/importlib/metadata/__init__.py", line 1009, in entry_points
          return SelectableGroups.load(eps).select(**params)
        File "/usr/local/lib/python3.10/importlib/metadata/__init__.py", line 459, in load
          ordered = sorted(eps, key=by_group)
        File "/usr/local/lib/python3.10/importlib/metadata/__init__.py", line 1006, in <genexpr>
          eps = itertools.chain.from_iterable(
        File "/usr/local/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen
          k = key(element)
      AttributeError: 'PathDistribution' object has no attribute '_normalized_name'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Issue Analytics

  • State:open
  • Created a year ago
  • Reactions:1
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
jaracocommented, Jul 14, 2022

Hi @jaraco, I think that the “workaround” in c28caa6 is also the only thing we can do right now to “address” #3319, right?

I don’t believe the workaround will help that situation at all, as it only affects Python 3.10+ and the issue in #3319 is for Python ❤️.10.

1reaction
jaracocommented, Jul 14, 2022

First, let me say a huge thanks for providing a docker-based repro. This one contrib made it possible for me to dive deep on the issue to understand it better.

Background

This issue stems from the fact that importlib_metadata gives itself precedence for supplying metadata, even on later versions of Python. See python/importlib_metadata#91 for background.

What this means is that later versions of importlib.metadata that expect for _normalized_name to be present on Distribution objects is missing.

This issue isn’t specific to Setuptools. Any calls to importlib.metadata.entry_points in that environment will fail with that same error. It doesn’t make sense for Setuptools to trap this error, as any other client, including twine or keyring or pmxbot could encounter the same error independent of Setuptools.

The reason the issue doesn’t occur older Setuptools versions is because Setuptools 60.9.0 intentionally switched to importlib metadata for loading entry points.

Investigation

What I don’t yet understand is what is causing importlib_metadata to get imported. Without being imported, the primacy of the third-party library is not relevant. On Python 3.10, Setuptools doesn’t import it. I thought maybe pip was importing it but that doesn’t appear to be the case. I looked at the setup.py for that package, but it’s not doing anything that interacts with importlib_metadata. Interestingly, the issue doesn’t replicate unless jira is included in the install, so it must be implicated.

I can replicate the issue with this simpler repro:

docker run --rm -it python:3.10.5 bash -c 'pip install -q -U setuptools && pip install -q importlib-metadata==2.0.0 jira==2 && pip install git+https://github.com/izdi/django-slack-oauth.git@1.5.2'

I tried uninstalling jira before running the last command, but that had no effect (still failed), so I looked at the dependencies of jira to determine what other factor might be triggering the failure, and found that this command succeeds:

docker run --rm -it python:3.10.5 bash -c 'pip install -q -U setuptools && pip install -q importlib-metadata==2.0.0 jira==2 && pip uninstall -y -q pbr && pip install git+https://github.com/izdi/django-slack-oauth.git@1.5.2'

So it seems the presence of pbr is affecting the build of django-slack-oauth from source, meaning a more essential repro is:

docker run --rm -it python:3.10.5 bash -c 'pip install -q -U setuptools && pip install -q importlib-metadata<4.3 pbr && pip install git+https://github.com/izdi/django-slack-oauth.git@1.5.2'

I find that surprising, because I would have expected pip to perform an isolated build such that pbr would not be a factor. However, clearly that’s not the case. Confirmed that pbr imports importlib_metadata and pbr is probably invoked as part of setuptools as it provides setuptools plugins.

Also curiously, I was unable to replicate the issue by building another project like pypa/sampleproject from source or from github. So there’s still a factor in django-slack-oauth that’s relevant.

Aha. After some more investigation, I found that django-slack-oauth is being built using a legacy process because it doesn’t declare any PEP 518 build info (pyproject.toml). I can replicate the issue with the sampleproject if I go back to the commit before pyproject.toml was introduced in sampleproject:

docker run --rm -it python:3.10 bash -c 'pip install -q -U setuptools && pip install -q "importlib-metadata<4.3" pbr && pip install git+https://github.com/pypa/sampleproject@83148c568c09179ed93ef9fccd1574549c69749f'

That also explains why pbr affects the environment: pip is probably including the site-packages for compatibility.

For Python 3.9, I had to also pin setuptools as the issue was addressed in 60.9.1.

docker run --rm -it python:3.9 bash -c 'pip install -q -U setuptools==60.9 && pip install -q "importlib-metadata<4.3" pbr && pip install git+https://github.com/pypa/sampleproject@83148c568c09179ed93ef9fccd1574549c69749f'

Conclusion

I see the _importlib shim mentions #3102, which reports a similar error message, and the workaround there was to provide a better error message to the user.

As you’ve already identified, updating the version of importlib_metadata that’s present will fix the issue.

Based on the investigation above, I believe there may be several workarounds:

  • Get packages to update to PEP 517/518 (supply pyproject.toml build system configuration).
  • Remove pbr from running environments (and get projects not to declare them as install requirements).
  • Update to importlib_metadata 4.3 or later.
  • Uninstall importlib_metadata.
  • Downgrade to Setuptools<60.9.
  • Setuptools maybe could expand its workaround in _importlib to capture this situation (where a setuptools plugin is importing importlib_metadata even when it’s not needed).
  • Force use of PEP 517 when installing.
Read more comments on GitHub >

github_iconTop Results From Across the Web

"AttributeError: 'module' object has no attribute" with installed ...
I am following this guide to package this project for PiPY. The setup.py file looks like: #!/usr/bin/python3 # coding=utf8 from setuptools ...
Read more >
`setup.py install` from VCS checkout fails: AttributeError ...
#3 `setup.py install` from VCS checkout fails: AttributeError: 'module' object has no attribute 'ChangelogAwareDistribution'.
Read more >
Build error for Python 3.7 on two different projects
Preparing metadata (setup.py) ... error error: ... _search_paths(context.name, context.path) AttributeError: 'str' object has no attribute ...
Read more >
setuptools 6.1 - PyPI
If you have Python 3.3 or later, you can use the py command to install to ... Distribute #206: AttributeError: 'HTTPMessage' object has...
Read more >
Python AttributeError — What is it and how do you fix it?
AttributeError : '***' object has no attribute '***'What is an AttributeError in Python? What can you do to fix it? When does it...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found