Version solving failed for packages from private repository with Poetry > 1.0.10

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
OS version and name: Linux
Poetry version: 1.1.5
Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/itssimon/cd2ee0d6b407cf003cad3630ccb4776d

Issue

I have two dependencies which need to be installed from private repositories (GitLab). I have configured these two repositories in the pyproject.toml and added credentials as outlined in the documentation:

[tool.poetry.dependencies]
# ...
gcds-common = {extras = ["mosaiq"], version = "^0.12.5"}
gcds-ml = {extras = ["lightgbm", "roberta"], version = "^0.4.5"}
# ...

[[tool.poetry.source]]
name = "common"
url = "https://gitlab.com/api/v4/projects/xxx1/packages/pypi/simple"
secondary = true

[[tool.poetry.source]]
name = "ml"
url = "https://gitlab.com/api/v4/projects/xxx2/packages/pypi/simple"
secondary = true

However Poetry versions > 1.0.10 are not able to resolve these dependencies. poetry update -vvv shows the following output:

PyPI: No packages found for gcds-ml >=0.4.5,<0.5.0
   1: fact: no versions of gcds-ml match >=0.4.5,<0.5.0
   1: conflict: no versions of gcds-ml match >=0.4.5,<0.5.0
   1: !  gcds-ml (^0.4.5) is satisfied by  gcds-ml (^0.4.5)
   1: ! which is caused by "gcds-clinex depends on gcds-ml (^0.4.5)"
   1: ! thus: version solving failed
   1: Version solving took 24.006 seconds.
   1: Tried 1 solutions.

...

SolverProblemError
  Because gcds-clinex depends on gcds-ml (^0.4.5) which doesn't match any versions, version solving failed.

So it seems that the private repositories are ignored and Poetry tries to resolve these dependencies with the public PyPI.

Interestingly, this all works as expected in Poetry version 1.0.10, so there must’ve been a regression a while ago that has not been fixed since.

Issue Analytics

State:
Created 3 years ago
Reactions:7
Comments:12 (1 by maintainers)

Top GitHub Comments

3reactions

itssimoncommented, May 3, 2021

I think this is actually an issue with pip. It seems that when multiple secondary repositories share the same hostname (e.g. gitlab.com) but have different credentials, pip will use the first credentials and then think it’s already authenticated for the other repositories. GitLab returns a 404 if the credentials are wrong, so it seems to pip as if the packages don’t exist, but in fact it’s just an authentication error.

2reactions

cquick01commented, Dec 14, 2021

I’ve managed to fix this issue by using a GitLab personal access token instead of a deploy token. Using poetry 1.1.12.

I’m not sure if this is an issue with poetry, GitLab, or if this is intended behavior and I’m misunderstanding GitLab’s deploy tokens.

I believe this works because you are able to use the same Personal Access Token for all the different private repos that are under the same domain name. The project-level Deploy Tokens are different for each project.

The issue here seems to be that Poetry doesn’t handle multiple repository configurations under the same domain name with different credentials. Authentication will succeed for the first repo, but fail for subsequent ones.

This isn’t an issue with a Personal Access Token, because Poetry will use the same token for all the repos.

Edit: Adding these print statements in LegacyRepository.__init__() shows that the same credentials are being used for all the private repos under the same domain

        self._basic_auth = None
        username, password = self._authenticator.get_credentials_for_url(self._url)
+       print(f"{self._url=}")
+       print(f"{username=}")
+       print(f"{password=}")

shows

self._url='https://gitlab.domain.tld/api/v4/projects/xxx2/packages/pypi/simple'
username='__token__'
password='glpat-<TOKEN>'
self._url='https://gitlab.domain.tld/api/v4/projects/xxx5/packages/pypi/simple'
username='__token__'
password='glpat-<TOKEN>'
self._url='https://gitlab.domain.tld/api/v4/projects/xxx0/packages/pypi/simple'
username='__token__'
password='glpat-<TOKEN>'

I’m trying to dig deeper to see if I can get it working.

Top Results From Across the Web

python - Poetry trying to install everything from private repository

poetry first checks if the project is in your private-pypi and then checks public-pypi . Here the error is mostly because of 403...

History | Poetry - Python dependency management and ...

Fix an issue where the deprecated JSON API was used to query PyPI for available versions of a package (#6081). Fix an issue...

Poetry: Dependency Management for Python - PyPI

Poetry helps you declare, manage and install dependencies of Python projects, ensuring you have the right stack everywhere. Poet Install. The package is...

Poetry - The Blue Book

--dependency : Package to require with a version constraint. Should be in format foo:1.0.0 . ... In this case, you will need to...

Package List — Spack 0.20.0.dev0 documentation

Spack package: alquimia/package.py. Versions: develop, 1.0.10, 1.0.9, xsdk-0.6.0, xsdk-0.5.0; Build Dependencies: cmake, ninja, mpi, hdf5, pflotran, petsc ...