Misleading error message mentioning etcdv2 when etcdv3 is unavailable

See original GitHub issue

Describe the bug The install instructions seem to indicate one should install the python-etcd package for etcd3 support as well; pip install patroni[etcd3] also installs that:

Installing collected packages: python-etcd
    Running setup.py install for python-etcd ... done
Successfully installed python-etcd-0.4.5

However, it seems to me, that python-etcd 0.4.5 does not have support for v3, see e.g. https://github.com/jplana/python-etcd/issues/252

Also, if I try to start patroni, I get a reference to v2 even if etcd3 is in DCS:

2021-12-15 16:52:11,264 DEBUG: New etcd client created for https://XXXX:443
2021-12-15 16:52:11,266 DEBUG: Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=0, status=None)
2021-12-15 16:52:11,266 DEBUG: Starting new HTTPS connection (1): XXX:443
2021-12-15 16:52:11,278 DEBUG: https://XXX:443 "GET /version HTTP/1.1" 502 150
2021-12-15 16:53:16,535 ERROR: Failed to get list of machines from https://XXX:443/v2: <Unknown error: '<html>
<head><title>502 Bad Gateway</title></head>
[...]

I checked that manually running ETCDCTL_API=3 etcdctl --endpoints [...] get members works.

Is this a problem on my side, is there some code needed, or is this just a documentation issue?

Environment

Patroni version: 2.1.2
PostgreSQL version: n/a
DCS (and its version): etcd 3.4.3

Patroni configuration file

etcd3:
  protocol: "https"
  url: "https://XXX:443"

Issue Analytics

State:
Created 2 years ago
Comments:6 (3 by maintainers)

Top GitHub Comments

1reaction

CyberDem0ncommented, Dec 15, 2021

but I assumed that if etcdctl can get the members fine, it should work?

Not necessarily. etcdctl uses gRPC directly, but Patroni have no other choice as relying on JSON gRPC gateway, which seems to be not enabled by default when etcd uses config file. Ref: https://github.com/etcd-io/etcd/issues/12093

0reactions

mbanckcommented, Feb 16, 2022

However, If I understand correctly, there’s some v2 fallback if Patroni can’t talk to v3? (ERROR: Failed to get list of machines from https://XXX:443/v2), shouldn’t there at least be a big WARNING that v3 didn’t work, and probably an ERROR really?

This is not really a fallback, just seems to be somewhat a wrong error message, because it failed to resolve which version_prefix to use:

https://github.com/zalando/patroni/blob/dc9ff4cb8ae974ae5cc5b2521a12a53143936c2c/patroni/dcs/etcd3.py#L222-L247 due to the failure of GET /version call.

Ok, thanks for the explanation (and sorry for the late reply); it took me hours of debugging trying to figure out why it was using v2 so I think it would be useful to make this error clearer, I’ve renamed the issue title accordingly