Bootstrap patroni with PGbackrest

Hello,

if i get it correct, to restore whole cluster from a backup the correct way is to remove patroni cluster, then restart patroni so it would bootstrap a new one, but i must specify custom bootstrap method. I am having trouble with this. My bootstrap section of patroni.yml looks like this:

bootstrap:
    dcs:
        ttl: 30
        loop_wait: 10
        retry_timeout: 10
        maximum_lag_on_failover: 1048576
        postgresql:
            use_pg_rewind: true
            recovery_conf:
                restore_command: 'pgbackrest --stanza=testpg archive-get %f %p'
    method: pgbackrest
    pgbackrest:
        command: pgbackrest --stanza=testpg  --process-max=4 restore
        keep_existing_recovery_conf: True
        no_params: True


    initdb:
    - encoding: UTF8
    - data-checksums

< pg_hba section here>.

So i remove cluster, clear PGDATA directory. Restart patroni and it starts to bootstrap, but fails with following messages:

Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: 2019-10-11 15:07:24,104 INFO: Lock owner: None; I am testpgdb1
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: 2019-10-11 15:07:24,127 INFO: trying to bootstrap a new cluster
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: 2019-10-11 15:07:24,129 INFO: Running custom bootstrap script: pgbackrest --stanza=testpg  --process-max=4 restore
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: ERROR: [031]: invalid option '--scope=postgres'
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: 2019-10-11 15:07:24,172 INFO: removing initialize key after failed attempt to bootstrap the cluster
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: 2019-10-11 15:07:24,278 INFO: Lock owner: None; I am testpgdb1
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: Traceback (most recent call last):
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: File "/usr/bin/patroni", line 11, in <module>
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: load_entry_point('patroni==1.6.0', 'console_scripts', 'patroni')()
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 196, in main
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: return patroni_main()
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 160, in patroni_main
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: patroni.run()
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 125, in run
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: logger.info(self.ha.run_cycle())
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1344, in run_cycle
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: info = self._run_cycle()
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1253, in _run_cycle
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: return self.post_bootstrap()
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1149, in post_bootstrap
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: self.cancel_initialization()
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1144, in cancel_initialization
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: raise PatroniException('Failed to bootstrap cluster')
Oct 11 15:07:24 testpgdb1.foo.bar patroni[42256]: patroni.exceptions.PatroniException: 'Failed to bootstrap cluster'

As i got from the docs (https://patroni.readthedocs.io/en/latest/replica_bootstrap.html), patroni supplies the parameter, that causes problem, as pgbackrest doesn’t have one with such name. So i added “no_params: True” line, but no luck. What am i doing wrong?

P.S.: i’ve managed to successfully restore cluster doing following:

stopped patroni on each node
removed contents of old pgdata data dir on each node
manualy restored from pgbackrest on one node
after starting patroni, it became leader
after starting patroni on other nodes the successfully restored themselves with pgbackrest and start ed follwing the leader.

so custom ‘create_replica_methods’ using pbackrest works correctly.

But i still can’t get bootstrapping master with pgbackrest to work.

Issue Analytics

State:
Created 4 years ago
Comments:23

Top GitHub Comments

3reactions

kolbaskycommented, Oct 16, 2019

@vjtm, glad you’ve figured out. @CyberDem0n, maybe you could implement “no params” option for bootstraping in some future release. That would be neater than writing wrappers for one-line command.

1reaction

guihechengcommented, Mar 27, 2020

@amadhaka ok

To config the archive stuff, you need the following:

bootstrap:
    dcs:
        ttl: 30
        loop_wait: 10
        retry_timeout: 10
        postgresql:
            use_pg_rewind: true
            parameters:
              wal_level: replica
              archive_mode: 'on'
              archive_command: pgbackrest --stanza=patroni_stanza archive-push %p
              archive_timeout: 60
           recovery_conf:
             restore_command: pgbackrest --stanza=patroni_stanza archive-get %f %p
             standby_mode: 'on'

Then, there may be 2 operations you want to do:

Bootstrap
PITR

To do the 1st, you should follow the discussion in this thread above, because this is not the case I faced. To do the 2nd, you should have the following:

    # should be commented out
    #initdb:
    #- encoding: UTF8
    #- data-checksums
    
    method: pgbackrest
    pgbackrest:
        command: /pgbackrest_bootstrap.sh
        keep_existing_recovery_conf: True

And the script pgbackrest_bootstrap.sh

#!/bin/bash
sudo -u postgres mv /data/patroni.bak/* /data/patroni/

After you stop patroni, pgbackrest restore, and restart your patroni service, do the operation as suggested by the thread as I was suggested above. etcdctl rm /db/postgres/initialize Then you have a complete PITR.

I hope it helps and don’t mess up things, good luck 😉

Top Results From Across the Web

How to use pgbackrest to bootstrap or add replica to HA Patroni

By default, Patroni uses initdb to create a new Postgres instance (bootstrapping) or pg_basebackup to create a new replica based on whether ...

Patroni and pgBackRest combined - pgstef's blog

I see more and more questions about pgBackRest in a Patroni cluster on community channels. ... Let's edit the bootstrap configuration part:.

Point-in-time-recovery (PITR) with Patroni - Support Community

A: As of pgBackrest 2.23 and Patroni 1.6.4, it is only possible to perform a PITR if you bootstrap the cluster.

Replica imaging and bootstrap — Patroni 2.1.5 documentation

This is useful when bootstrapping from a backup with tools like pgBackRest that generate the appropriate recovery.conf for you. Note. Bootstrap methods are ......

Protecting your data with Patroni and pgbackrest / Federico ...

It will be shown how to bootstrap a new patroni cluster by using an existing pgbackrest repository and how patroni can create and...