If db master removes a necessary WAL segment, changeserver goes into an infinite loop

[kriswehner]

This happened after failure recovery, when a test db server had filled its disks with WAL. After cleaning out the WAL, we removed some WAL that changeserver needs. I would expect this to simply terminate the process, but instead it goes into an infinite loop.

Log output:

INFO[33951] Connected to Postgres using replication slot "test_changeserver"
WARN[33951] Server returned an error: ERROR 58P01 requested WAL segment 0000000E000004FC000000D6 has already been removed walsender.c 2096 XLogRead

[gbrail]

I put some code in 30x/goscaffold -- you should be able to send a HUP to the process to get it to output a stack dump (like Java ;-) That might help us understand where this happens. In general, if PG has a fatal replication error, the change server should disconnect and exponentially back off just like it does when we can't reach the DB at all.

…

On Wed, Nov 30, 2016 at 6:54 AM, Kris Wehner ***@***.***> wrote: This happened after failure recovery, when a test db server had filled its disks with WAL. After cleaning out the WAL, we removed some WAL that changeserver needs. I would expect this to simply terminate the process, but instead it goes into an infinite loop. Log output: INFO[33951] Connected to Postgres using replication slot "test_changeserver" WARN[33951] Server returned an error: ERROR 58P01 requested WAL segment 0000000E000004FC000000D6 has already been removed walsender.c 2096 XLogRead — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#32>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAf0ayyCpJkw0YoPC2nCGynteH7oaq4Fks5rDY4tgaJpZM4LAR7E> .