logical replication snapshots - Mailing list pgsql-general

From Dimitri Maziuk
Subject logical replication snapshots
Date
Msg-id 3bf34856-efb9-01e8-3147-1e0dff1f7131@bmrb.wisc.edu
Whole thread Raw
Responses Re: logical replication snapshots
List pgsql-general
Hi everyone,

I'm playing with logical replication in postgres-10 and I got it to hang
on startup with

> 2018-07-23 13:15:06.577 CDT [18624] FATAL:  the database system is starting up
> 2018-07-23 13:15:06.577 CDT [18624] DEBUG:  shmem_exit(1): 0 before_shmem_exit callbacks to make
> 2018-07-23 13:15:06.577 CDT [18624] DEBUG:  shmem_exit(1): 0 on_shmem_exit callbacks to make
> 2018-07-23 13:15:06.577 CDT [18624] DEBUG:  proc_exit(1): 1 callbacks to make
> 2018-07-23 13:15:06.577 CDT [18624] DEBUG:  exit(1)
> 2018-07-23 13:15:06.577 CDT [18624] DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
> 2018-07-23 13:15:06.577 CDT [18624] DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
> 2018-07-23 13:15:06.577 CDT [18624] DEBUG:  proc_exit(-1): 0 callbacks to make
... forever ...

After poking around:

# find /var/lib/pgsql/10/data/pg_logical/snapshots -printf . | wc -c
12727975

It looks like ZFS simply can't handle this many files: `ls` etc. hang
forever.

So my question is, does it have to be that many? And if not, is there a
knob to keep the number down?

TIA
--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu


Attachment

pgsql-general by date:

Previous
From: Adrian Klaver
Date:
Subject: Re: Connections on cluster not being logged
Next
From: Kevin Brannen
Date:
Subject: RE: Improving pg_dump performance