Re: Slow shutdowns sometimes on RDS Postgres - Mailing list pgsql-general

On 09/13/2018 03:04 PM, Chris Williams wrote:
> Hi,
>
> I'm using AWS RDS Postgres (9.6.6) and have run into very slow 
> shutdowns (10+ minutes) a few times when making database modifications 
> (e.g. reboot, changing instance size, etc.).  Other times, it shuts 
> down quickly (1 minute or so).  I have not been able to figure out why 
> sometimes it takes a long time to shutdown.

This is probably something you would have to talk to Amazon about. AWS 
RDS Postgres is a fork of PostgreSQL and not 100% compatible from an 
administrative perspective.

JD



>
> When it happens, I see a bunch of lines in the postgres log like the 
> following over and over (almost every second or two) during this 10 
> minute shutdown period:
> 2018-09-12 06:37:01 
> UTC:XXX.XXX.XXX.XXX(19712):my_user@my_db:[16495]:FATAL:
> 2018-09-12 06:37:01 
> UTC:localhost(31368):rdsadmin@rdsadmin:[16488]:FATAL: the database 
> system is shutting down
>
> Once I start seeing these messages, I start manually shutting down all 
> of our applications that are connected to the db.  I'm not sure if 
> shutting down the apps fixes it or if there's some timeout on the RDS 
> side, but it seems like once I start doing this, the database finally 
> shuts down.
>
> When it takes this long to shut down, it ends up causing a lot more 
> downtime than I would like.  I've tried asking AWS's support why it 
> takes so long to shutdown sometimes, but they basically just told me 
> that's "how it works" and that I should try to shut down all of my 
> connections ahead of time before making database modifications.
>
> We just have a few ruby on rails applications connected to the 
> database, and don't really have any long running or heavy queries and 
> the db is under very light load, so I don't understand why it takes so 
> long to shutdown.  We do have a sizeable number of connections though 
> (about 600) and there are two replicas connected to it.  I also tried 
> setting idle_in_transaction_session_timeout to 300 seconds to see if 
> that would help, but it made no difference.
>
> I was wondering if anyone else had seen this behavior on their RDS 
> Postgres instances or had any suggestions on how I could shorten the 
> shutdown time?
>
> Thanks,
> Chris
>

-- 
Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc
***  A fault and talent of mine is to tell it exactly how it is.  ***
PostgreSQL centered full stack support, consulting and development.
Advocate: @amplifypostgres || Learn: https://postgresconf.org
*****     Unless otherwise stated, opinions are my own.   *****