Thread: Pg_repack

Pg_repack

From
Sathish Reddy
Date:
Hi 
 I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid the bash on host EC2 to run run directly with in postgres instance.it getting successful but not clearing bloat by using repack fuction in pg_repack extension.please help on these to sort out .


Thanks 
Sathish Reddy 

Re: Pg_repack

From
Ron Johnson
Date:
On Tue, Jul 23, 2024 at 1:22 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:
Hi 
 I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid the bash on host EC2 to run run directly with in postgres instance.it getting successful but not clearing bloat by using repack fuction in pg_repack extension.please help on these to sort out .

Out of curiosity, why do you feel the need to regularly run pg_repack?  IOW, why doesn't plain old VACUUM suit your needs?
 

Re: Pg_repack

From
"Deepak Pahuja ."
Date:
Hi John

Vacuum full generally takes very long time on large tables and also does exclusive lock on table causing it unavailable.
Moreover it generates huge wal files and causes replication lag on secondary in streaming replication setup.
Pg_repacknis online and removes complete bloating.

But experts can tell more.

Thanks 

From: Ron Johnson <ronljohnsonjr@gmail.com>
Sent: Tuesday, July 23, 2024 1:28:28 PM
To: Pgsql-admin <pgsql-admin@lists.postgresql.org>
Subject: Re: Pg_repack
 
On Tue, Jul 23, 2024 at 1:22 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:
Hi 
 I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid the bash on host EC2 to run run directly with in postgres instance.it getting successful but not clearing bloat by using repack fuction in pg_repack extension.please help on these to sort out .

Out of curiosity, why do you feel the need to regularly run pg_repack?  IOW, why doesn't plain old VACUUM suit your needs?
 

Re: Pg_repack

From
khan Affan
Date:
Hi 

First, use Vaccum Full or Vaccumlo if storing largeobject for clearing bloat, If you are running a script, dump it outside the RDS, such as you can dump it to EC2, and then apply PG_repack on the schema and then restore it to RDS.

As you know such services are not available on RDS. 

https://www.postgresql.org/docs/current/vacuumlo.html

https://www.postgresql.org/docs/current/sql-vacuum.html

Thanks & regards


Muhammad Affan (아판)

PostgreSQL Technical Support Engineer / Pakistan R&D

Interlace Plaza 4th floor Twinhub office 32 I8 Markaz, Islamabad, Pakistan


On Tue, Jul 23, 2024 at 10:22 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:
Hi 
 I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid the bash on host EC2 to run run directly with in postgres instance.it getting successful but not clearing bloat by using repack fuction in pg_repack extension.please help on these to sort out .


Thanks 
Sathish Reddy 

Re: Pg_repack

From
Sathish Reddy
Date:

We are planning to run pg repack from pg_cron in RDS environment not in EC2  help me schedule job pg_repack

On Tue, Jul 23, 2024, 11:13 AM khan Affan <bawag773@gmail.com> wrote:
Hi 

First, use Vaccum Full or Vaccumlo if storing largeobject for clearing bloat, If you are running a script, dump it outside the RDS, such as you can dump it to EC2, and then apply PG_repack on the schema and then restore it to RDS.

As you know such services are not available on RDS. 

https://www.postgresql.org/docs/current/vacuumlo.html

https://www.postgresql.org/docs/current/sql-vacuum.html

Thanks & regards


Muhammad Affan (아판)

PostgreSQL Technical Support Engineer / Pakistan R&D

Interlace Plaza 4th floor Twinhub office 32 I8 Markaz, Islamabad, Pakistan


On Tue, Jul 23, 2024 at 10:22 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:
Hi 
 I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid the bash on host EC2 to run run directly with in postgres instance.it getting successful but not clearing bloat by using repack fuction in pg_repack extension.please help on these to sort out .


Thanks 
Sathish Reddy 

Re: Pg_repack

From
khan Affan
Date:
As stated before, security constraints prevent pg_repack from being executed directly in an RDS PostgreSQL environment.
Because RDS maintains a secure environment, installing custom extensions like pg_repack is restricted.
pg_cron in RDS is intended to be used for scheduling internal PostgreSQL functions or operations; it is not intended to be used with external utilities such as pg_repack. 
The alternative approach is to export your database schema and data (excluding large objects) to an external PostgreSQL instance, run pg_repack on the external instance to reclaim space, and then import the cleaned data back into your RDS instance.

Thanks & regards


Muhammad Affan (아판)

PostgreSQL Technical Support Engineer / Pakistan R&D

Interlace Plaza 4th floor Twinhub office 32 I8 Markaz, Islamabad, Pakistan



On Tue, Jul 23, 2024 at 10:53 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:

We are planning to run pg repack from pg_cron in RDS environment not in EC2  help me schedule job pg_repack

On Tue, Jul 23, 2024, 11:13 AM khan Affan <bawag773@gmail.com> wrote:
Hi 

First, use Vaccum Full or Vaccumlo if storing largeobject for clearing bloat, If you are running a script, dump it outside the RDS, such as you can dump it to EC2, and then apply PG_repack on the schema and then restore it to RDS.

As you know such services are not available on RDS. 

https://www.postgresql.org/docs/current/vacuumlo.html

https://www.postgresql.org/docs/current/sql-vacuum.html

Thanks & regards


Muhammad Affan (아판)

PostgreSQL Technical Support Engineer / Pakistan R&D

Interlace Plaza 4th floor Twinhub office 32 I8 Markaz, Islamabad, Pakistan


On Tue, Jul 23, 2024 at 10:22 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:
Hi 
 I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid the bash on host EC2 to run run directly with in postgres instance.it getting successful but not clearing bloat by using repack fuction in pg_repack extension.please help on these to sort out .


Thanks 
Sathish Reddy 

Re: Pg_repack

From
Wells Oliver
Date:
Consider running pg_repack in a Docker container on ECS/Fargate etc if you don't want to spin up an EC2 instance. I think a free t2.micro sort of EC2 instance would be plenty to repack against an RDS host, might want to spring for a few cores to make the indexes faster...


On Mon, Jul 22, 2024 at 11:29 PM khan Affan <bawag773@gmail.com> wrote:
As stated before, security constraints prevent pg_repack from being executed directly in an RDS PostgreSQL environment.
Because RDS maintains a secure environment, installing custom extensions like pg_repack is restricted.
pg_cron in RDS is intended to be used for scheduling internal PostgreSQL functions or operations; it is not intended to be used with external utilities such as pg_repack. 
The alternative approach is to export your database schema and data (excluding large objects) to an external PostgreSQL instance, run pg_repack on the external instance to reclaim space, and then import the cleaned data back into your RDS instance.

Thanks & regards


Muhammad Affan (아판)

PostgreSQL Technical Support Engineer / Pakistan R&D

Interlace Plaza 4th floor Twinhub office 32 I8 Markaz, Islamabad, Pakistan



On Tue, Jul 23, 2024 at 10:53 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:

We are planning to run pg repack from pg_cron in RDS environment not in EC2  help me schedule job pg_repack

On Tue, Jul 23, 2024, 11:13 AM khan Affan <bawag773@gmail.com> wrote:
Hi 

First, use Vaccum Full or Vaccumlo if storing largeobject for clearing bloat, If you are running a script, dump it outside the RDS, such as you can dump it to EC2, and then apply PG_repack on the schema and then restore it to RDS.

As you know such services are not available on RDS. 

https://www.postgresql.org/docs/current/vacuumlo.html

https://www.postgresql.org/docs/current/sql-vacuum.html

Thanks & regards


Muhammad Affan (아판)

PostgreSQL Technical Support Engineer / Pakistan R&D

Interlace Plaza 4th floor Twinhub office 32 I8 Markaz, Islamabad, Pakistan


On Tue, Jul 23, 2024 at 10:22 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:
Hi 
 I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid the bash on host EC2 to run run directly with in postgres instance.it getting successful but not clearing bloat by using repack fuction in pg_repack extension.please help on these to sort out .


Thanks 
Sathish Reddy 


--

Re: Pg_repack

From
Laurenz Albe
Date:
On Tue, 2024-07-23 at 10:52 +0530, Sathish Reddy wrote:
> I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid
> the bash on host EC2 to run run directly with in postgres instance.it getting successful
> but not clearing bloat by using repack fuction in pg_repack extension.please help on
> these to sort out .

If you have the need to do this regularly, use pg_squeeze, which avoids the need
for pg_cron.  It can automatically trigger a rebuild of the table according to
conditions you specify.

Yours,
Laurenz Albe



Re: Pg_repack

From
Sathish Reddy
Date:

Provide me example how can we use pg_squeeze from RDS environment

On Tue, Jul 23, 2024, 12:31 PM Laurenz Albe <laurenz.albe@cybertec.at> wrote:
On Tue, 2024-07-23 at 10:52 +0530, Sathish Reddy wrote:
> I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid
> the bash on host EC2 to run run directly with in postgres instance.it getting successful
> but not clearing bloat by using repack fuction in pg_repack extension.please help on
> these to sort out .

If you have the need to do this regularly, use pg_squeeze, which avoids the need
for pg_cron.  It can automatically trigger a rebuild of the table according to
conditions you specify.

Yours,
Laurenz Albe

Re: Pg_repack

From
Laurenz Albe
Date:
On Tue, 2024-07-23 at 12:50 +0530, Sathish Reddy wrote:
> Provide me example how can we use pg_squeeze from RDS environment

With a hosted service, that would only work if Amazon provides the
extension.  Sorry, I didn't see that.  Then you probably have to go
the hard way.

Yours,
Laurenz Albe



Re: Pg_repack

From
Ron Johnson
Date:
Pahu,

I asked about VACUUM, not VACUUM FULL.

On Tue, Jul 23, 2024 at 1:36 AM Deepak Pahuja . <deepakpahuja@hotmail.com> wrote:
Hi John

Vacuum full generally takes very long time on large tables and also does exclusive lock on table causing it unavailable.
Moreover it generates huge wal files and causes replication lag on secondary in streaming replication setup.
Pg_repacknis online and removes complete bloating.

But experts can tell more.

Thanks 

From: Ron Johnson <ronljohnsonjr@gmail.com>
Sent: Tuesday, July 23, 2024 1:28:28 PM
To: Pgsql-admin <pgsql-admin@lists.postgresql.org>
Subject: Re: Pg_repack
 
On Tue, Jul 23, 2024 at 1:22 AM Sathish Reddy <sathishreddy.postgresql@gmail.com> wrote:
Hi 
 I am trying to schedule pg_repack from pg_cron in RDS postgres environment on avoid the bash on host EC2 to run run directly with in postgres instance.it getting successful but not clearing bloat by using repack fuction in pg_repack extension.please help on these to sort out .

Out of curiosity, why do you feel the need to regularly run pg_repack?  IOW, why doesn't plain old VACUUM suit your needs?