Thread: vacuumdb -a -f
Hello, Even though I run vacuumdb -a -f every night with no exceptions or problems, my database size remains 5.6 GB. After I do a dump/restore, the new database size is 4.0 GB. How could that be possible? That's a significant amount of space. If I can clean up my existing databases by 30%, I'd be in much better shape. Has anyone else noticed this? Thanks, ~DjK
"D Kavan" <bitsandbytes88@hotmail.com> writes:
> Even though I run vacuumdb -a -f every night with no exceptions or problems,
> my database size remains 5.6 GB. After I do a dump/restore, the new
> database size is 4.0 GB. How could that be possible?
The extra 1.6GB probably represents the amount of junk you generate in
one day. So, forget the -f and instead do plain vacuums on a more
frequent basis. Make sure your FSM settings are large enough, too.
regards, tom lane
Hi, Thanks for the tips. Unfortunatley for me, even after started doing vacuumdb -a 3 times a day and increasing fsm dramatically , the size of the database won't go down even 1 MB. It's stil at 5.6 GB, size after restore = 4 GB. I even did a stop/start instead of a re-load to make sure the settings took affect. Would a reboot help? max_fsm_pages = 16000001 max_fsm_relations = 1000000 shared_buffers = 65536 work_mem = 32768 maintenance work mem = 786432 checkpoint_segments = 18 ##/etc/sysctl.conf nel.shmall = 524288 #kernel.shmall = 2097152 #kernel.shmmax = 2147483648 #kernel.shmmax = 1073741824 kernel.shmmax = 6979321856 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 fs.file-max = 65536 net.ipv4.ip_local_port_range = 1024 65000 vm.overcommit_memory = 2 ~DjK >From: Tom Lane <tgl@sss.pgh.pa.us> >To: "D Kavan" <bitsandbytes88@hotmail.com> >CC: pgsql-admin@postgresql.org >Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 -0400 > >"D Kavan" <bitsandbytes88@hotmail.com> writes: > > Even though I run vacuumdb -a -f every night with no exceptions or >problems, > > my database size remains 5.6 GB. After I do a dump/restore, the new > > database size is 4.0 GB. How could that be possible? > >The extra 1.6GB probably represents the amount of junk you generate in >one day. So, forget the -f and instead do plain vacuums on a more >frequent basis. Make sure your FSM settings are large enough, too. > > regards, tom lane
D Kavan wrote: > Hi, > > Thanks for the tips. > > Unfortunatley for me, even after started doing vacuumdb -a 3 times a > day and increasing fsm dramatically , the size of the database won't go > down even 1 MB. It's stil at 5.6 GB, size after restore = 4 GB. I > even did a stop/start instead of a re-load to make sure the settings > took affect. Would a reboot help? A vacuum full will decrease the size but I am not sure why the size is a problem. I normal vacuum will not remove rows, it only make them reusable. So at any given point your database is going to grow after use. I didn't see this whole thread but is there something specific that seems to be the issue beyond size? Sincerely, Joshua D. Drake > > max_fsm_pages = 16000001 > max_fsm_relations = 1000000 > > shared_buffers = 65536 > work_mem = 32768 > maintenance work mem = 786432 > > checkpoint_segments = 18 > > > ##/etc/sysctl.conf > > nel.shmall = 524288 > #kernel.shmall = 2097152 > #kernel.shmmax = 2147483648 > #kernel.shmmax = 1073741824 > kernel.shmmax = 6979321856 > kernel.shmmni = 4096 > kernel.sem = 250 32000 100 128 > fs.file-max = 65536 > net.ipv4.ip_local_port_range = 1024 65000 > vm.overcommit_memory = 2 > > > ~DjK > > > > > >> From: Tom Lane <tgl@sss.pgh.pa.us> >> To: "D Kavan" <bitsandbytes88@hotmail.com> >> CC: pgsql-admin@postgresql.org >> Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 -0400 >> >> "D Kavan" <bitsandbytes88@hotmail.com> writes: >> > Even though I run vacuumdb -a -f every night with no exceptions or >> problems, >> > my database size remains 5.6 GB. After I do a dump/restore, the new >> > database size is 4.0 GB. How could that be possible? >> >> The extra 1.6GB probably represents the amount of junk you generate in >> one day. So, forget the -f and instead do plain vacuums on a more >> frequent basis. Make sure your FSM settings are large enough, too. >> >> regards, tom lane > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: explain analyze is your friend -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/
How are you finding out the DB size?
G.-
On 8/17/05, D Kavan <bitsandbytes88@hotmail.com> wrote:
Hi,
Thanks for the tips.
Unfortunatley for me, even after started doing vacuumdb -a 3 times a day
and increasing fsm dramatically , the size of the database won't go down
even 1 MB. It's stil at 5.6 GB, size after restore = 4 GB. I even did a
stop/start instead of a re-load to make sure the settings took affect. Would
a reboot help?
max_fsm_pages = 16000001
max_fsm_relations = 1000000
shared_buffers = 65536
work_mem = 32768
maintenance work mem = 786432
checkpoint_segments = 18
##/etc/sysctl.conf
nel.shmall = 524288
#kernel.shmall = 2097152
#kernel.shmmax = 2147483648
#kernel.shmmax = 1073741824
kernel.shmmax = 6979321856
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
vm.overcommit_memory = 2
~DjK
>From: Tom Lane <tgl@sss.pgh.pa.us>
>To: "D Kavan" < bitsandbytes88@hotmail.com>
>CC: pgsql-admin@postgresql.org
>Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 -0400
>
>"D Kavan" <bitsandbytes88@hotmail.com> writes:
> > Even though I run vacuumdb -a -f every night with no exceptions or
>problems,
> > my database size remains 5.6 GB. After I do a dump/restore, the new
> > database size is 4.0 GB. How could that be possible?
>
>The extra 1.6GB probably represents the amount of junk you generate in
>one day. So, forget the -f and instead do plain vacuums on a more
>frequent basis. Make sure your FSM settings are large enough, too.
>
> regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend
--
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.
CASE CLOSED! Thanks for the tip on the reindex script. That did the trick. It was one table in particuliar. ~DjK >From: Chris Hoover <revoohc@gmail.com> >To: Guido Barosio <gbarosio@gmail.com> >CC: D Kavan <bitsandbytes88@hotmail.com>, pgsql-admin@postgresql.org >Subject: Re: [ADMIN] vacuumdb -a -f >Date: Wed, 17 Aug 2005 12:57:16 -0400 > >You might want to try doing a reindexdb -a -e (reindexdb is in the >contrib directory of your pg source). The first time I ran this, I >gained back a significant amount of space. > >I now run a vacuumdb -v -f -a and then a reindexdb -a -e every weekend >to have PostgreSQL give back as much space as it can. I have ended up >doing this do to space and i/o constraints. > >Give the reindexdb script a try and see if you don't get your space back. > >HTH, > >Chris > >On 8/17/05, Guido Barosio <gbarosio@gmail.com> wrote: > > How are you finding out the DB size? > > > > G.- > > > > > > > > On 8/17/05, D Kavan <bitsandbytes88@hotmail.com> wrote: > > > Hi, > > > > > > Thanks for the tips. > > > > > > Unfortunatley for me, even after started doing vacuumdb -a 3 times a >day > > > and increasing fsm dramatically , the size of the database won't go >down > > > even 1 MB. It's stil at 5.6 GB, size after restore = 4 GB. I even >did a > > > stop/start instead of a re-load to make sure the settings took affect. > > Would > > > a reboot help? > > > > > > max_fsm_pages = 16000001 > > > max_fsm_relations = 1000000 > > > > > > shared_buffers = 65536 > > > work_mem = 32768 > > > maintenance work mem = 786432 > > > > > > checkpoint_segments = 18 > > > > > > > > > ##/etc/sysctl.conf > > > > > > nel.shmall = 524288 > > > #kernel.shmall = 2097152 > > > #kernel.shmmax = 2147483648 > > > #kernel.shmmax = 1073741824 > > > kernel.shmmax = 6979321856 > > > kernel.shmmni = 4096 > > > kernel.sem = 250 32000 100 128 > > > fs.file-max = 65536 > > > net.ipv4.ip_local_port_range = 1024 65000 > > > vm.overcommit_memory = 2 > > > > > > > > > ~DjK > > > > > > > > > > > > > > > > > > >From: Tom Lane <tgl@sss.pgh.pa.us> > > > >To: "D Kavan" < bitsandbytes88@hotmail.com> > > > >CC: pgsql-admin@postgresql.org > > > >Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 >-0400 > > > > > > > >"D Kavan" <bitsandbytes88@hotmail.com> writes: > > > > > Even though I run vacuumdb -a -f every night with no exceptions or > > > >problems, > > > > > my database size remains 5.6 GB. After I do a dump/restore, the >new > > > > > database size is 4.0 GB. How could that be possible? > > > > > > > >The extra 1.6GB probably represents the amount of junk you generate >in > > > >one day. So, forget the -f and instead do plain vacuums on a more > > > >frequent basis. Make sure your FSM settings are large enough, too. > > > > > > > > regards, tom lane > > > > > > > > > > > > ---------------------------(end of > > broadcast)--------------------------- > > > TIP 6: explain analyze is your friend > > > > > > > > > > > -- > > "Adopting the position that you are smarter than an automatic > > optimization algorithm is generally a good way to achieve less > > performance, not more" - Tom Lane.
You might want to try doing a reindexdb -a -e (reindexdb is in the contrib directory of your pg source). The first time I ran this, I gained back a significant amount of space. I now run a vacuumdb -v -f -a and then a reindexdb -a -e every weekend to have PostgreSQL give back as much space as it can. I have ended up doing this do to space and i/o constraints. Give the reindexdb script a try and see if you don't get your space back. HTH, Chris On 8/17/05, Guido Barosio <gbarosio@gmail.com> wrote: > How are you finding out the DB size? > > G.- > > > > On 8/17/05, D Kavan <bitsandbytes88@hotmail.com> wrote: > > Hi, > > > > Thanks for the tips. > > > > Unfortunatley for me, even after started doing vacuumdb -a 3 times a day > > and increasing fsm dramatically , the size of the database won't go down > > even 1 MB. It's stil at 5.6 GB, size after restore = 4 GB. I even did a > > stop/start instead of a re-load to make sure the settings took affect. > Would > > a reboot help? > > > > max_fsm_pages = 16000001 > > max_fsm_relations = 1000000 > > > > shared_buffers = 65536 > > work_mem = 32768 > > maintenance work mem = 786432 > > > > checkpoint_segments = 18 > > > > > > ##/etc/sysctl.conf > > > > nel.shmall = 524288 > > #kernel.shmall = 2097152 > > #kernel.shmmax = 2147483648 > > #kernel.shmmax = 1073741824 > > kernel.shmmax = 6979321856 > > kernel.shmmni = 4096 > > kernel.sem = 250 32000 100 128 > > fs.file-max = 65536 > > net.ipv4.ip_local_port_range = 1024 65000 > > vm.overcommit_memory = 2 > > > > > > ~DjK > > > > > > > > > > > > >From: Tom Lane <tgl@sss.pgh.pa.us> > > >To: "D Kavan" < bitsandbytes88@hotmail.com> > > >CC: pgsql-admin@postgresql.org > > >Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 -0400 > > > > > >"D Kavan" <bitsandbytes88@hotmail.com> writes: > > > > Even though I run vacuumdb -a -f every night with no exceptions or > > >problems, > > > > my database size remains 5.6 GB. After I do a dump/restore, the new > > > > database size is 4.0 GB. How could that be possible? > > > > > >The extra 1.6GB probably represents the amount of junk you generate in > > >one day. So, forget the -f and instead do plain vacuums on a more > > >frequent basis. Make sure your FSM settings are large enough, too. > > > > > > regards, tom lane > > > > > > > > ---------------------------(end of > broadcast)--------------------------- > > TIP 6: explain analyze is your friend > > > > > > -- > "Adopting the position that you are smarter than an automatic > optimization algorithm is generally a good way to achieve less > performance, not more" - Tom Lane.
'du -h' in the base directory. >How are you finding out the DB size? > G.-
On Wed, Aug 17, 2005 at 02:15:42PM -0400, D Kavan wrote: > > 'du -h' in the base directory. > > >How are you finding out the DB size? You are considering the difference in XLog segment size, right? The pg_xlog directory may start almost empty and then start filling. The space will not be recovered by any maintenance command. -- Alvaro Herrera (<alvherre[a]alvh.no-ip.org>) Y dijo Dios: "Que sea Satanás, para que la gente no me culpe de todo a mí." "Y que hayan abogados, para que la gente no culpe de todo a Satanás"