Thread: Vacuum full is slow
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Vacuum full is very slow for me . I dont know how to speed it up. It takes between 60 and 90 minutes. I have set up autovacuum but I also run vacuum full once per week. The slowest parts in the vacuum full output are : INFO: "a": moved 14076 row versions, truncated 6013 to 1005 pages DETAIL: CPU 3.51s/2.16u sec elapsed 1156.00 sec. INFO: "b": moved 22174 row versions, truncated 1285 to 933 pages DETAIL: CPU 3.77s/1.52u sec elapsed 443.79 sec. INFO: "c": moved 36897 row versions, truncated 2824 to 1988 pages DETAIL: CPU 3.26s/1.45u sec elapsed 676.18 sec. How can I speed it up? Postgres version 8.1.3 Thanks in advance -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF/m3UIo1XmbAXRboRAnfHAKCVobTZGF9MlTjuAOkzIQESv1SDoQCfah67 hdCkn/4KtnlYk1mqcS1u8bY= =/3Y4 -----END PGP SIGNATURE-----
Ruben Rubio wrote: > Vacuum full is very slow for me . I dont know how to speed it up. It > takes between 60 and 90 minutes. > > I have set up autovacuum but I also run vacuum full once per week. Do you really need to run vacuum full? I don't know you're workload, but usually you're better off just not running it. One alternative is to run CLUSTER instead of VACUUM FULL. It's usually faster, but beware that it's not safe if you're concurrently running serializable transactions that access the table. pg_dump in particular is a problem. In a maintenance window with no other activity, however, it's ok. > The slowest parts in the vacuum full output are : > > INFO: "a": moved 14076 row versions, truncated 6013 to 1005 pages > DETAIL: CPU 3.51s/2.16u sec elapsed 1156.00 sec. > > INFO: "b": moved 22174 row versions, truncated 1285 to 933 pages > DETAIL: CPU 3.77s/1.52u sec elapsed 443.79 sec. > > INFO: "c": moved 36897 row versions, truncated 2824 to 1988 pages > DETAIL: CPU 3.26s/1.45u sec elapsed 676.18 sec. > > How can I speed it up? You don't have vacuum_cost_delay set, do you? How long does normal vacuum run? The manual suggests dropping all indexes before running vacuum full, and recreating them afterwards. That's worth trying. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > > You don't have vacuum_cost_delay set, do you? How long does normal > vacuum run? vacuum_cost_delay = 100 No idea how long will take normal vacuum. I ll try tonight when there is not too much load. > > The manual suggests dropping all indexes before running vacuum full, and > recreating them afterwards. That's worth trying. > I ll try that also. Is there any way to do it? Do i have to delete / create each one manually? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF/pxLIo1XmbAXRboRAjR1AJ9V4kBDCd++HSmUm8+ZCLs2RY0xnACfZ7Mp uBC031TFhO2NGOihfWPAQQ8= =QCYi -----END PGP SIGNATURE-----
>>vacuum_cost_delay = 100
>>No idea how long will take normal vacuum. I ll try tonight when there is
>>not too much load.
That can really take the VACUUM a long time to complete, but you might want to have it there as it will be good for performance by setting it a little high in a high OLTP environment.
I will recommend setting it to 0 first and then you can start moving it high as per your needs...
--
Shoaib Mir
EnterpriseDB (www.enterprisedb.com)
>>No idea how long will take normal vacuum. I ll try tonight when there is
>>not too much load.
That can really take the VACUUM a long time to complete, but you might want to have it there as it will be good for performance by setting it a little high in a high OLTP environment.
I will recommend setting it to 0 first and then you can start moving it high as per your needs...
--
Shoaib Mir
EnterpriseDB (www.enterprisedb.com)
On 3/19/07, Ruben Rubio <ruben@rentalia.com> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
>
> You don't have vacuum_cost_delay set, do you? How long does normal
> vacuum run?
vacuum_cost_delay = 100
No idea how long will take normal vacuum. I ll try tonight when there is
not too much load.
>
> The manual suggests dropping all indexes before running vacuum full, and
> recreating them afterwards. That's worth trying.
>
I ll try that also. Is there any way to do it? Do i have to delete /
create each one manually?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFF/pxLIo1XmbAXRboRAjR1AJ9V4kBDCd++HSmUm8+ZCLs2RY0xnACfZ7Mp
uBC031TFhO2NGOihfWPAQQ8=
=QCYi
-----END PGP SIGNATURE-----
---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Hello all, I sent a similar post to a FreeBSD group, but thought I'd might try here too. I am completing a box for PostgreSQL server on FreeBSD. Selecting a RAID controller I decided to go with 3ware SE9650-16, following good opinions about 3ware controllers found on FreeBSD and PostgreSQL groups. However my dealer suggest me not to go with 3ware, and take Promise SuperTrak EX16350, instead. This suggestion does not have any technical background and it comes generally from the fact of limited availability of 16x 3ware controllers on the local market and immediate availability of Promise. Is this technically a good idea to take Promise instead of 3ware or rather I definitely should insist on 3ware and wait for it? Thank you Ireneusz Pluta
On 3/20/07, Ireneusz Pluta <ipluta@wp.pl> wrote: > Hello all, > > I sent a similar post to a FreeBSD group, but thought I'd might try here too. > > I am completing a box for PostgreSQL server on FreeBSD. Selecting a RAID controller I decided to go > with 3ware SE9650-16, following good opinions about 3ware controllers found on FreeBSD and > PostgreSQL groups. > > However my dealer suggest me not to go with 3ware, and take Promise SuperTrak EX16350, instead. This > suggestion does not have any technical background and it comes generally from the fact of limited > availability of 16x 3ware controllers on the local market and immediate availability of Promise. > > Is this technically a good idea to take Promise instead of 3ware or rather I definitely should > insist on 3ware and wait for it? Promise raid controllers are famous for being software based with all the real work being done in the driver. Without doing the research this may or may not be the case with this particular controller. Another issue with cheap RAID controllers is the performance may not be as good as software raid...in fact it may be worse. Look for benchmarks on the web and be skeptical. merlin
On Tue, Mar 20, 2007 at 10:18:45AM -0400, Merlin Moncure wrote: > On 3/20/07, Ireneusz Pluta <ipluta@wp.pl> wrote: > >Hello all, > > > >I sent a similar post to a FreeBSD group, but thought I'd might try here > >too. > > > >I am completing a box for PostgreSQL server on FreeBSD. Selecting a RAID > >controller I decided to go > >with 3ware SE9650-16, following good opinions about 3ware controllers > >found on FreeBSD and > >PostgreSQL groups. > > > >However my dealer suggest me not to go with 3ware, and take Promise > >SuperTrak EX16350, instead. This > >suggestion does not have any technical background and it comes generally > >from the fact of limited > >availability of 16x 3ware controllers on the local market and immediate > >availability of Promise. > > > >Is this technically a good idea to take Promise instead of 3ware or rather > >I definitely should > >insist on 3ware and wait for it? > > > Promise raid controllers are famous for being software based with all > the real work being done in the driver. Without doing the research > this may or may not be the case with this particular controller. > Another issue with cheap RAID controllers is the performance may not > be as good as software raid...in fact it may be worse. Look for > benchmarks on the web and be skeptical. A Promise RAID is the only hardware RAID I've ever had eat an entire array for me... Granted this was one of those "external array with SCSI to the host", but it's certainly turned me away from Promise.. Probably not related to the controller in question, just their general quality level. //Magnus
> Is this technically a good idea to take Promise instead of 3ware or > rather I definitely should insist on 3ware and wait for it? Use 3Ware they are proven to provide a decent raid controller for SATA/PATA. Promise on the other hand... not so much. Joshua D. Drake > > Thank you > > Ireneusz Pluta > > > ---------------------------(end of broadcast)--------------------------- > TIP 7: You can help support the PostgreSQL project by donating at > > http://www.postgresql.org/about/donate > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/
On 20-Mar-07, at 9:23 AM, Ireneusz Pluta wrote: > Hello all, > > I sent a similar post to a FreeBSD group, but thought I'd might try > here too. > > I am completing a box for PostgreSQL server on FreeBSD. Selecting a > RAID controller I decided to go with 3ware SE9650-16, following > good opinions about 3ware controllers found on FreeBSD and > PostgreSQL groups. > > However my dealer suggest me not to go with 3ware, and take Promise > SuperTrak EX16350, instead. This suggestion does not have any > technical background and it comes generally from the fact of > limited availability of 16x 3ware controllers on the local market > and immediate availability of Promise. > > Is this technically a good idea to take Promise instead of 3ware or > rather I definitely should insist on 3ware and wait for it? > The reality is that most dealers have no idea what is "good" for a database application. It is likely that this card is better for him somehow ( more margin, easier to get, etc.) I'd stick with 3ware, areca, or lsi. And even then I'd check it when I got it to make sure it lived up to it's reputation. Dave > Thank you > > Ireneusz Pluta > > > ---------------------------(end of > broadcast)--------------------------- > TIP 7: You can help support the PostgreSQL project by donating at > > http://www.postgresql.org/about/donate >
On 20-Mar-07, at 1:53 PM, Benjamin Arai wrote: > This is a little biased but I would stay away from areca only > because they have fans on the card. At some point down the line > that card is going to die. When it does there is really no telling > what it will do to your data. I personally use 3Ware cards, they > work well but I have had one die before (1/10). > Well, they are also the only one of the bunch that I am aware of that will sell you 1G of cache. Plus if you use battery backup sooner or later you have to replace the batteries. I use areca all the time and I've never had a fan die, but I admit it is a point of failure. Dave
At 02:08 PM 3/20/2007, Dave Cramer wrote: >On 20-Mar-07, at 1:53 PM, Benjamin Arai wrote: > >>This is a little biased but I would stay away from areca only >>because they have fans on the card. At some point down the line >>that card is going to die. When it does there is really no telling >>what it will do to your data. Ummm ?what? fan? The Intel IOP341 (AKA 81341) based ARC-12xx cards are what people are most likely going to want to buy at this point, and they are fanless: http://www.areca.us/support/photo_gallery.htm The "lore" is that +3ware is best at random IO and Areca is best at streaming IO. OLTP => 3ware. OLAP => Areca. - stay away from Adaptec or Promise for any mission critical role. = LSI is a mixed bag. >Well, they are also the only one of the bunch that I am aware of >that will sell you 1G of cache. Actually, it's up to 2GB of BB cache... 2GB DDR2 SDRAMs are cheap and easy to get now. I've actually been agitating for Areca to support 4GB of RAM. >Plus if you use battery backup sooner or later you have to replace >the batteries. I use areca all the time and I've never had a fan >die, but I admit it is a point of failure. I've had the whole card die (massive cooling failure in NOC led to ...), but never any component on the card. OTOH, I'm conservative about how much heat per unit area I'm willing to allow to occur in or near my DB servers. Cheers, Ron
On Mon, 2007-03-19 at 06:02, Ruben Rubio wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > Vacuum full is very slow for me . I dont know how to speed it up. It > takes between 60 and 90 minutes. > > I have set up autovacuum but I also run vacuum full once per week. Note two things. 1: you need to update your pgsql version. 8.1.3 is a bit old. 2: You shouldn't normally need to run vacuum full. Vacuum full is there to get you out of problems created when regular vacuum falls behind. It contributes to index bloat as well. If routine vacuuming isn't working, regular vacuum full is not the answer (well, 99% of the time it's not). Fixing routing vacuuming is the answer. If you don't have an actual problem with routine vacuuming, you would be better off writing a monitoring script to keep track of bloat in tables and send you an email than running vacuum full all the time.