Thread: Vacuum and Reindex hangs
I am attempting to vacuum and reindex my database. It keeps timing out. See commands and last part of output below. The vacuum or reindex only takes a short time to complete normally because the database it less than 50 mb. I have the query timeout set to 2 minutes, but I do not know if this can affect vacuumdb or reindexdb and the time should take much less than 2 minutes anyway.
I am using 8.3.3 on Centos 5.
Any advice would be greatly appreciated.
vacuumdb -f -v -z -e -d pipetracker-live
INFO: analyzing "general.t_access"
INFO: "t_access": scanned 3000 of 20186 pages, containing 100449 live rows and 0 dead rows; 3000 rows in sample, 675888 estimated total rows
INFO: vacuuming "information_schema.sql_parts"
INFO: "sql_parts": found 0 removable, 9 nonremovable row versions in 1 pages
DETAIL: 0 dead row versions cannot be removed yet.
Nonremovable row versions range from 56 to 96 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 7500 bytes.
0 pages are or will become empty, including 0 at the end of the table.
1 pages containing 7500 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: "sql_parts": moved 0 row versions, truncated 1 to 1 pages
DETAIL: CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: vacuuming "pg_toast.pg_toast_11449"
INFO: "pg_toast_11449": found 0 removable, 0 nonremovable row versions in 0 pages
DETAIL: 0 dead row versions cannot be removed yet.
Nonremovable row versions range from 0 to 0 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 0 bytes.
0 pages are or will become empty, including 0 at the end of the table.
0 pages containing 0 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "pg_toast_11449_index" now contains 0 row versions in 1 pages
DETAIL: 0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: analyzing "information_schema.sql_parts"
INFO: "sql_parts": scanned 1 of 1 pages, containing 9 live rows and 0 dead rows; 9 rows in sample, 9 estimated total rows
vacuumdb: vacuuming of database "pipetracker-live" failed: ERROR: canceling statement due to statement timeout
reindexdb -d pipetracker-live
NOTICE: table "pg_class" was reindexed
NOTICE: table "pg_type" was reindexed
NOTICE: table "sql_features" was reindexed
NOTICE: table "sql_implementation_info" was reindexed
NOTICE: table "sql_languages" was reindexed
NOTICE: table "pg_statistic" was reindexed
NOTICE: table "sql_packages" was reindexed
NOTICE: table "t_access" was reindexed
NOTICE: table "sql_parts" was reindexed
reindexdb: reindexing of database "pipetracker-live" failed: ERROR: canceling statement due to statement timeout
I am using 8.3.3 on Centos 5.
Any advice would be greatly appreciated.
vacuumdb -f -v -z -e -d pipetracker-live
INFO: analyzing "general.t_access"
INFO: "t_access": scanned 3000 of 20186 pages, containing 100449 live rows and 0 dead rows; 3000 rows in sample, 675888 estimated total rows
INFO: vacuuming "information_schema.sql_parts"
INFO: "sql_parts": found 0 removable, 9 nonremovable row versions in 1 pages
DETAIL: 0 dead row versions cannot be removed yet.
Nonremovable row versions range from 56 to 96 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 7500 bytes.
0 pages are or will become empty, including 0 at the end of the table.
1 pages containing 7500 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: "sql_parts": moved 0 row versions, truncated 1 to 1 pages
DETAIL: CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: vacuuming "pg_toast.pg_toast_11449"
INFO: "pg_toast_11449": found 0 removable, 0 nonremovable row versions in 0 pages
DETAIL: 0 dead row versions cannot be removed yet.
Nonremovable row versions range from 0 to 0 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 0 bytes.
0 pages are or will become empty, including 0 at the end of the table.
0 pages containing 0 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "pg_toast_11449_index" now contains 0 row versions in 1 pages
DETAIL: 0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: analyzing "information_schema.sql_parts"
INFO: "sql_parts": scanned 1 of 1 pages, containing 9 live rows and 0 dead rows; 9 rows in sample, 9 estimated total rows
vacuumdb: vacuuming of database "pipetracker-live" failed: ERROR: canceling statement due to statement timeout
reindexdb -d pipetracker-live
NOTICE: table "pg_class" was reindexed
NOTICE: table "pg_type" was reindexed
NOTICE: table "sql_features" was reindexed
NOTICE: table "sql_implementation_info" was reindexed
NOTICE: table "sql_languages" was reindexed
NOTICE: table "pg_statistic" was reindexed
NOTICE: table "sql_packages" was reindexed
NOTICE: table "t_access" was reindexed
NOTICE: table "sql_parts" was reindexed
reindexdb: reindexing of database "pipetracker-live" failed: ERROR: canceling statement due to statement timeout
On 15/01/2009 20:06, Jason Long wrote: <big><big><b>I am attempting to vacuum...[snip] I don't mean to be a pain, but could you please avoid HUGE type sizes such as the above....or better still, avoid using HTML altogether in your emails to this list. It makes it look as if you are not just shouting, but SCREAMING at the top of your lungs! :-) Thanks in advance..... Ray. ------------------------------------------------------------------ Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland rod@iol.ie Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals ------------------------------------------------------------------
On Thu, 2009-01-15 at 14:06 -0600, Jason Long wrote: > I am attempting to vacuum and reindex my database. It keeps timing > out. See commands and last part of output below. The vacuum or > reindex only takes a short time to complete normally because the > database it less than 50 mb. I have the query timeout set to 2 > minutes, but I do not know if this can affect vacuumdb or reindexdb > and the time should take much less than 2 minutes anyway. > > I am using 8.3.3 on Centos 5. You have statement_timeout set. You can use SET in psql to change this.... SET statement_timeout TO ... Joshua D. Drake -- PostgreSQL - XMPP: jdrake@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Thursday 15 January 2009, Jason Long <mailing.list@supernovasoftware.com> wrote: > *I am attempting to vacuum and reindex my database. It keeps timing > out. See commands and last part of output below. The vacuum or reindex > only takes a short time to complete normally because the database it > less than 50 mb. I have the query timeout set to 2 minutes, but I do > not know if this can affect vacuumdb or reindexdb and the time should > take much less than 2 minutes anyway. Obviously it is affecting it and it is taking longer than 2 minutes regardless of how long you think it "should take". Set it higher for these tasks or buy a faster server. -- Current Peeve: The mindset that the Internet is some sort of school for novice sysadmins and that everyone -not- doing stupid dangerous things should act like patient teachers with the ones who are. -- Bill Cole, NANAE
On Thu, 2009-01-15 at 20:13 +0000, Raymond O'Donnell wrote: > On 15/01/2009 20:06, Jason Long wrote: > > <big><big><b>I am attempting to vacuum...[snip] > > I don't mean to be a pain, but could you please avoid HUGE type sizes > such as the above....or better still, avoid using HTML altogether in > your emails to this list. > > It makes it look as if you are not just shouting, but SCREAMING at the > top of your lungs! :-) The answer to this is to not allow HTML email at all to your client. It is the first thing I disable on any family/friend/church member that asks for help. Joshua D. Drake -- PostgreSQL - XMPP: jdrake@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On 15/01/2009 20:21, Joshua D. Drake wrote: > On Thu, 2009-01-15 at 20:13 +0000, Raymond O'Donnell wrote: >> I don't mean to be a pain, but could you please avoid HUGE type sizes >> such as the above....or better still, avoid using HTML altogether in >> your emails to this list. > The answer to this is to not allow HTML email at all to your client. It > is the first thing I disable on any family/friend/church member that > asks for help. Indeed - 'tis now done, and you won't hear any more complaints from me.... I still think I made a fair point, though. :-) Ray. ------------------------------------------------------------------ Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland rod@iol.ie Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals ------------------------------------------------------------------
Alan Hodgson wrote:
Well the sever is plenty fast. It has 2 quad core 1600MHz FSB 3.0 GHz Xeon 5472 CPUs and a very light workload.
My statement about the time is that it has never taken that long. Ever. Not even close.
I will increase the setting and see if that works.
A faster server.On Thursday 15 January 2009, Jason Long <mailing.list@supernovasoftware.com> wrote:*I am attempting to vacuum and reindex my database. It keeps timing out. See commands and last part of output below. The vacuum or reindex only takes a short time to complete normally because the database it less than 50 mb. I have the query timeout set to 2 minutes, but I do not know if this can affect vacuumdb or reindexdb and the time should take much less than 2 minutes anyway.Obviously it is affecting it and it is taking longer than 2 minutes regardless of how long you think it "should take". Set it higher for these tasks or buy a faster server.
Well the sever is plenty fast. It has 2 quad core 1600MHz FSB 3.0 GHz Xeon 5472 CPUs and a very light workload.
My statement about the time is that it has never taken that long. Ever. Not even close.
I will increase the setting and see if that works.
I don't mean to be a pain either and I mean no disrespect to anyone on this list in the following comments.
However, this is about the most anal list ever.
I see so many emails on here about people complaining regarding the proper way to reply or post to the list.
I used larger font to point point out my statement from the code. I also did not realize it appeared that large to you.
My res is 2560X1600 so it didn't look that large.
I apologize.
Just out of curiosity, why are you so apposed to HTML in a email?
Raymond O'Donnell wrote:
However, this is about the most anal list ever.
I see so many emails on here about people complaining regarding the proper way to reply or post to the list.
I used larger font to point point out my statement from the code. I also did not realize it appeared that large to you.
My res is 2560X1600 so it didn't look that large.
I apologize.
Just out of curiosity, why are you so apposed to HTML in a email?
Raymond O'Donnell wrote:
On 15/01/2009 20:06, Jason Long wrote: <big><big><b>I am attempting to vacuum...[snip] I don't mean to be a pain, but could you please avoid HUGE type sizes such as the above....or better still, avoid using HTML altogether in your emails to this list. It makes it look as if you are not just shouting, but SCREAMING at the top of your lungs! :-) Thanks in advance..... Ray. ------------------------------------------------------------------ Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland rod@iol.ie Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals ------------------------------------------------------------------
and we also oppose to answering on top of message, and citing everything underneeth. Why? Because your words should say what you mean, not show it by its look. Hence, plain ascii is enough for us - and should be for every intelligent human being. On Thu, Jan 15, 2009 at 8:32 PM, Jason Long <mailing.list@supernovasoftware.com> wrote: > I don't mean to be a pain either and I mean no disrespect to anyone on this > list in the following comments. > > However, this is about the most anal list ever. > I see so many emails on here about people complaining regarding the proper > way to reply or post to the list. > > I used larger font to point point out my statement from the code. I also > did not realize it appeared that large to you. > > My res is 2560X1600 so it didn't look that large. > > I apologize. > > Just out of curiosity, why are you so apposed to HTML in a email? > > Raymond O'Donnell wrote: > > On 15/01/2009 20:06, Jason Long wrote: > > <big><big><b>I am attempting to vacuum...[snip] > > I don't mean to be a pain, but could you please avoid HUGE type sizes > such as the above....or better still, avoid using HTML altogether in > your emails to this list. > > It makes it look as if you are not just shouting, but SCREAMING at the > top of your lungs! :-) > > Thanks in advance..... > > Ray. > > ------------------------------------------------------------------ > Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland > rod@iol.ie > Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals > ------------------------------------------------------------------ > > -- GJ
On Thu, 2009-01-15 at 14:32 -0600, Jason Long wrote: > I don't mean to be a pain either and I mean no disrespect to anyone on > this list in the following comments. > > However, this is about the most anal list ever. You haven't been to the debian list have you? :). > I see so many emails on here about people complaining regarding the > proper way to reply or post to the list. The basic gist is this. We have to be anal otherwise there is no conformity to a standard (even if the standard is arbitrary). Without that conformity it is impossible to be productive on the list. > > I used larger font to point point out my statement from the code. I > also did not realize it appeared that large to you. Right but just as a tip that is really not needed on a technical list. Your words should be able to convey your meaning without the requirement of a cosmetic modification. If it can't it would be better to post to a pastebin or static web page and link a URL. Just FYI most on this list probably never saw your HTML. They, like I said have already turned off HTML email. > > Just out of curiosity, why are you so apposed to HTML in a email? http://www.american.edu/econ/notes/htmlmail.htm http://www.georgedillon.com/web/html_email_is_evil.shtml Sincerely, Joshua D. Drake -- PostgreSQL - XMPP: jdrake@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Thu, Jan 15, 2009 at 1:28 PM, Jason Long <mailing.list@supernovasoftware.com> wrote: > A faster server. > Well the sever is plenty fast. It has 2 quad core 1600MHz FSB 3.0 GHz Xeon > 5472 CPUs and a very light workload. A few things. That doesn't make a fast server. The disk i/o subsystem makes a fast server. And you've mentioned nothing of that yet. If you've got a 16 SAS Disk RAID -10 array on a fast RAID controller with battery backed cache, you've got a fast database server. If you've got a single SATA drive or a mirror set of two SATA drives, you do not have a fast database server. > My statement about the time is that it has never taken that long. Ever. > Not even close. I wonder if you're getting a lot of bloating in your indexes from the full vacuums. Is there a reason you're running full vacuums over regular vacuums? While there are quite a few circumstances where full vacuums are the right answer, most of the time they are not, at least not on a regular basis. A lot of things can cause your current vacuums to run slow. Maybe there's a competing regular autovacuum that's kicked in at the same time, someone is backing up the database, and so on. As for : > However, this is about the most anal list ever. > I see so many emails on here about people complaining regarding the proper way to reply > or post to the list. That's because many of us receive hundreds of emails a week, and if everyone starts sending html email,using bouncing email addresses, or sending emails to 5 lists at once, things can get out of hand pretty quickly. Since your email agent is sending multi-part mime email with regular text and html email, there's no real reason to complain, as any agent worth its salt can be set to show only the text part. I'm pretty sure the email archive process also lops off the html part before storing it. Busy lists tend to be anal. Wanna get a bunch of people mad at once? Break the rules on the lkml. We're a bunch of fuzzy little kittens playing with balls of yarn by comparison. :)
On 15/01/2009 20:32, Jason Long wrote: > However, this is about the most anal list ever. I see so many emails on > here about people complaining regarding the proper way to reply or post > to the list. Well, as someone else has just pointed out, it's all about readability and making your words easy to understand. However, let's not re-open old flame wars - this has been raked over *many* times in the past. > *Just out of curiosity, why are you so apposed to HTML in a email?* I'm not opposed to HTML email, as long as it's easy to understand and looks reasonably OK. Ray. ------------------------------------------------------------------ Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland rod@iol.ie Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals ------------------------------------------------------------------
On Jan 15, 2009, at 12:32 PM, Jason Long wrote: > I don't mean to be a pain either and I mean no disrespect to anyone > on this list in the following comments. > > However, this is about the most anal list ever. > I see so many emails on here about people complaining regarding the > proper way to reply or post to the list. > > I used larger font to point point out my statement from the code. I > also did not realize it appeared that large to you. > > My res is 2560X1600 so it didn't look that large. > > I apologize. > > Just out of curiosity, why are you so apposed to HTML in a email? There are technical reasons. One of them is that most clients sending html mail are actually sending multipart/alternative mail with an html part and a plain text generate mechanically from the html part. People using plain text mail clients (and there tend to be a lot of them on technical lists) will see the plain text part only. That's fine for some sorts of email, but leads to tears when someone insists that they've hilighted the problem in red or bold or whatever, and half the recipients are reading the plain text version. Also, HTML mail tends to not use standard email quoting, meaning that it tends to discard context about who said what, which makes it very difficult to follow discussions. And it often plays hell with list digests and archives. There are also social reasons - it tends to be used by people who don't realize how it looks when received by the recipient, and who don't care. It's generally a sign of someone who has little experience of normal technical mailing list etiquette or polite online behaviour (such as following community norms). It also correlates strongly with people whose behaviour is antisocial in other respects (not so much use of html per-se as use of large font sizes, colours and suchlike, which are perceived by most recipients as SHOUTING, or vehement defense of html email). And it tends to derail threads into discussions like this, which is always bad. I'm sure none of that other than the last actually applies to you, but those are the expectations you set by using HTML email and then insulting all the list members when someone asks you to stop. That's not the way to get useful help from a technical peer support list. Cheers, Steve
On 15/01/2009 20:44, Scott Marlowe wrote: > We're a bunch of fuzzy little kittens > playing with balls of yarn by comparison. :) Now *there's* an image! :-) Ray. ------------------------------------------------------------------ Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland rod@iol.ie Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals ------------------------------------------------------------------
Steve Atkins wrote: > > On Jan 15, 2009, at 12:32 PM, Jason Long wrote: > >> I don't mean to be a pain either and I mean no disrespect to anyone >> on this list in the following comments. >> >> However, this is about the most anal list ever. >> I see so many emails on here about people complaining regarding the >> proper way to reply or post to the list. >> >> I used larger font to point point out my statement from the code. I >> also did not realize it appeared that large to you. >> >> My res is 2560X1600 so it didn't look that large. >> >> I apologize. >> >> Just out of curiosity, why are you so apposed to HTML in a email? > > There are technical reasons. One of them is that most clients sending > html mail are actually sending multipart/alternative mail with an html > part and a plain text generate mechanically from the html part. People > using plain text mail clients (and there tend to be a lot of them on > technical > lists) will see the plain text part only. That's fine for some sorts > of email, > but leads to tears when someone insists that they've hilighted the > problem > in red or bold or whatever, and half the recipients are reading the plain > text version. > > Also, HTML mail tends to not use standard email quoting, meaning that > it tends to discard context about who said what, which makes it very > difficult to follow discussions. And it often plays hell with list > digests and > archives. > > There are also social reasons - it tends to be used by people who > don't realize > how it looks when received by the recipient, and who don't care. It's > generally a > sign of someone who has little experience of normal technical mailing > list etiquette or > polite online behaviour (such as following community norms). > > It also correlates strongly with people whose behaviour is antisocial > in other > respects (not so much use of html per-se as use of large font sizes, > colours and suchlike, which are perceived by most recipients as SHOUTING, > or vehement defense of html email). > > And it tends to derail threads into discussions like this, which is > always > bad. > > I'm sure none of that other than the last actually applies to you, but > those are > the expectations you set by using HTML email and then insulting all > the list members when someone asks you to stop. That's not the way to get > useful help from a technical peer support list. Fair enough and I had no intention of insulting anyone. Being anal is not necessarily a bad thing. :) > > Cheers, > Steve >
On Thu, 2009-01-15 at 20:39 +0000, Grzegorz Jaśkiewicz wrote: > and we also oppose to answering on top of message, and citing > everything underneeth. > Why? Because your words should say what you mean, not show it by its > look. Hence, plain ascii is enough for us - and should be for every > intelligent human being. Well now that we are completely off topic :). Let me just say that it is exactly the type of thinking above that makes Open Source people seem like jerks. I know many perfectly intelligent people that are better served through diagrams, pdf and color than a mailing list. Most of them make sure geeks like us, *EAT*. Does that mean they are not intelligent or perhaps that there talent set is just different? Sincerely, Joshua D. Drake -- PostgreSQL - XMPP: jdrake@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997
On Thu, Jan 15, 2009 at 1:56 PM, Jason Long <mailing.list@supernovasoftware.com> wrote: > Steve Atkins wrote: >> >> I'm sure none of that other than the last actually applies to you, but >> those are >> the expectations you set by using HTML email and then insulting all >> the list members when someone asks you to stop. That's not the way to get >> useful help from a technical peer support list. > > Fair enough and I had no intention of insulting anyone. Being anal is not > necessarily a bad thing. :) I certainly wasn't offended. It seemed like an honest question.
On Jan 15, 2009, at 1:02 PM, Scott Marlowe wrote: > On Thu, Jan 15, 2009 at 1:56 PM, Jason Long > <mailing.list@supernovasoftware.com> wrote: >> Steve Atkins wrote: >>> >>> I'm sure none of that other than the last actually applies to you, >>> but >>> those are >>> the expectations you set by using HTML email and then insulting all >>> the list members when someone asks you to stop. That's not the way >>> to get >>> useful help from a technical peer support list. >> >> Fair enough and I had no intention of insulting anyone. Being anal >> is not >> necessarily a bad thing. :) > > I certainly wasn't offended. It seemed like an honest question. I wasn't either. But while this is a less prickly environment than some technical mailing lists, it's still pretty prickly, and there's a risk some people were offended by being described as anal (even if, or perhaps especially when, it's an accurate description :) ). I find HTML mail quite useful when I know that all the recipients can read it - but it tends to harm communication when some of the recipients can't. In the latter case I find plain text mail, possibly with attached diagrams or embedded URLs to web docs more useful than inline HTML. That's technical mailing lists, pretty much. Cheers, Steve
Scott Marlowe wrote:
The whole DB is 50 mb with minimal users.
Would a 16 SAS Disk RAID -10 really help me that much?
The dataset is small, but contains a complex data structure with many joins between tables.
I would appreciate any advice on the effect of a high end disk setup for my case.
The reason for the vacuum now it that I have a dynamic query that sometimes brings the server to a grinding halt.
This is why I set the timeout to 2 minutes.
Now all I get is users pissed about 2 times a day and the problem corrects itself before they finish complaining to me.
I tried to add some more indexes and then tried to vacuum full and reindex the database.
You got me. I have a set of mirrored raptors. I am not sure the disk i/o subsystem is a bottleneck.On Thu, Jan 15, 2009 at 1:28 PM, Jason Long <mailing.list@supernovasoftware.com> wrote:A faster server. Well the sever is plenty fast. It has 2 quad core 1600MHz FSB 3.0 GHz Xeon 5472 CPUs and a very light workload.A few things. That doesn't make a fast server. The disk i/o subsystem makes a fast server. And you've mentioned nothing of that yet. If you've got a 16 SAS Disk RAID -10 array on a fast RAID controller with battery backed cache, you've got a fast database server. If you've got a single SATA drive or a mirror set of two SATA drives, you do not have a fast database server.
The whole DB is 50 mb with minimal users.
Would a 16 SAS Disk RAID -10 really help me that much?
The dataset is small, but contains a complex data structure with many joins between tables.
I would appreciate any advice on the effect of a high end disk setup for my case.
I used to use full vacuum and reindex ever night just before I did a dump backup. Then I started to try the autovacuum.My statement about the time is that it has never taken that long. Ever. Not even close.I wonder if you're getting a lot of bloating in your indexes from the full vacuums. Is there a reason you're running full vacuums over regular vacuums? While there are quite a few circumstances where full vacuums are the right answer, most of the time they are not, at least not on a regular basis.
The reason for the vacuum now it that I have a dynamic query that sometimes brings the server to a grinding halt.
This is why I set the timeout to 2 minutes.
Now all I get is users pissed about 2 times a day and the problem corrects itself before they finish complaining to me.
I tried to add some more indexes and then tried to vacuum full and reindex the database.
Autovacuum could be the case, but I have total control of the database an no backups are in progress.A lot of things can cause your current vacuums to run slow. Maybe there's a competing regular autovacuum that's kicked in at the same time, someone is backing up the database, and so on.
I totally understand and will limit my use of HTML in the future.As for :However, this is about the most anal list ever. I see so many emails on here about people complaining regarding the proper way to reply or post to the list.That's because many of us receive hundreds of emails a week, and if everyone starts sending html email,using bouncing email addresses, or sending emails to 5 lists at once, things can get out of hand pretty quickly. Since your email agent is sending multi-part mime email with regular text and html email, there's no real reason to complain, as any agent worth its salt can be set to show only the text part. I'm pretty sure the email archive process also lops off the html part before storing it.
Busy lists tend to be anal. Wanna get a bunch of people mad at once? Break the rules on the lkml. We're a bunch of fuzzy little kittens playing with balls of yarn by comparison. :)
On Thu, Jan 15, 2009 at 2:24 PM, Jason Long <mailing.list@supernovasoftware.com> wrote: > Scott Marlowe wrote: > You got me. I have a set of mirrored raptors. I am not sure the disk i/o > subsystem is a bottleneck. > The whole DB is 50 mb with minimal users. Then you're only ever writing to the db, and 50Meg is teeny tiny. Even my laptop can write out at 50Megs in about 5 seconds. > Would a 16 SAS Disk RAID -10 really help me that much? Depends on your usage pattern. We use a 12 disk one with 15k5 seagates to handle a couple of 30G databases running 2000 to 5000 requests per minute, 97% or so being reads. > The dataset is small, but contains a complex data structure with many joins > between tables. > I would appreciate any advice on the effect of a high end disk setup for my > case. Given how small your dataset is, a simple caching RAID controller should offer enough throughput that you don't need more drives. > I used to use full vacuum and reindex ever night just before I did a dump > backup. Then I started to try the autovacuum. > The reason for the vacuum now it that I have a dynamic query that sometimes > brings the server to a grinding halt. You might be better served by a cluster command than a vacuum full. It rewrites the table much like a vacuum full, but it's faster, doesn't bloat the index, and results in a table who's order follows that of the index you clustered on. We have a large table that went from 5 to 300 seconds to .5 to 3 seconds avg query speed because of a cluster command. Took 80 minutes to cluster the first time, but it was well worth it.
On Thu, 15 Jan 2009 12:56:51 -0800 "Joshua D. Drake" <jd@commandprompt.com> wrote: > I know many perfectly intelligent people that are better served > through diagrams, pdf and color than a mailing list. Most of them > make sure geeks like us, *EAT*. > Does that mean they are not intelligent or perhaps that there > talent set is just different? Inspired by your mail I just wrote a post that explain why jerky geeks may be fine esthetes but still they've good reason to avoid HTML in emails. I think I followed a slight different approach to the more traditional ones. http://www.webthatworks.it/d1/node/page/are_geeks_rich_media_impaired_or_why_html_emails_are_evil -- Ivan Sergio Borgonovo http://www.webthatworks.it