Thread: Vacuum and Reindex hangs

Vacuum and Reindex hangs

From
Jason Long
Date:
I am attempting to vacuum and reindex my database.  It keeps timing out.  See commands and last part of output below.  The vacuum or reindex only takes a short time to complete normally because the database it less than 50 mb.  I have the query timeout set to 2 minutes, but I do not know if this can affect vacuumdb or reindexdb and the time should take much less than 2 minutes anyway.

I am using 8.3.3 on Centos 5.

Any advice would be greatly appreciated.

vacuumdb -f -v -z -e -d pipetracker-live


INFO:  analyzing "general.t_access"
INFO:  "t_access": scanned 3000 of 20186 pages, containing 100449 live rows and 0 dead rows; 3000 rows in sample, 675888 estimated total rows
INFO:  vacuuming "information_schema.sql_parts"
INFO:  "sql_parts": found 0 removable, 9 nonremovable row versions in 1 pages
DETAIL:  0 dead row versions cannot be removed yet.
Nonremovable row versions range from 56 to 96 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 7500 bytes.
0 pages are or will become empty, including 0 at the end of the table.
1 pages containing 7500 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  "sql_parts": moved 0 row versions, truncated 1 to 1 pages
DETAIL:  CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  vacuuming "pg_toast.pg_toast_11449"
INFO:  "pg_toast_11449": found 0 removable, 0 nonremovable row versions in 0 pages
DETAIL:  0 dead row versions cannot be removed yet.
Nonremovable row versions range from 0 to 0 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 0 bytes.
0 pages are or will become empty, including 0 at the end of the table.
0 pages containing 0 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  index "pg_toast_11449_index" now contains 0 row versions in 1 pages
DETAIL:  0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO:  analyzing "information_schema.sql_parts"
INFO:  "sql_parts": scanned 1 of 1 pages, containing 9 live rows and 0 dead rows; 9 rows in sample, 9 estimated total rows
vacuumdb: vacuuming of database "pipetracker-live" failed: ERROR:  canceling statement due to statement timeout

reindexdb -d pipetracker-live

NOTICE:  table "pg_class" was reindexed
NOTICE:  table "pg_type" was reindexed
NOTICE:  table "sql_features" was reindexed
NOTICE:  table "sql_implementation_info" was reindexed
NOTICE:  table "sql_languages" was reindexed
NOTICE:  table "pg_statistic" was reindexed
NOTICE:  table "sql_packages" was reindexed
NOTICE:  table "t_access" was reindexed
NOTICE:  table "sql_parts" was reindexed
reindexdb: reindexing of database "pipetracker-live" failed: ERROR:  canceling statement due to statement timeout


Re: Vacuum and Reindex hangs

From
Raymond O'Donnell
Date:
On 15/01/2009 20:06, Jason Long wrote:

<big><big><b>I am attempting to vacuum...[snip]

I don't mean to be a pain, but could you please avoid HUGE type sizes
such as the above....or better still, avoid using HTML altogether in
your emails to this list.

It makes it look as if you are not just shouting, but SCREAMING at the
top of your lungs! :-)

Thanks in advance.....

Ray.

------------------------------------------------------------------
Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland
rod@iol.ie
Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals
------------------------------------------------------------------

Re: Vacuum and Reindex hangs

From
"Joshua D. Drake"
Date:
On Thu, 2009-01-15 at 14:06 -0600, Jason Long wrote:
> I am attempting to vacuum and reindex my database.  It keeps timing
> out.  See commands and last part of output below.  The vacuum or
> reindex only takes a short time to complete normally because the
> database it less than 50 mb.  I have the query timeout set to 2
> minutes, but I do not know if this can affect vacuumdb or reindexdb
> and the time should take much less than 2 minutes anyway.
>
> I am using 8.3.3 on Centos 5.

You have statement_timeout set. You can use SET in psql to change
this....

SET statement_timeout TO ...

Joshua D. Drake

--
PostgreSQL - XMPP: jdrake@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


Re: Vacuum and Reindex hangs

From
Alan Hodgson
Date:
On Thursday 15 January 2009, Jason Long <mailing.list@supernovasoftware.com>
wrote:
> *I am attempting to vacuum and reindex my database.  It keeps timing
> out.  See commands and last part of output below.  The vacuum or reindex
> only takes a short time to complete normally because the database it
> less than 50 mb.  I have the query timeout set to 2 minutes, but I do
> not know if this can affect vacuumdb or reindexdb and the time should
> take much less than 2 minutes anyway.

Obviously it is affecting it and it is taking longer than 2 minutes
regardless of how long you think it "should take". Set it higher for these
tasks or buy a faster server.

--
Current Peeve: The mindset that the Internet is some sort of school for
novice sysadmins and that everyone -not- doing stupid dangerous things
should act like patient teachers with the ones who are. -- Bill Cole, NANAE

Re: Vacuum and Reindex hangs

From
"Joshua D. Drake"
Date:
On Thu, 2009-01-15 at 20:13 +0000, Raymond O'Donnell wrote:
> On 15/01/2009 20:06, Jason Long wrote:
>
> <big><big><b>I am attempting to vacuum...[snip]
>
> I don't mean to be a pain, but could you please avoid HUGE type sizes
> such as the above....or better still, avoid using HTML altogether in
> your emails to this list.
>
> It makes it look as if you are not just shouting, but SCREAMING at the
> top of your lungs! :-)

The answer to this is to not allow HTML email at all to your client. It
is the first thing I disable on any family/friend/church member that
asks for help.

Joshua D. Drake


--
PostgreSQL - XMPP: jdrake@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


Re: Vacuum and Reindex hangs

From
Raymond O'Donnell
Date:
On 15/01/2009 20:21, Joshua D. Drake wrote:
> On Thu, 2009-01-15 at 20:13 +0000, Raymond O'Donnell wrote:
>> I don't mean to be a pain, but could you please avoid HUGE type sizes
>> such as the above....or better still, avoid using HTML altogether in
>> your emails to this list.

> The answer to this is to not allow HTML email at all to your client. It
> is the first thing I disable on any family/friend/church member that
> asks for help.

Indeed - 'tis now done, and you won't hear any more complaints from
me.... I still think I made a fair point, though. :-)

Ray.

------------------------------------------------------------------
Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland
rod@iol.ie
Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals
------------------------------------------------------------------

Re: Vacuum and Reindex hangs

From
Jason Long
Date:
Alan Hodgson wrote:
On Thursday 15 January 2009, Jason Long <mailing.list@supernovasoftware.com> 
wrote: 
*I am attempting to vacuum and reindex my database.  It keeps timing
out.  See commands and last part of output below.  The vacuum or reindex
only takes a short time to complete normally because the database it
less than 50 mb.  I have the query timeout set to 2 minutes, but I do
not know if this can affect vacuumdb or reindexdb and the time should
take much less than 2 minutes anyway.   
Obviously it is affecting it and it is taking longer than 2 minutes 
regardless of how long you think it "should take". Set it higher for these 
tasks or buy a faster server. 
 
A faster server. 
Well the sever is plenty fast.  It has 2 quad core 1600MHz FSB 3.0 GHz Xeon 5472 CPUs and a very light workload.

My statement about the time is that it has never taken that long.  Ever.  Not even close.

I will increase the setting and see if that works.


Re: Vacuum and Reindex hangs

From
Jason Long
Date:
I don't mean to be a pain either and I mean no disrespect to anyone on this list in the following comments.

However, this is about the most anal list ever. 
I see so many emails on here about people complaining regarding the proper way to reply or post to the list.

I used larger font to point point out my statement from the code.  I also did not realize it appeared that large to you.

My res is 2560X1600 so it didn't look that large.

I apologize.

Just out of curiosity, why are you so apposed to HTML in a email?

Raymond O'Donnell wrote:
On 15/01/2009 20:06, Jason Long wrote:

<big><big><b>I am attempting to vacuum...[snip]

I don't mean to be a pain, but could you please avoid HUGE type sizes
such as the above....or better still, avoid using HTML altogether in
your emails to this list.

It makes it look as if you are not just shouting, but SCREAMING at the
top of your lungs! :-)

Thanks in advance.....

Ray.

------------------------------------------------------------------
Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland
rod@iol.ie
Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals
------------------------------------------------------------------ 

Re: Vacuum and Reindex hangs

From
Grzegorz Jaśkiewicz
Date:
and we also oppose to answering on top of message, and citing
everything underneeth.
Why? Because your words should say what you mean, not show it by its
look. Hence, plain ascii is enough for us - and should be for every
intelligent human being.


On Thu, Jan 15, 2009 at 8:32 PM, Jason Long
<mailing.list@supernovasoftware.com> wrote:
> I don't mean to be a pain either and I mean no disrespect to anyone on this
> list in the following comments.
>
> However, this is about the most anal list ever.
> I see so many emails on here about people complaining regarding the proper
> way to reply or post to the list.
>
> I used larger font to point point out my statement from the code.  I also
> did not realize it appeared that large to you.
>
> My res is 2560X1600 so it didn't look that large.
>
> I apologize.
>
> Just out of curiosity, why are you so apposed to HTML in a email?
>
> Raymond O'Donnell wrote:
>
> On 15/01/2009 20:06, Jason Long wrote:
>
> <big><big><b>I am attempting to vacuum...[snip]
>
> I don't mean to be a pain, but could you please avoid HUGE type sizes
> such as the above....or better still, avoid using HTML altogether in
> your emails to this list.
>
> It makes it look as if you are not just shouting, but SCREAMING at the
> top of your lungs! :-)
>
> Thanks in advance.....
>
> Ray.
>
> ------------------------------------------------------------------
> Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland
> rod@iol.ie
> Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals
> ------------------------------------------------------------------
>
>



--
GJ

Re: Vacuum and Reindex hangs

From
"Joshua D. Drake"
Date:
On Thu, 2009-01-15 at 14:32 -0600, Jason Long wrote:
> I don't mean to be a pain either and I mean no disrespect to anyone on
> this list in the following comments.
>
> However, this is about the most anal list ever.

You haven't been to the debian list have you? :).

> I see so many emails on here about people complaining regarding the
> proper way to reply or post to the list.

The basic gist is this. We have to be anal otherwise there is no
conformity to a standard (even if the standard is arbitrary). Without
that conformity it is impossible to be productive on the list.
>
> I used larger font to point point out my statement from the code.  I
> also did not realize it appeared that large to you.

Right but just as a tip that is really not needed on a technical list.
Your words should be able to convey your meaning without the requirement
of a cosmetic modification. If it can't it would be better to post to a
pastebin or static web page and link a URL.

Just FYI most on this list probably never saw your HTML. They, like I
said have already turned off HTML email.
>
> Just out of curiosity, why are you so apposed to HTML in a email?

http://www.american.edu/econ/notes/htmlmail.htm
http://www.georgedillon.com/web/html_email_is_evil.shtml

Sincerely,

Joshua D. Drake



--
PostgreSQL - XMPP: jdrake@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


Re: Vacuum and Reindex hangs

From
"Scott Marlowe"
Date:
On Thu, Jan 15, 2009 at 1:28 PM, Jason Long
<mailing.list@supernovasoftware.com> wrote:
> A faster server.
> Well the sever is plenty fast.  It has 2 quad core 1600MHz FSB 3.0 GHz Xeon
> 5472 CPUs and a very light workload.

A few things.

That doesn't make a fast server.  The disk i/o subsystem makes a fast
server.  And you've mentioned nothing of that yet.  If you've got a 16
SAS Disk RAID -10 array on a fast RAID controller with battery backed
cache, you've got a fast database server.

If you've got a single SATA drive or a mirror set of two SATA drives,
you do not have a fast database server.

> My statement about the time is that it has never taken that long.  Ever.
> Not even close.

I wonder if you're getting a lot of bloating in your indexes from the
full vacuums.  Is there a reason you're running full vacuums over
regular vacuums?  While there are quite a few circumstances where full
vacuums are the right answer, most of the time they are not, at least
not on a regular basis.

A lot of things can cause your current vacuums to run slow.  Maybe
there's a competing regular autovacuum that's kicked in at the same
time, someone is backing up the database, and so on.

As for :

> However, this is about the most anal list ever.
> I see so many emails on here about people complaining regarding the proper way to reply
> or post to the list.

That's because many of us receive hundreds of emails a week, and if
everyone starts sending html email,using bouncing email addresses, or
sending emails to 5 lists at once, things can get out of hand pretty
quickly.

Since your email agent is sending multi-part mime email with regular
text and html email, there's no real reason to complain, as any agent
worth its salt can be set to show only the text part.  I'm pretty sure
the email archive process also lops off the html part before storing
it.

Busy lists tend to be anal.  Wanna get a bunch of people mad at once?
Break the rules on the lkml.  We're a bunch of fuzzy little kittens
playing with balls of yarn by comparison.  :)

Re: Vacuum and Reindex hangs

From
Raymond O'Donnell
Date:
On 15/01/2009 20:32, Jason Long wrote:

> However, this is about the most anal list ever. I see so many emails on
> here about people complaining regarding the proper way to reply or post
> to the list.

Well, as someone else has just pointed out, it's all about readability
and making your words easy to understand.

However, let's not re-open old flame wars - this has been raked over
*many* times in the past.

> *Just out of curiosity, why are you so apposed to HTML in a email?*

I'm not opposed to HTML email, as long as it's easy to understand and
looks reasonably OK.

Ray.


------------------------------------------------------------------
Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland
rod@iol.ie
Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals
------------------------------------------------------------------

HTML email (was Re: Vacuum and Reindex hangs

From
Steve Atkins
Date:
On Jan 15, 2009, at 12:32 PM, Jason Long wrote:

> I don't mean to be a pain either and I mean no disrespect to anyone
> on this list in the following comments.
>
> However, this is about the most anal list ever.
> I see so many emails on here about people complaining regarding the
> proper way to reply or post to the list.
>
> I used larger font to point point out my statement from the code.  I
> also did not realize it appeared that large to you.
>
> My res is 2560X1600 so it didn't look that large.
>
> I apologize.
>
> Just out of curiosity, why are you so apposed to HTML in a email?

There are technical reasons. One of them is that most clients sending
html mail are actually sending multipart/alternative mail with an html
part and a plain text generate mechanically from the html part. People
using plain text mail clients (and there tend to be a lot of them on
technical
lists) will see the plain text part only. That's fine for some sorts
of email,
but leads to tears when someone insists that they've hilighted the
problem
in red or bold or whatever, and half the recipients are reading the
plain
text version.

Also, HTML mail tends to not use standard email quoting, meaning that
it tends to discard context about who said what, which makes it very
difficult to follow discussions. And it often plays hell with list
digests and
archives.

There are also social reasons - it tends to be used by people who
don't realize
how it looks when received by the recipient, and who don't care. It's
generally a
sign of someone who has little experience of normal technical mailing
list etiquette or
polite online behaviour (such as following community norms).

It also correlates strongly with people whose behaviour is antisocial
in other
respects (not so much use of html per-se as use of large font sizes,
colours and suchlike, which are perceived by most recipients as
SHOUTING,
or vehement defense of html email).

And it tends to derail threads into discussions like this, which is
always
bad.

I'm sure none of that other than the last actually applies to you, but
those are
the expectations you set by using HTML email and then insulting all
the list members when someone asks you to stop. That's not the way to
get
useful help from a technical peer support list.

Cheers,
   Steve

Re: Vacuum and Reindex hangs

From
Raymond O'Donnell
Date:
On 15/01/2009 20:44, Scott Marlowe wrote:

> We're a bunch of fuzzy little kittens
> playing with balls of yarn by comparison.  :)

Now *there's* an image! :-)

Ray.

------------------------------------------------------------------
Raymond O'Donnell, Director of Music, Galway Cathedral, Ireland
rod@iol.ie
Galway Cathedral Recitals: http://www.galwaycathedral.org/recitals
------------------------------------------------------------------

Re: HTML email (was Re: Vacuum and Reindex hangs

From
Jason Long
Date:
Steve Atkins wrote:
>
> On Jan 15, 2009, at 12:32 PM, Jason Long wrote:
>
>> I don't mean to be a pain either and I mean no disrespect to anyone
>> on this list in the following comments.
>>
>> However, this is about the most anal list ever.
>> I see so many emails on here about people complaining regarding the
>> proper way to reply or post to the list.
>>
>> I used larger font to point point out my statement from the code.  I
>> also did not realize it appeared that large to you.
>>
>> My res is 2560X1600 so it didn't look that large.
>>
>> I apologize.
>>
>> Just out of curiosity, why are you so apposed to HTML in a email?
>
> There are technical reasons. One of them is that most clients sending
> html mail are actually sending multipart/alternative mail with an html
> part and a plain text generate mechanically from the html part. People
> using plain text mail clients (and there tend to be a lot of them on
> technical
> lists) will see the plain text part only. That's fine for some sorts
> of email,
> but leads to tears when someone insists that they've hilighted the
> problem
> in red or bold or whatever, and half the recipients are reading the plain
> text version.
>
> Also, HTML mail tends to not use standard email quoting, meaning that
> it tends to discard context about who said what, which makes it very
> difficult to follow discussions. And it often plays hell with list
> digests and
> archives.
>
> There are also social reasons - it tends to be used by people who
> don't realize
> how it looks when received by the recipient, and who don't care. It's
> generally a
> sign of someone who has little experience of normal technical mailing
> list etiquette or
> polite online behaviour (such as following community norms).
>
> It also correlates strongly with people whose behaviour is antisocial
> in other
> respects (not so much use of html per-se as use of large font sizes,
> colours and suchlike, which are perceived by most recipients as SHOUTING,
> or vehement defense of html email).
>
> And it tends to derail threads into discussions like this, which is
> always
> bad.
>
> I'm sure none of that other than the last actually applies to you, but
> those are
> the expectations you set by using HTML email and then insulting all
> the list members when someone asks you to stop. That's not the way to get
> useful help from a technical peer support list.
Fair enough and I had no intention of insulting anyone.  Being anal is
not necessarily a bad thing. :)
>
> Cheers,
>   Steve
>


Re: Vacuum and Reindex hangs

From
"Joshua D. Drake"
Date:
On Thu, 2009-01-15 at 20:39 +0000, Grzegorz Jaśkiewicz wrote:
> and we also oppose to answering on top of message, and citing
> everything underneeth.
> Why? Because your words should say what you mean, not show it by its
> look. Hence, plain ascii is enough for us - and should be for every
> intelligent human being.

Well now that we are completely off topic :). Let me just say that it is
exactly the type of thinking above that makes Open Source people seem
like jerks.

I know many perfectly intelligent people that are better served through
diagrams, pdf and color than a mailing list. Most of them make sure
geeks like us, *EAT*.

Does that mean they are not intelligent or perhaps that there talent set
is just different?

Sincerely,

Joshua D. Drake

--
PostgreSQL - XMPP: jdrake@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


Re: HTML email (was Re: Vacuum and Reindex hangs

From
"Scott Marlowe"
Date:
On Thu, Jan 15, 2009 at 1:56 PM, Jason Long
<mailing.list@supernovasoftware.com> wrote:
> Steve Atkins wrote:
>>
>> I'm sure none of that other than the last actually applies to you, but
>> those are
>> the expectations you set by using HTML email and then insulting all
>> the list members when someone asks you to stop. That's not the way to get
>> useful help from a technical peer support list.
>
> Fair enough and I had no intention of insulting anyone.  Being anal is not
> necessarily a bad thing. :)

I certainly wasn't offended.  It seemed like an honest question.

Re: HTML email (was Re: Vacuum and Reindex hangs

From
Steve Atkins
Date:
On Jan 15, 2009, at 1:02 PM, Scott Marlowe wrote:

> On Thu, Jan 15, 2009 at 1:56 PM, Jason Long
> <mailing.list@supernovasoftware.com> wrote:
>> Steve Atkins wrote:
>>>
>>> I'm sure none of that other than the last actually applies to you,
>>> but
>>> those are
>>> the expectations you set by using HTML email and then insulting all
>>> the list members when someone asks you to stop. That's not the way
>>> to get
>>> useful help from a technical peer support list.
>>
>> Fair enough and I had no intention of insulting anyone.  Being anal
>> is not
>> necessarily a bad thing. :)
>
> I certainly wasn't offended.  It seemed like an honest question.

I wasn't either. But while this is a less prickly environment than some
technical mailing lists, it's still pretty prickly, and there's a risk
some people
were offended by being described as anal (even if, or perhaps especially
when, it's an accurate description :) ).

I find HTML mail quite useful when I know that all the recipients can
read it - but it tends to harm communication when some of the recipients
can't. In the latter case I find plain text mail, possibly with attached
diagrams or embedded URLs to web docs more useful than inline
HTML. That's technical mailing lists, pretty much.

Cheers,
   Steve


Re: Vacuum and Reindex hangs

From
Jason Long
Date:
Scott Marlowe wrote:
On Thu, Jan 15, 2009 at 1:28 PM, Jason Long
<mailing.list@supernovasoftware.com> wrote: 
A faster server.
Well the sever is plenty fast.  It has 2 quad core 1600MHz FSB 3.0 GHz Xeon
5472 CPUs and a very light workload.   
A few things.

That doesn't make a fast server.  The disk i/o subsystem makes a fast
server.  And you've mentioned nothing of that yet.  If you've got a 16
SAS Disk RAID -10 array on a fast RAID controller with battery backed
cache, you've got a fast database server.

If you've got a single SATA drive or a mirror set of two SATA drives,
you do not have a fast database server. 
You got me.  I have a set of mirrored raptors.  I am not sure the disk i/o subsystem is a bottleneck.
The whole DB is 50 mb with minimal users.
Would a 16 SAS Disk RAID -10 really help me that much?
The dataset is small, but contains a complex data structure with many joins between tables.
I would appreciate any advice on the effect of a high end disk setup for my case.
 
My statement about the time is that it has never taken that long.  Ever.
Not even close.   
I wonder if you're getting a lot of bloating in your indexes from the
full vacuums.  Is there a reason you're running full vacuums over
regular vacuums?  While there are quite a few circumstances where full
vacuums are the right answer, most of the time they are not, at least
not on a regular basis. 
I used to use full vacuum and reindex ever night just before I did a dump backup.  Then I started to try the autovacuum.
The reason for the vacuum now it that I have a dynamic query that sometimes brings the server to a grinding halt.
This is why I set the timeout to 2 minutes.
Now all I get is users pissed about 2 times a day and the problem corrects itself before they finish complaining to me.

I tried to add some more indexes and then tried to vacuum full and reindex the database.
A lot of things can cause your current vacuums to run slow.  Maybe
there's a competing regular autovacuum that's kicked in at the same
time, someone is backing up the database, and so on. 
Autovacuum could be the case, but I have total control of the database an no backups are in progress.
As for :
 
However, this is about the most anal list ever.
I see so many emails on here about people complaining regarding the proper way to reply
or post to the list.   
That's because many of us receive hundreds of emails a week, and if
everyone starts sending html email,using bouncing email addresses, or
sending emails to 5 lists at once, things can get out of hand pretty
quickly.

Since your email agent is sending multi-part mime email with regular
text and html email, there's no real reason to complain, as any agent
worth its salt can be set to show only the text part.  I'm pretty sure
the email archive process also lops off the html part before storing
it. 
I totally understand and will limit my use of HTML in the future.
Busy lists tend to be anal.  Wanna get a bunch of people mad at once?
Break the rules on the lkml.  We're a bunch of fuzzy little kittens
playing with balls of yarn by comparison.  :) 

Re: Vacuum and Reindex hangs

From
"Scott Marlowe"
Date:
On Thu, Jan 15, 2009 at 2:24 PM, Jason Long
<mailing.list@supernovasoftware.com> wrote:
> Scott Marlowe wrote:

> You got me.  I have a set of mirrored raptors.  I am not sure the disk i/o
> subsystem is a bottleneck.
> The whole DB is 50 mb with minimal users.

Then you're only ever writing to the db, and 50Meg is teeny tiny.
Even my laptop can write out at 50Megs in about 5 seconds.

> Would a 16 SAS Disk RAID -10 really help me that much?

Depends on your usage pattern.  We use a 12 disk one with 15k5
seagates to handle a couple of 30G databases running 2000 to 5000
requests per minute, 97% or so being reads.

> The dataset is small, but contains a complex data structure with many joins
> between tables.
> I would appreciate any advice on the effect of a high end disk setup for my
> case.

Given how small your dataset is, a simple caching RAID controller
should offer enough throughput that you don't need more drives.

> I used to use full vacuum and reindex ever night just before I did a dump
> backup.  Then I started to try the autovacuum.
> The reason for the vacuum now it that I have a dynamic query that sometimes
> brings the server to a grinding halt.

You might be better served by a cluster command than a vacuum full.
It rewrites the table much like a vacuum full, but it's faster,
doesn't bloat the index, and results in a table who's order follows
that of the index you clustered on.  We have a large table that went
from 5 to 300 seconds to .5 to 3 seconds avg query speed because of a
cluster command.  Took 80 minutes to cluster the first time, but it
was well worth it.

HTML email (was: Re: Vacuum and Reindex hangs)

From
Ivan Sergio Borgonovo
Date:
On Thu, 15 Jan 2009 12:56:51 -0800
"Joshua D. Drake" <jd@commandprompt.com> wrote:

> I know many perfectly intelligent people that are better served
> through diagrams, pdf and color than a mailing list. Most of them
> make sure geeks like us, *EAT*.

> Does that mean they are not intelligent or perhaps that there
> talent set is just different?

Inspired by your mail I just wrote a post that explain why jerky
geeks may be fine esthetes but still they've good reason to
avoid HTML in emails.
I think I followed a slight different approach to the more
traditional ones.

http://www.webthatworks.it/d1/node/page/are_geeks_rich_media_impaired_or_why_html_emails_are_evil

--
Ivan Sergio Borgonovo
http://www.webthatworks.it