Thread: Re: [HACKERS] What can we learn from MySQL?

Re: [HACKERS] What can we learn from MySQL?

From
Bruce Momjian
Date:
D'Arcy J.M. Cain wrote:
> On Fri, 23 Apr 2004 11:07:20 -0400
> Dave Cramer <pg@fastcrypt.com> wrote:
> > Does the current implementation of pg_autovacuum have a way of setting
> > windows where it is allowed to vacuum? Many large 24/7 will only allow
> > vacuumming at certain times of the day.
>
> It seems to me that the point of pg_autovacuum would be to run 24/7 so
> that there is never big hit on the system.  Perhaps it could be designed
> to throttle itself based on current system usage though.

Or the number of connected backends, or both?

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: [HACKERS] What can we learn from MySQL?

From
Christopher Browne
Date:
Martha Stewart called it a Good Thing when pgman@candle.pha.pa.us (Bruce Momjian) wrote:
> D'Arcy J.M. Cain wrote:
>> On Fri, 23 Apr 2004 11:07:20 -0400
>> Dave Cramer <pg@fastcrypt.com> wrote:
>> > Does the current implementation of pg_autovacuum have a way of setting
>> > windows where it is allowed to vacuum? Many large 24/7 will only allow
>> > vacuumming at certain times of the day.
>>
>> It seems to me that the point of pg_autovacuum would be to run 24/7 so
>> that there is never big hit on the system.  Perhaps it could be designed
>> to throttle itself based on current system usage though.
>
> Or the number of connected backends, or both?

This is what suggests to me the possibility that perhaps a good
answer would be to redo it in some scripting language, and have the
capability to either:
 a) Configure multiple targets via some language-specific approach
    (e.g. - reading Perl data structures, or some such thing) or
 b) Configure multiple targets via having the configuration stored
    in one database/schema.

I would somewhat favor the latter.

The initial design was set up jointly with a view to
 a) minimizing the number of extra dependancies, and
 b) ultimately being a stop-gap measure until a _proper_ scheme
    could get integrated into the postmaster.

The existing implementation has remained sufficiently fragile that we
have a hard time trusting it with the _important_ systems, and since
those systems tend to involve multiple backends, it sure would be nice
to have something that could get run across ALL of them, where we
could be confident that it wouldn't demolish I/O at inconvenient times
by trying to simultaneously vacuum huge tables on multiple backends.

I have lately been working on some (not quite yet sufficiently
generic) tools for managing sets of replication instances; I think I
may want to take a similar "tack" on this.  pg_autovacuum has been
handy for systems that I _don't_ want to pay much attention to, but it
hasn't been totally adequate for the more vital ones.
--
If this was helpful, <http://svcs.affero.net/rm.php?r=cbbrowne> rate me
http://cbbrowne.com/info/linuxdistributions.html
A real  patriot is the fellow  who gets a parking  ticket and rejoices
that the system works.

Re: [HACKERS] What can we learn from MySQL?

From
"D'Arcy J.M. Cain"
Date:
On Fri, 23 Apr 2004 13:08:30 -0400 (EDT)
Bruce Momjian <pgman@candle.pha.pa.us> wrote:
> D'Arcy J.M. Cain wrote:
> > It seems to me that the point of pg_autovacuum would be to run 24/7
> > so that there is never big hit on the system.  Perhaps it could be
> > designed to throttle itself based on current system usage though.
>
> Or the number of connected backends, or both?

I am sure that there are lots of ways to guage.  Not sure which is best
but I am sure that the smart people here will figure it out.  The
important thing, I think, is to let the engine make the decision
dynamically.  Personally I don't have a "quiet time" per se but there
are random quiet periods.  Something that jumps into the fray at those
points would be really nicwe.

--
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.