Thread: BUG in pg_autovacuum - with patch
Hi folks.
There's a problem in the console detach code for pg_autovacuum.
"setsid()" is NOT sufficient. Specifically, you must disassociate the
control terminal (standard in, out and error) for it to be complete.
This causes any attempt to execute pg_autovacuum from the cron to hang the
cron processor, stalling it.
Find below a diff that fixes the problem and allows proper disassociation:
*** pg_autovacuum.c.old Fri Apr 1 21:13:22 2005
--- pg_autovacuum.c Fri Apr 1 20:55:41 2005
***************
*** 196,201 ****
--- 196,202 ----
daemonize(void)
{
pid_t pid;
+ int i;
pid = fork();
if (pid == (pid_t) -1)
***************
*** 219,224 ****
--- 220,230 ----
fflush(LOGOUTPUT);
_exit(1);
}
+ i = open(NULL_DEV, O_RDWR);
+ dup2(i, 0);
+ dup2(i, 1);
+ dup2(i, 2);
+ close(i);
#endif
}
--
--
Karl Denninger (karl@denninger.net) Internet Consultant & Kids Rights Activist
http://www.denninger.net My home on the net - links to everything I do!
http://scubaforum.org Your UNCENSORED place to talk about DIVING!
http://www.spamcuda.net SPAM FREE mailboxes - FREE FOR A LIMITED TIME!
http://genesis3.blogspot.com Musings Of A Sentient Mind
Why would you run pg_autovacuum from cron?
Karl Denninger wrote:
>Hi folks.
>
>There's a problem in the console detach code for pg_autovacuum.
>
>"setsid()" is NOT sufficient. Specifically, you must disassociate the
>control terminal (standard in, out and error) for it to be complete.
>
>This causes any attempt to execute pg_autovacuum from the cron to hang the
>cron processor, stalling it.
>
>Find below a diff that fixes the problem and allows proper disassociation:
>
>
>*** pg_autovacuum.c.old Fri Apr 1 21:13:22 2005
>--- pg_autovacuum.c Fri Apr 1 20:55:41 2005
>***************
>*** 196,201 ****
>--- 196,202 ----
> daemonize(void)
> {
> pid_t pid;
>+ int i;
>
> pid = fork();
> if (pid == (pid_t) -1)
>***************
>*** 219,224 ****
>--- 220,230 ----
> fflush(LOGOUTPUT);
> _exit(1);
> }
>+ i = open(NULL_DEV, O_RDWR);
>+ dup2(i, 0);
>+ dup2(i, 1);
>+ dup2(i, 2);
>+ close(i);
> #endif
>
> }
>
>
>--
>
>
I have a process which does backups by mounting a disk into a RAID array, allowing it to sync, then it must stop the database before detaching it so as to insure that the DBMS is consistent on the backup disk. Once detached, Postgres must be restarted, of course...... This process is MUCH faster than dumping the disks to tape and results in a bootable backup volume - the latter is of great value for disaster recovery! -- -- Karl Denninger (karl@denninger.net) Internet Consultant & Kids Rights Activist http://www.denninger.net My home on the net - links to everything I do! http://scubaforum.org Your UNCENSORED place to talk about DIVING! http://www.spamcuda.net SPAM FREE mailboxes - FREE FOR A LIMITED TIME! http://genesis3.blogspot.com Musings Of A Sentient Mind On Sat, Apr 02, 2005 at 01:50:03AM -0500, Matthew T. O'Connor wrote: > Why would you run pg_autovacuum from cron? > > Karl Denninger wrote: > > >Hi folks. > > > >There's a problem in the console detach code for pg_autovacuum. > > > >"setsid()" is NOT sufficient. Specifically, you must disassociate the > >control terminal (standard in, out and error) for it to be complete. > > > >This causes any attempt to execute pg_autovacuum from the cron to hang the > >cron processor, stalling it. > > > >Find below a diff that fixes the problem and allows proper disassociation: > > > > > >*** pg_autovacuum.c.old Fri Apr 1 21:13:22 2005 > >--- pg_autovacuum.c Fri Apr 1 20:55:41 2005 > >*************** > >*** 196,201 **** > >--- 196,202 ---- > > daemonize(void) > > { > > pid_t pid; > >+ int i; > > > > pid = fork(); > > if (pid == (pid_t) -1) > >*************** > >*** 219,224 **** > >--- 220,230 ---- > > fflush(LOGOUTPUT); > > _exit(1); > > } > >+ i = open(NULL_DEV, O_RDWR); > >+ dup2(i, 0); > >+ dup2(i, 1); > >+ dup2(i, 2); > >+ close(i); > > #endif > > > > } > > > > > >-- > > > > > > > > %SPAMBLOCK-SYS: Matched [@postgresql.org], message ok
Karl Denninger wrote: >I have a process which does backups by mounting a disk into a RAID array, >allowing it to sync, then it must stop the database before detaching it so >as to insure that the DBMS is consistent on the backup disk. > >Once detached, Postgres must be restarted, of course...... > >This process is MUCH faster than dumping the disks to tape and results in a >bootable backup volume - the latter is of great value for disaster recovery! > > The problem is that if pg_autovacuum exits and then is relaunched, it doesn't remember any of it's state information from when it last exited. So if you are stopping and starting autovacuum one a day, it will be less effective. If you have some very active tables that autovacuum will vacuum several times a day then I can still see it's usefullness, but it's never going to vacuum a table that doesn't have enough activity to cause a vacuum in one day. Anyway, if pg_autovacuum is causing problems for cron I'm sure we would still benefit from this patch. However, while I claim no expertise related to detaching from the console, I will say that I copied the code detach code directly from postgresql itself, so I would have thought it was OK. Can someone more informed than I take a look at this patch? Thanks, Matthew
On Sat, Apr 02, 2005 at 12:16:00PM -0500, Matthew T. O'Connor wrote: > Karl Denninger wrote: > > >I have a process which does backups by mounting a disk into a RAID array, > >allowing it to sync, then it must stop the database before detaching it so > >as to insure that the DBMS is consistent on the backup disk. > > > >Once detached, Postgres must be restarted, of course...... > > > >This process is MUCH faster than dumping the disks to tape and results in a > >bootable backup volume - the latter is of great value for disaster recovery! > > > > > > The problem is that if pg_autovacuum exits and then is relaunched, it > doesn't remember any of it's state information from when it last > exited. So if you are stopping and starting autovacuum one a day, it > will be less effective. If you have some very active tables that > autovacuum will vacuum several times a day then I can still see it's > usefullness, but it's never going to vacuum a table that doesn't have > enough activity to cause a vacuum in one day. > > Anyway, if pg_autovacuum is causing problems for cron I'm sure we would > still benefit from this patch. However, while I claim no expertise > related to detaching from the console, I will say that I copied the code > detach code directly from postgresql itself, so I would have thought it > was OK. Can someone more informed than I take a look at this patch? > > Thanks, > > Matthew You left out the closeout of the stdio streams from the Postgresql code you copied. :-> -- -- Karl Denninger (karl@denninger.net) Internet Consultant & Kids Rights Activist http://www.denninger.net My home on the net - links to everything I do! http://scubaforum.org Your UNCENSORED place to talk about DIVING! http://www.spamcuda.net SPAM FREE mailboxes - FREE FOR A LIMITED TIME! http://genesis3.blogspot.com Musings Of A Sentient Mind
Karl Denninger <karl@denninger.net> writes:
> On Sat, Apr 02, 2005 at 12:16:00PM -0500, Matthew T. O'Connor wrote:
>> Anyway, if pg_autovacuum is causing problems for cron I'm sure we would
>> still benefit from this patch. However, while I claim no expertise
>> related to detaching from the console, I will say that I copied the code
>> detach code directly from postgresql itself, so I would have thought it
>> was OK. Can someone more informed than I take a look at this patch?
> You left out the closeout of the stdio streams from the Postgresql code you
> copied. :->
Indeed. Patch applied (along with addition of the #includes it
requires).
regards, tom lane