Thread: Why isn't stats_temp_directory automatically created?
Hi, log_directory is automatically created if not present when starting the database server. But, stats_temp_directory is not created. Why? ISTM that current behavior is undesirable. Is it worth making the patch which creates stats_temp_directory if not present? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Fujii Masao escreveu: > Is it worth making the patch which creates stats_temp_directory > if not present? > +1. -- Euler Taveira de Oliveira http://www.timbira.com/
Euler Taveira de Oliveira <euler@timbira.com> wrote: > Fujii Masao escreveu: > > Is it worth making the patch which creates stats_temp_directory > > if not present? > > > +1. +1, but AFAIK stats_temp_directory was designed to symlink to a RAM drive. Even if stats_temp_directory exists as a symbolic link, the destination directory might be lost in such a situation. If you try to make servers more robust, you might also need to consider broken symlinks. Regards, --- ITAGAKI Takahiro NTT Open Source Software Center
Hi, On Tue, Apr 14, 2009 at 10:26 PM, Euler Taveira de Oliveira <euler@timbira.com> wrote: > Fujii Masao escreveu: >> >> Is it worth making the patch which creates stats_temp_directory >> if not present? >> > +1. Here is the patch. This patch should be added to CommitFest-2009-First?, or committed before 8.4 release? The patch is very small, so I don't think that it'll block 8.4 release. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Attachment
Fujii Masao wrote: > Hi, > > On Tue, Apr 14, 2009 at 10:26 PM, Euler Taveira de Oliveira > <euler@timbira.com> wrote: >> Fujii Masao escreveu: >>> Is it worth making the patch which creates stats_temp_directory >>> if not present? >>> >> +1. > > Here is the patch. > > This patch should be added to CommitFest-2009-First?, > or committed before 8.4 release? The patch is very small, > so I don't think that it'll block 8.4 release. I think the fix should go into 8.4 - this is a fix for a new feature. However, a couple of comments: This does not take into account the effect of symlinks as mentioned by Itakagi Takahiro. I haven't looked at the details, but I don't think it would be that much more work to deal with it - and as he mentions, this is a very common usecase. Also, wouldn't it be better to isolate this to the first time when we try to create the file - then we don't have to export the symbol? //Magnus
Hi, On Wed, Apr 15, 2009 at 5:37 PM, Magnus Hagander <magnus@hagander.net> wrote: > This does not take into account the effect of symlinks as mentioned by > Itakagi Takahiro. I haven't looked at the details, but I don't think it > would be that much more work to deal with it - and as he mentions, this > is a very common usecase. Okey, I'll revise the patch; create also the directory which is referenced by symlink if not present. > Also, wouldn't it be better to isolate this to the first time when we > try to create the file - then we don't have to export the symbol? You mean having assign_pgstat_temp_directory() create the directory instead of pgstat_start()? In this case, the directory is created automatically not only at the beginning but also when a configuration file is reloaded. This seems to be better behavior. One problem of this is that some directories may be created unexpectedly if stats_temp_directory is specified at multiple locations. It's because assign_hook is called for each setting. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Fujii Masao wrote: > Hi, Hi! Sorry about the very late response - I've been out of the country and generally busy. > > On Wed, Apr 15, 2009 at 5:37 PM, Magnus Hagander <magnus@hagander.net> wrote: >> This does not take into account the effect of symlinks as mentioned by >> Itakagi Takahiro. I haven't looked at the details, but I don't think it >> would be that much more work to deal with it - and as he mentions, this >> is a very common usecase. > > Okey, I'll revise the patch; create also the directory which is > referenced by symlink if not present. Great. >> Also, wouldn't it be better to isolate this to the first time when we >> try to create the file - then we don't have to export the symbol? > > You mean having assign_pgstat_temp_directory() create the > directory instead of pgstat_start()? In this case, the directory is > created automatically not only at the beginning but also when > a configuration file is reloaded. This seems to be better behavior. No, I meant creating it when we open the file - in pgstat_write_statsfile(). //Magnus
Hi, On Mon, Apr 20, 2009 at 1:29 AM, Magnus Hagander <magnus@hagander.net> wrote: > Sorry about the very late response - I've been out of the country and > generally busy. Thanks for taking the time to comment! >> On Wed, Apr 15, 2009 at 5:37 PM, Magnus Hagander <magnus@hagander.net> wrote: >>> This does not take into account the effect of symlinks as mentioned by >>> Itakagi Takahiro. I haven't looked at the details, but I don't think it >>> would be that much more work to deal with it - and as he mentions, this >>> is a very common usecase. >> >> Okey, I'll revise the patch; create also the directory which is >> referenced by symlink if not present. > > Great. Here is the revised patch; If stats_temp_directory indicates the symlink, we pursue the chain of symlinks and create the referenced directory. >>> Also, wouldn't it be better to isolate this to the first time when we >>> try to create the file - then we don't have to export the symbol? >> >> You mean having assign_pgstat_temp_directory() create the >> directory instead of pgstat_start()? In this case, the directory is >> created automatically not only at the beginning but also when >> a configuration file is reloaded. This seems to be better behavior. > > No, I meant creating it when we open the file - in pgstat_write_statsfile(). OK, I changed the patch so. Thanks. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Attachment
Hi, On Tue, Apr 21, 2009 at 4:33 PM, Fujii Masao <masao.fujii@gmail.com> wrote: > Here is the revised patch; If stats_temp_directory indicates the symlink, > we pursue the chain of symlinks and create the referenced directory. BTW, this patch is useful also as the foundation for improving creation of log_directory. Attached patch fixes the following problem of log_directory by using that fundamental patch. - log_directory is not created when a configuration file is reloaded - creation of log_directory fails if the parent directory of it doesn't exist - if log_directory indicates the symlink, it's not resolved Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Attachment
Fujii Masao <masao.fujii@gmail.com> writes:
> Here is the revised patch; If stats_temp_directory indicates the symlink,
> we pursue the chain of symlinks and create the referenced directory.
I looked at this patch a bit. I'm still entirely unconvinced that we
should be doing this at all --- if the directory is not there, there's
a significant probability that there's something wrong that is beyond
the backend's ability to understand or correct. However, even ignoring
that objection, the patch is not ready to commit for a number of
reasons:
* Repeating the operation every time the stats file is written doesn't
seem like a particularly good idea; it eats cycles, and if the directory
disappears during live operation then there is *definitely* something
fishy going on. Can't we fix it so that the work is only done when the
path setting changes? (In principle you could do it in
assign_pgstat_temp_directory(), but I think something would be needed to
ensure that only the stats collector process actually tries to create
the directory. Or maybe it would be simplest to try to run the code only
when we get a failure from trying to create the stats temp file.)
* I don't think the mkdir_p code belongs in fd.c. It looks like
you copied-and-pasted it from initdb.c, which isn't any good either;
we don't want to maintain multiple copies of this. Maybe a new
src/port/ file is indicated.
* elog(LOG) is not exactly an adequate response if the final chdir fails
--- you have just broken the process beyond recovery. That alone may be
sufficient reason to reject the attempt to deal with symlinks. As far
as pgstat_temp_directory is concerned, I'm not sure of the point of
making the GUC point to a symlink anyway --- if you have a GUC why not
just point it where you want the directory to be?
regards, tom lane
Tom Lane wrote: > Fujii Masao <masao.fujii@gmail.com> writes: >> Here is the revised patch; If stats_temp_directory indicates the symlink, >> we pursue the chain of symlinks and create the referenced directory. > > I looked at this patch a bit. I'm still entirely unconvinced that we > should be doing this at all --- if the directory is not there, there's > a significant probability that there's something wrong that is beyond > the backend's ability to understand or correct. However, even ignoring > that objection, the patch is not ready to commit for a number of > reasons: > > * Repeating the operation every time the stats file is written doesn't > seem like a particularly good idea; it eats cycles, and if the directory > disappears during live operation then there is *definitely* something > fishy going on. Can't we fix it so that the work is only done when the > path setting changes? (In principle you could do it in > assign_pgstat_temp_directory(), but I think something would be needed to > ensure that only the stats collector process actually tries to create > the directory. Or maybe it would be simplest to try to run the code only > when we get a failure from trying to create the stats temp file.) My idea was to have it run when it tries to create the temp file and fails. Seems simpler than in the assign hook. > * I don't think the mkdir_p code belongs in fd.c. It looks like > you copied-and-pasted it from initdb.c, which isn't any good either; > we don't want to maintain multiple copies of this. Maybe a new > src/port/ file is indicated. Yes, that's what I thought as well. How about src/port/dirmod.c though - it has rename/unlink/symlink now which seem at least remotely related. //Magnus
I assume we decided we didn't want this. --------------------------------------------------------------------------- Tom Lane wrote: > Fujii Masao <masao.fujii@gmail.com> writes: > > Here is the revised patch; If stats_temp_directory indicates the symlink, > > we pursue the chain of symlinks and create the referenced directory. > > I looked at this patch a bit. I'm still entirely unconvinced that we > should be doing this at all --- if the directory is not there, there's > a significant probability that there's something wrong that is beyond > the backend's ability to understand or correct. However, even ignoring > that objection, the patch is not ready to commit for a number of > reasons: > > * Repeating the operation every time the stats file is written doesn't > seem like a particularly good idea; it eats cycles, and if the directory > disappears during live operation then there is *definitely* something > fishy going on. Can't we fix it so that the work is only done when the > path setting changes? (In principle you could do it in > assign_pgstat_temp_directory(), but I think something would be needed to > ensure that only the stats collector process actually tries to create > the directory. Or maybe it would be simplest to try to run the code only > when we get a failure from trying to create the stats temp file.) > > * I don't think the mkdir_p code belongs in fd.c. It looks like > you copied-and-pasted it from initdb.c, which isn't any good either; > we don't want to maintain multiple copies of this. Maybe a new > src/port/ file is indicated. > > * elog(LOG) is not exactly an adequate response if the final chdir fails > --- you have just broken the process beyond recovery. That alone may be > sufficient reason to reject the attempt to deal with symlinks. As far > as pgstat_temp_directory is concerned, I'm not sure of the point of > making the GUC point to a symlink anyway --- if you have a GUC why not > just point it where you want the directory to be? > > regards, tom lane > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.comPG East: http://www.enterprisedb.com/community/nav-pg-east-2010.do + If your life is a hard drive,Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> I assume we decided we didn't want this.
I thought the risk/reward ratio was pretty bad.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > I assume we decided we didn't want this. > > I thought the risk/reward ratio was pretty bad. Yea, the symlink issue killed it for me. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.comPG East: http://www.enterprisedb.com/community/nav-pg-east-2010.do + If your life is a hard drive,Christ can be your backup. +