Thread: 7.1.2 temporary file area
I'm using 7.1.2 on a Solaris machine. The back end crashed
and now it won't come back up. The reason is that I have
a disk quota on the machine the server is running on, and that
disk quota is being exceeded. But I have 9 megs free in the account,
and it has been working fine.
Here is the error I get (I changed the names of the paths listed):
postmaster successfully started
DEBUG: database system was shut down at 2001-06-13 09:28:28 EST
DEBUG: CheckPoint record at (0, 12584784)
DEBUG: Redo record at (0, 12584784); Undo record at (0, 0); Shutdown TRUE
DEBUG: NextTransactionId: 2123; NextOid: 45698
FATAL 2: ZeroFill([a-path]/xlogtemp.7890) failed:
Disc quota exceeded : [path-to-executable] Startup proc 7890 exited
with status 512 - abort
I believe postgres is creating a large temporary file. How can I
tell is to put the files in another temporary directory where I
have more space?
Thanks,
--
----------------------------------------------------------------
Travis Bauer | CS Grad Student | IU |www.cs.indiana.edu/~trbauer
----------------------------------------------------------------
Travis Bauer writes: > Here is the error I get (I changed the names of the paths listed): > > postmaster successfully started > DEBUG: database system was shut down at 2001-06-13 09:28:28 EST > DEBUG: CheckPoint record at (0, 12584784) > DEBUG: Redo record at (0, 12584784); Undo record at (0, 0); Shutdown TRUE > DEBUG: NextTransactionId: 2123; NextOid: 45698 > FATAL 2: ZeroFill([a-path]/xlogtemp.7890) failed: > Disc quota exceeded : [path-to-executable] Startup proc 7890 exited > with status 512 - abort > > I believe postgres is creating a large temporary file. How can I > tell is to put the files in another temporary directory where I > have more space? You can't. This temporary file is the precursor of a transaction log file, so if your disk is full then you're out of luck anyway. Btw., we're talking about 16 MB here. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Travis Bauer <trbauer@indiana.edu> writes:
> FATAL 2: ZeroFill([a-path]/xlogtemp.7890) failed:
> Disc quota exceeded : [path-to-executable] Startup proc 7890 exited
> with status 512 - abort
> I believe postgres is creating a large temporary file.
That's not a temp file (at least not in our usual meaning of the term).
It's a WAL segment.
You could change $PGDATA/pg_xlog/ to be a symlink to some directory on a
different disk --- but shut down the postmaster first, and be sure to
copy the existing contents of the directory to the new location.
Current sources (7.2-to-be) also have provisions for relocating real
temp files in a similar fashion --- change the directory they live in
to be a symlink. But that won't help you today.
regards, tom lane
Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> Current sources (7.2-to-be) also have provisions for relocating real
>> temp files in a similar fashion --- change the directory they live in
>> to be a symlink. But that won't help you today.
> We have temp sort file in a separate directory. Did you put temp tables
> in there too?
No. I don't think we want to mess with that until we have tablespaces.
Then there should be a provision for designating the tablespace that
temp tables go into...
regards, tom lane
> Current sources (7.2-to-be) also have provisions for relocating real > temp files in a similar fashion --- change the directory they live in > to be a symlink. But that won't help you today. We have temp sort file in a separate directory. Did you put temp tables in there too? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
> Bruce Momjian <pgman@candle.pha.pa.us> writes: > >> Current sources (7.2-to-be) also have provisions for relocating real > >> temp files in a similar fashion --- change the directory they live in > >> to be a symlink. But that won't help you today. > > > We have temp sort file in a separate directory. Did you put temp tables > > in there too? > > No. I don't think we want to mess with that until we have tablespaces. > Then there should be a provision for designating the tablespace that > temp tables go into... Just checking. When you said "real temp files", I wasn't sure. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
I just ran into an interesting problem. My /var partition was full, and the culprit turned out to be the postgres log file, which was only evident through lsof. It was actually a deleted file which was being kept open by the running process. The interesting thing is that if I redirect the output of postmaster, then every connection has this file open. My environment is Apache/jserv/postgres. So in order to rotate the postgres log, I would have to 1) Shut apache down, 2) Shut jserv down 3) Shut postgres down rotate the logs, then bring them all back up again Is there a way to tell postgres to release the log file, or am I completely off base here? Dave
Dave Cramer writes: > Is there a way to tell postgres to release the log file, or am I completely > off base here? Pipe the output through the rotatelogs program from apache, or a similar program. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
On Wednesday 13 June 2001 13:20, Dave Cramer wrote: > The interesting thing is that if I redirect the output of postmaster, then > every connection has this file open. > Is there a way to tell postgres to release the log file, or am I completely > off base here? Use syslog. Syslog has many advantages: 1.) Easily rollable. 2.) Redirectable to various files and/or remote machines -- including hardcopy, and including multiple files at once (splitting). 3.) PostgreSQL 7.1 has excellent syslog support. 4.) You don't have to shut anything down to roll the logs. 5.) Thanks to item 2, you could have a single syslog machine accept the syslog output of multiple PostgreSQL servers, integrating the log into one, if you wish. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Mié 13 Jun 2001 22:53, Lamar Owen wrote:
> On Wednesday 13 June 2001 13:20, Dave Cramer wrote:
> > The interesting thing is that if I redirect the output of postmaster,
> > then every connection has this file open.
> >
> > Is there a way to tell postgres to release the log file, or am I
> > completely off base here?
>
> Use syslog. Syslog has many advantages:
> 1.) Easily rollable.
> 2.) Redirectable to various files and/or remote machines -- including
> hardcopy, and including multiple files at once (splitting).
> 3.) PostgreSQL 7.1 has excellent syslog support.
> 4.) You don't have to shut anything down to roll the logs.
> 5.) Thanks to item 2, you could have a single syslog machine accept the
> syslog output of multiple PostgreSQL servers, integrating the log into one,
> if you wish.
OK, if I enable syslog at configure time, how can I filter out the messages
that come from postgres? Something like what I have for mail going to
/var/log/maillog.
Is it posible?
If syslog is the best option, why shouldn't it come for default, and why
should there be a -l <file> option in pg_ctl?
If you satisfy my doubts, I'll recompile enabling syslog right now!
Saludos... :-)
--
Cualquiera administra un NT.
Ese es el problema, que cualquiera administre.
-----------------------------------------------------------------
Martin Marques | mmarques@unl.edu.ar
Programador, Administrador | Centro de Telematica
Universidad Nacional
del Litoral
-----------------------------------------------------------------
Lamar Owen writes: > Use syslog. Syslog has many advantages: > 1.) Easily rollable. > 2.) Redirectable to various files and/or remote machines -- including > hardcopy, and including multiple files at once (splitting). > 3.) PostgreSQL 7.1 has excellent syslog support. > 4.) You don't have to shut anything down to roll the logs. > 5.) Thanks to item 2, you could have a single syslog machine accept the > syslog output of multiple PostgreSQL servers, integrating the log into one, > if you wish. Disadvantage: Some messages will get lost. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
On Wednesday 13 June 2001 16:39, Peter Eisentraut wrote: > Lamar Owen writes: > > Use syslog. Syslog has many advantages: > Disadvantage: > Some messages will get lost. I have yet to see a 'lost' syslog message here, in over three years. Which is worse, though: 1.) An occasional lost log message; 2.) Downtime due to pulling down the entire server to rotate a log. If syslog looses messages, let's try helping fix syslog rather than recommending Yet Another Log Rotating Solution. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Wednesday 13 June 2001 10:38, Martín Marqués wrote: > OK, if I enable syslog at configure time, how can I filter out the messages > that come from postgres? Something like what I have for mail going to > /var/log/maillog. > Is it posible? Yes. In postgresql.conf you can assign the syslog facility to use, along with other good stuff. Then in /etc/syslog.conf you direct that facility to whereever you wish -- even splitting out FATAL and ERROR to the console, if you so desire. Then configure your log roller (such as the logrotate system Red Hat uses) to do whatever you want. > If syslog is the best option, why shouldn't it come for default, and why > should there be a -l <file> option in pg_ctl? > If you satisfy my doubts, I'll recompile enabling syslog right now! Multiple logging types is good. Syslog isn't available on all platforms PostgreSQL works on. But you _can_ configure and compile with syslog support _on_ and either not use it, or use it in conjunction with a regular log -- things are flexible. And flexible is good. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Wed, 13 Jun 2001, Lamar Owen wrote:
> On Wednesday 13 June 2001 16:39, Peter Eisentraut wrote:
> > Lamar Owen writes:
> > > Use syslog. Syslog has many advantages:
> > Disadvantage:
>
> > Some messages will get lost.
>
> I have yet to see a 'lost' syslog message here, in over three years.
I've actually seen lost and partial/mangled messages come out of syslog -
during a 7Mbit/second DDoS that was being logged due to ipchains ... -l.
Not that this happens _too_ often - we're talking 58k lines of log in a
very short amount of time.
(I agree with Lamar - it's just a "I've seen it happen" :)
--
Dominic J. Eidson
"Baruk Khazad! Khazad ai-menu!" - Gimli
-------------------------------------------------------------------------------
http://www.the-infinite.org/ http://www.the-infinite.org/~dominic/
Lamar Owen <lamar.owen@wgcr.org> writes:
> On Wednesday 13 June 2001 16:39, Peter Eisentraut wrote:
> > Lamar Owen writes:
> > > Use syslog. Syslog has many advantages:
> > Disadvantage:
>
> > Some messages will get lost.
>
> I have yet to see a 'lost' syslog message here, in over three years.
>
> Which is worse, though:
> 1.) An occasional lost log message;
> 2.) Downtime due to pulling down the entire server to rotate a log.
>
> If syslog looses messages, let's try helping fix syslog rather than
> recommending Yet Another Log Rotating Solution.
The implementation of syslog--routing messages through a named pipe
with a limited amount of buffer space--makes it inevitable that
messages will be lost under heavy load. Sometimes heavy load is when
you most need the messages.
Also, under heavy load, syslog can fill up your disk.
DJB's multilog program is a reliable alternative:
http://cr.yp.to/daemontools.html
Ian
On Wed, 13 Jun 2001, Lamar Owen wrote: > On Wednesday 13 June 2001 16:39, Peter Eisentraut wrote: > > Lamar Owen writes: > > > Use syslog. Syslog has many advantages: > > Disadvantage: > > > Some messages will get lost. > > I have yet to see a 'lost' syslog message here, in over three years. Hate to say it, but: Me too! > > Which is worse, though: > 1.) An occasional lost log message; > 2.) Downtime due to pulling down the entire server to rotate a log. > > If syslog looses messages, let's try helping fix syslog rather than > recommending Yet Another Log Rotating Solution. Absolutely. There are multiple syslogd's out there which use reliable means of transmitting syslog messages from one machine to another... -alex
On Wednesday 13 June 2001 17:45, Dominic J. Eidson wrote: > On Wed, 13 Jun 2001, Lamar Owen wrote: > > I have yet to see a 'lost' syslog message here, in over three years. > I've actually seen lost and partial/mangled messages come out of syslog - > during a 7Mbit/second DDoS that was being logged due to ipchains ... -l. Well, I guess my slow 1.544Mbps T1 isn't fast enough to cause my PIII-600 to croak under the load of 24x7 service. And I've seen the DDoS attacks as well -- which is one reason the packet filter logs from the cisco 2514 go to a dedicated host (that cannot be reached from the outside, thanks to NAT) that is also running snort. > Not that this happens _too_ often - we're talking 58k lines of log in a > very short amount of time. > (I agree with Lamar - it's just a "I've seen it happen" :) Well, I've also seen PostgreSQL 'lose' 15k tuples during a vacuum before (6.3.2 -- 7.0 apparently fixed the problem). Judicious placement of the logging hosts can prevent this lossage -- IOW, don't put all your eggs in one basket. Which is why my snort machine handles the heavy traffic logs -- it has a 27GB drive in it and does nothing else. And systems capable of doing that are not expensive -- I saw a P5 150 system on computersurplusoutlet.com for $39 US. I've seen alot of oddball things -- including a tape deck running by itself when not plugged in (in a 1000V/m RF field) -- but I've yet to see a dropped syslog message -- not that it can't or won't happen, but that it is unlikely. The suggestion to use DJB's multilog (by another poster) is a relatively good one -- but I am very wary of DJ's license -- when he's gone,the user of his software will be just plain stuck. I'd like to see the potential problems with syslog fixed, rather than another solution entirely. This is, after all, open source we're talking about, where a body can jump in to a project and help out any time one wishes :-). But make sure you block the UDP port syslog uses from coming in from the outside..... -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Lamar Owen writes: > On Wednesday 13 June 2001 16:39, Peter Eisentraut wrote: > > Lamar Owen writes: > > > Use syslog. Syslog has many advantages: > > Disadvantage: > > > Some messages will get lost. > > I have yet to see a 'lost' syslog message here, in over three years. I didn't mean sylog losing messages, but some PostgreSQL messages not getting there. > Which is worse, though: > 1.) An occasional lost log message; > 2.) Downtime due to pulling down the entire server to rotate a log. > > If syslog looses messages, let's try helping fix syslog rather than > recommending Yet Another Log Rotating Solution. YALRS is fine by me. Note that syslog requires root access, which we don't want to require. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
On Thursday 14 June 2001 10:47, Peter Eisentraut wrote: > Lamar Owen writes: > > I have yet to see a 'lost' syslog message here, in over three years. > I didn't mean sylog losing messages, but some PostgreSQL messages not > getting there. Then whose fault is that? Is it our syslog calling, or the receiving syslog not hearing? (Yes, I know that by default syslog uses UDP).... > > If syslog looses messages, let's try helping fix syslog rather than > > recommending Yet Another Log Rotating Solution. > YALRS is fine by me. Why reinvent the wheel? Particularly such a generic, OS function wheel as logging? > Note that syslog requires root access, which we > don't want to require. This is a tired mantra -- most PostgreSQL installations are being run by DBA's with either direct root access or a friendly sysadmin (the proof is that the RPMset probably has more users than the regular build, and installation of the RPMset requires root on most RPM-supported OSes). Besides, _recommending_ syslog != _requiring_ syslog -- the current build system for logging isn't broken and doesn't need fixing -- but rather than recommend a non-syslog solution out of hand, an even presentation of the options is more productive. We have good syslog support -- and we have good non-syslog support. Neither are really _great_, but both work. Neither should be removed, by any means, and neither should be required, either. Which means a consistent elog() usage is to be preferred over a mix of elog() and fputs/fprintf. For those that can't get either root access or a friendly enough sysadmin, a presentation of the options available to them is good as well. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
>>>>> "LO" == Lamar Owen <lamar.owen@wgcr.org> writes: LO> Then whose fault is that? Is it our syslog calling, or the LO> receiving syslog not hearing? (Yes, I know that by default syslog LO> uses UDP).... Not on FreeBSD (probably most other BSD's as well), the default is to use a named pipe. Anyhow what is the best way to make postgres stop stderr logging when syslog logging is turned on? Right now I use "-l /dev/null" to pg_ctl when starting the server. This seems to me that the server will still waste cycles doing the stderr printing. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D. Khera Communications, Inc. Internet: khera@kciLink.com Rockville, MD +1-240-453-8497 AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
On Vie 15 Jun 2001 21:08, you wrote:
> >>>>> "LO" == Lamar Owen <lamar.owen@wgcr.org> writes:
>
> LO> Then whose fault is that? Is it our syslog calling, or the
> LO> receiving syslog not hearing? (Yes, I know that by default syslog
> LO> uses UDP)....
>
> Not on FreeBSD (probably most other BSD's as well), the default is to
> use a named pipe.
>
> Anyhow what is the best way to make postgres stop stderr logging when
> syslog logging is turned on? Right now I use "-l /dev/null" to pg_ctl
> when starting the server. This seems to me that the server will still
> waste cycles doing the stderr printing.
Talking aout sql.log rotate, my rotates are quite strange, or give strange
results. After a rotate I have an sql file that has a binary part at the
begining, and after quite a few pages of ^@s the logs, properly, starts.
I'm using this script to rotate syslog, maillog, etc. but I'm having trble
with sql.log.
Saludos.... ;-)
It's rotating, but the file that is left to have he output of postgres has a
bunch of bianry data first, and then the log.
--
Cualquiera administra un NT.
Ese es el problema, que cualquiera administre.
-----------------------------------------------------------------
Martin Marques | mmarques@unl.edu.ar
Programador, Administrador | Centro de Telematica
Universidad Nacional
del Litoral
-----------------------------------------------------------------
Just so you don't think I am not listening. I am reading each post with great interest, waiting to see the outcome of this. FWIW, I have been using Redhat's logrotate facility, and am starting to see that it is flawed. I have always wondered how sites stayed up 24x7 Thanks for the excellent discussion Dave ----- Original Message ----- From: "Ian Lance Taylor" <ian@airs.com> To: "Lamar Owen" <lamar.owen@wgcr.org> Cc: "Peter Eisentraut" <peter_e@gmx.net>; "Dave Cramer" <Dave@micro-automation.net>; <pgsql-general@postgresql.org> Sent: Wednesday, June 13, 2001 8:39 PM Subject: Re: [GENERAL] Log files, how to rotate properly > Lamar Owen <lamar.owen@wgcr.org> writes: > > > On Wednesday 13 June 2001 16:39, Peter Eisentraut wrote: > > > Lamar Owen writes: > > > > Use syslog. Syslog has many advantages: > > > Disadvantage: > > > > > Some messages will get lost. > > > > I have yet to see a 'lost' syslog message here, in over three years. > > > > Which is worse, though: > > 1.) An occasional lost log message; > > 2.) Downtime due to pulling down the entire server to rotate a log. > > > > If syslog looses messages, let's try helping fix syslog rather than > > recommending Yet Another Log Rotating Solution. > > The implementation of syslog--routing messages through a named pipe > with a limited amount of buffer space--makes it inevitable that > messages will be lost under heavy load. Sometimes heavy load is when > you most need the messages. > > Also, under heavy load, syslog can fill up your disk. > > DJB's multilog program is a reliable alternative: > http://cr.yp.to/daemontools.html > > Ian > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html > >