Thread: server crash in very big transaction [postgresql 8.0beta1]
BEGIN; ... ... ... END; PANIC: invalid xlog record length 236052 server closed the connection unexpectedly This probably means the server terminated abnormally before or whileprocessing the request. The connection to the server was lost. Attempting reset: Failed. !>
姜维 <jiangwei_1976@yahoo.com.cn> writes:
> BEGIN;
> ...
> ...
> ...
> END;
> PANIC: invalid xlog record length 236052
> server closed the connection unexpectedly
This is quite unhelpful, if you're not going to show us what you did to
cause it.
regards, tom lane
On Sun, Aug 22, 2004 at 09:39:07AM +0800, ?????? wrote: > BEGIN; > ... > ... > ... > END; > > PANIC: invalid xlog record length 236052 Huh, so what kind of operations did you execute within the transaction? -- Alvaro Herrera (<alvherre[a]dcc.uchile.cl>) Voy a acabar con todos los humanos / con los humanos yo acabaré voy a acabar con todos / con todos los humanos acabaré (Bender)
Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
> On Sun, Aug 22, 2004 at 09:39:07AM +0800, ?????? wrote:
>> PANIC: invalid xlog record length 236052
> Huh, so what kind of operations did you execute within the transaction?
I found one possible explanation, though I don't know if it's the
submitter's problem or not. Create a SQL file that generates a whole
lot of subtransactions, like more than 16000. I used
begin;
create table foo(d1 int);
drop table foo;
savepoint x;
release x;
-- repeat above 2 lines 20000 times
commit;
First try gave me
WARNING: out of shared memory
ERROR: out of shared memory
HINT: You may need to increase max_locks_per_transaction.
WARNING: StartAbortedSubTransaction while in START state
ERROR: current transaction is aborted, commands ignored until end of transaction block
ERROR: current transaction is aborted, commands ignored until end of transaction block
... etc ...
which is fine except for the StartAbortedSubTransaction warning; that
may indicate a problem. (What do you think about it, Alvaro?)
I bumped up max_locks_per_transaction to 1000 and tried again, and got
psql:zzbig.sql:40004: PANIC: invalid xlog record length 80024
server closed the connection unexpectedly
What is happening of course is that more than 16K subtransaction IDs
won't fit in a commit record (since XLOG records have a 16-bit length
field). We're gonna have to rethink the representation of subxact
commit in XLOG.
regards, tom lane
On Tue, Aug 24, 2004 at 06:10:40PM -0400, Tom Lane wrote: > WARNING: out of shared memory > ERROR: out of shared memory > HINT: You may need to increase max_locks_per_transaction. > WARNING: StartAbortedSubTransaction while in START state > ERROR: current transaction is aborted, commands ignored until end of transaction block > ERROR: current transaction is aborted, commands ignored until end of transaction block > ... etc ... > > which is fine except for the StartAbortedSubTransaction warning; that > may indicate a problem. (What do you think about it, Alvaro?) I think the problem here is that we can't "safely" call StartAbortedSubTransaction when in TRANS_START state. This is because we could have some subsystems already initialized, and we will be initializing them again. For example, AtSubStart_Memory() will be called twice, which will lead to the loss of an (empty) memory context. It doesn't seem a big problem, only a memory leak. (We also lose a ResourceOwner, but since it doesn't own anything yet, it isn't a problem.) Fortunately I think we are good regarding other subsystems --- for example if we happened to do something with sinval.c list-of-lists, it could get out of sync with the transaction stack and it could get ugly. But we don't. I don't think we should get rid of the warning. It shows that there's a problem, but it's not critical. We could set a flag indicating that memory and resource owner are already initialized (another TRANS state? a static bool?), but I don't know if it's worth the trouble. -- Alvaro Herrera (<alvherre[a]dcc.uchile.cl>) "The West won the world not by the superiority of its ideas or values or religion but rather by its superiority in applying organized violence. Westerners often forget this fact, non-Westerners never do." (Samuel P. Huntington)
Re: server crash in very big transaction [postgresql 8.0beta1]
From
"ÿffffceÿffffac" "ÿffffbdÿffffaa"
Date:
--- Alvaro Herrera <alvherre@dcc.uchile.cl> wrote:
> On Sun, Aug 22, 2004 at 09:39:07AM +0800, ??????
> wrote:
> > BEGIN;
> > ...
> > ...
> > ...
> > END;
> >
> > PANIC: invalid xlog record length 236052
>
> Huh, so what kind of operations did you execute
> within the transaction?
>
> --
> Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
> Voy a acabar con todos los humanos / con los humanos
> yo acabar?> voy a acabar con todos / con todos los
humanos
> acabar?(Bender)
>
>
-------------------example 1--------------------
$ echo "BEGIN;" > backup.sql
$ pg_dump -o >> backup.sql
$ echo "END;" >> backup.sql
...
$ psql -f backup.sql
PANIC: invalid xlog record length 236052
----------------example 2 ------------------------
There are 1600 tables in database 'db1', I wrote a
pl/pgsql function "update_tables" like
"
FOR table IN SELECT relname FROM pg_class
LOOP
...
DROP INDEX ON ... ;
ALTER TABLE DROP CONSTRAINT ...;
...
CREATE INDEX xxx ON TABLE xxx;
...
ALTER TABLE xxx ADD PRIMARY KEY...
ALTER TABLE xxx ADD ...
...
END LOOP
...
"
$ select update_tables();
PANIC: invalid xlog record length 236052
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail
"ÿffffceÿffffac" "ÿffffbdÿffffaa" <jiangwei_1976@yahoo.com.cn> writes:
> --- Alvaro Herrera <alvherre@dcc.uchile.cl> wrote:
>> Huh, so what kind of operations did you execute
>> within the transaction?
> There are 1600 tables in database 'db1', I wrote a
> pl/pgsql function "update_tables" like
> "
> FOR table IN SELECT relname FROM pg_class
> LOOP
> ...
> DROP INDEX ON ... ;
> ALTER TABLE DROP CONSTRAINT ...;
> ...
> CREATE INDEX xxx ON TABLE xxx;
> ...
> ALTER TABLE xxx ADD PRIMARY KEY...
> ALTER TABLE xxx ADD ...
> ...
> END LOOP
Okay, so it was the number-of-deleted-files issue and not the
number-of-subtransactions issue. Still says we have to allow
commit records to be bigger than 64K ...
regards, tom lane
Is this fixed? --------------------------------------------------------------------------- Tom Lane wrote: > "ÿffffceÿffffac" "ÿffffbdÿffffaa" <jiangwei_1976@yahoo.com.cn> writes: > > --- Alvaro Herrera <alvherre@dcc.uchile.cl> wrote: > >> Huh, so what kind of operations did you execute > >> within the transaction? > > > There are 1600 tables in database 'db1', I wrote a > > pl/pgsql function "update_tables" like > > > " > > FOR table IN SELECT relname FROM pg_class > > LOOP > > ... > > DROP INDEX ON ... ; > > ALTER TABLE DROP CONSTRAINT ...; > > ... > > CREATE INDEX xxx ON TABLE xxx; > > ... > > ALTER TABLE xxx ADD PRIMARY KEY... > > ALTER TABLE xxx ADD ... > > ... > > END LOOP > > Okay, so it was the number-of-deleted-files issue and not the > number-of-subtransactions issue. Still says we have to allow > commit records to be bigger than 64K ... > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 8: explain analyze is your friend > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073