Thread: Deadlock on transaction
Hi list,
I am curious becouse I am seen my postgresql locking the transactions when i load a huge (in some case not to huge) numbers of records 70,000 records.
Could someone help me with it ?
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Atenciosamente (Sincerely)
Ezequias Rodrigues da Rocha
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
A pior das democracias ainda é melhor do que a melhor das ditaduras
The worst of democracies is still better than the better of dictatorships
http://ezequiasrocha.blogspot.com/
I am curious becouse I am seen my postgresql locking the transactions when i load a huge (in some case not to huge) numbers of records 70,000 records.
Could someone help me with it ?
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Atenciosamente (Sincerely)
Ezequias Rodrigues da Rocha
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
A pior das democracias ainda é melhor do que a melhor das ditaduras
The worst of democracies is still better than the better of dictatorships
http://ezequiasrocha.blogspot.com/
On Mon, 2007-02-12 at 11:43, Ezequias Rodrigues da Rocha wrote: > Hi list, > > I am curious becouse I am seen my postgresql locking the transactions > when i load a huge (in some case not to huge) numbers of records > 70,000 records. > > Could someone help me with it ? Are sure that what you're speaking of is a deadlock? Or is it just a stalled transaction while one one transaction waits for another to complete? A true deadlock means that two transactions are stalled waiting for the other to complete, and neither ever can complete, and they are DEADlocked. If you've just got a problem where an update is stalling all the other transactions, then give us a test case / example code of what's happening. If you've got a real deadlock that postgresql isn't catching, that's a much more serious problem.
I mean really deadlock. Other transactions can't access the database until the main transaction is complete. A question:
PostgreSQL doesn't permit multiple transactions concurrently ?
Ezequias
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Atenciosamente (Sincerely)
Ezequias Rodrigues da Rocha
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
A pior das democracias ainda é melhor do que a melhor das ditaduras
The worst of democracies is still better than the better of dictatorships
http://ezequiasrocha.blogspot.com/
PostgreSQL doesn't permit multiple transactions concurrently ?
Ezequias
2007/2/12, Scott Marlowe <smarlowe@g2switchworks.com>:
On Mon, 2007-02-12 at 11:43, Ezequias Rodrigues da Rocha wrote:
> Hi list,
>
> I am curious becouse I am seen my postgresql locking the transactions
> when i load a huge (in some case not to huge) numbers of records
> 70,000 records.
>
> Could someone help me with it ?
Are sure that what you're speaking of is a deadlock? Or is it just a
stalled transaction while one one transaction waits for another to
complete?
A true deadlock means that two transactions are stalled waiting for the
other to complete, and neither ever can complete, and they are
DEADlocked.
If you've just got a problem where an update is stalling all the other
transactions, then give us a test case / example code of what's
happening.
If you've got a real deadlock that postgresql isn't catching, that's a
much more serious problem.
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Atenciosamente (Sincerely)
Ezequias Rodrigues da Rocha
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
A pior das democracias ainda é melhor do que a melhor das ditaduras
The worst of democracies is still better than the better of dictatorships
http://ezequiasrocha.blogspot.com/
Ezequias Rodrigues da Rocha wrote: > I mean really deadlock. Other transactions can't access the database until > the main transaction is complete. A question: > > PostgreSQL doesn't permit multiple transactions concurrently ? PG has quite good concurrency behaviour. And "can't access the database" isn't anything to do with locking - clients should still be able to log in. We'll need more details: 1. How do you know this is a deadlock? You haven't shown us what's in pg_locks, but I assume you've identified the problem there. 2. What data are you loading, to what tables? 3. Are there any foreign-keys or triggers we would need to know about? -- Richard Huxton Archonet Ltd
Ezequias Rodrigues da Rocha <ezequias.rocha@gmail.com> schrieb: > I mean really deadlock. Other transactions can't access the database until the > main transaction is complete. A question: That's not true. > > PostgreSQL doesn't permit multiple transactions concurrently ? Why not? Show us a complete case. PG works fine with many users on many connections and many transactions, at the same time. > > Ezequias > > 2007/2/12, Scott Marlowe <_s_m_a_r_l_o_w_e_@_g_2_s_w_i_t_c_h_w_o_r_k_s_._c_o_m>: Please, no silly fullquote betlow the answer and no html. -- Really, I'm not out to destroy Microsoft. That will just be a completely unintentional side effect. (Linus Torvalds) "If I was god, I would recompile penguin with --enable-fly." (unknow) Kaufbach, Saxony, Germany, Europe. N 51.05082°, E 13.56889°
2007/2/12, Richard Huxton <dev@archonet.com>:
I knew becouse my Delphi application reports it to me. If there is anything in pg_log I could see it to you.
Simple data but allways with the same username (does it make any difference ?)
No.
Ps: When I do not use transactions the connections does not lock to other users.
Ezequias Rodrigues da Rocha wrote:
> I mean really deadlock. Other transactions can't access the database until
> the main transaction is complete. A question:
>
> PostgreSQL doesn't permit multiple transactions concurrently ?
PG has quite good concurrency behaviour. And "can't access the database"
isn't anything to do with locking - clients should still be able to log in.
We'll need more details:
1. How do you know this is a deadlock? You haven't shown us what's in
pg_locks, but I assume you've identified the problem there.
I knew becouse my Delphi application reports it to me. If there is anything in pg_log I could see it to you.
2. What data are you loading, to what tables?
Simple data but allways with the same username (does it make any difference ?)
3. Are there any foreign-keys or triggers we would need to know about?
No.
Ps: When I do not use transactions the connections does not lock to other users.
--
Richard Huxton
Archonet Ltd
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Atenciosamente (Sincerely)
Ezequias Rodrigues da Rocha
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
A pior das democracias ainda é melhor do que a melhor das ditaduras
The worst of democracies is still better than the better of dictatorships
http://ezequiasrocha.blogspot.com/
Ezequias Rodrigues da Rocha wrote: > 2007/2/12, Richard Huxton <dev@archonet.com>: >> >> Ezequias Rodrigues da Rocha wrote: >> > I mean really deadlock. Other transactions can't access the database >> until >> > the main transaction is complete. A question: >> > >> > PostgreSQL doesn't permit multiple transactions concurrently ? >> >> PG has quite good concurrency behaviour. And "can't access the database" >> isn't anything to do with locking - clients should still be able to log >> in. >> >> We'll need more details: >> 1. How do you know this is a deadlock? You haven't shown us what's in >> pg_locks, but I assume you've identified the problem there. > > > I knew becouse my Delphi application reports it to me. If there is anything > in pg_log I could see it to you. You still aren't showing us the deadlock in pg_locks. How does your Delphi application decide there is a deadlock? I can see how it might know a timeout has occurred, but not how it determines what any other connections to the database might be doing. You do know the difference between just waiting for locks and having a deadlock? http://en.wikipedia.org/wiki/Deadlock > 2. What data are you loading, to what tables? > > Simple data but allways with the same username (does it make any difference > ?) > > 3. Are there any foreign-keys or triggers we would need to know about? > > No. Then you don't have a deadlock. Without multiple processes trying to take the same locks in different orders you don't get deadlock. > Ps: When I do not use transactions the connections does not lock to other > users. You are always using transactions. There is no out-of-transaction way to execute a statement in PostgreSQL. You can't lock either a connection or users, just rows. OK - I think you'd better give a full example. Can you show us: 1. What query or queries you execute to load the data? 2. What query or queries you think are causing "deadlock"? 3. The order they execute in and what error you get. -- Richard Huxton Archonet Ltd
This is delphi. I don't intent you understand but the sql actions are quite simple (I am reading a list of numbers).
If not dm.database1.InTransaction then
dm.database1.StartTransaction ;
For i:= 0 to memo1.Lines.Count - 1 do
Begin
// Catching ID
dm.qQ1.SQL.text:= Concat('select id from base.cartao where numero = ', memo1.lines[i]);
dm.qQ1.open;
cartaoID:= dm.qQ1.fieldByName('id').asString;
// Cathing the ticket ID
dm.qQ1.SQL.clear;
dm.qQ1.SQL.text:= Concat('select nextval(',QuotedStr(' base.ingresso_id') , ')');
dm.qQ1.open;
IngressoID:= dm.qQ1.fieldByName('nextval').asString;
// $$$$$$$$$$ Recording Tickets $$$$$$$$$$
dm.Qq1.sql.text:= Concat('Insert into base.ingresso values (' , QuotedStr(IngressoID), ',' , EstadoID, ',' , 'now()', ',' , valor, ',' , valor_promotor, ',' , AssentoID, ',', CaixaID, ',' , CartaoID, ',' , PromocaoID, ',' , SessaoID, ',' , VendedorID, ')');
// Showmessage(dm.Qq1.sql.text);
dm.Qq1.execSQL;
// ########### Recording Tickets ###########
// Pegando o Id do Bilhete no PostgreSQL
dm.qQ1.SQL.clear;
dm.qQ1.SQL.text:= Concat('select nextval(',QuotedStr('base.bilhete_id') , ')');
dm.qQ1.open;
BilheteID:= dm.qQ1.fieldByName ('nextval').asString;
dm.qQ1.SQL.clear;
dm.qQ1.SQL.add(Concat('Insert into base.bilhete (id, estado, uso_sequencia, promocao_documento, assento_id, cartao_id, ingresso_id, promocao_id, sessao_id, vendedor_venda_id )'));
dm.qQ1.SQL.add(Concat('values(', BilheteID, ',' , QuotedStr(EstadoID), ',' , '0,0', ',' , QuotedStr(AssentoID), ',' , QuotedStr(CartaoID), ',' , QuotedStr(IngressoID) , ',' , QuotedStr(PromocaoID), ',' , QuotedStr(SessaoID) , ',' , QuotedStr(VendedorID), ')' ) );
dm.qQ1.execSQL;
end;
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Atenciosamente (Sincerely)
Ezequias Rodrigues da Rocha
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
A pior das democracias ainda é melhor do que a melhor das ditaduras
The worst of democracies is still better than the better of dictatorships
http://ezequiasrocha.blogspot.com/
If not dm.database1.InTransaction then
dm.database1.StartTransaction ;
For i:= 0 to memo1.Lines.Count - 1 do
Begin
// Catching ID
dm.qQ1.SQL.text:= Concat('select id from base.cartao where numero = ', memo1.lines[i]);
dm.qQ1.open;
cartaoID:= dm.qQ1.fieldByName('id').asString;
// Cathing the ticket ID
dm.qQ1.SQL.clear;
dm.qQ1.SQL.text:= Concat('select nextval(',QuotedStr(' base.ingresso_id') , ')');
dm.qQ1.open;
IngressoID:= dm.qQ1.fieldByName('nextval').asString;
// $$$$$$$$$$ Recording Tickets $$$$$$$$$$
dm.Qq1.sql.text:= Concat('Insert into base.ingresso values (' , QuotedStr(IngressoID), ',' , EstadoID, ',' , 'now()', ',' , valor, ',' , valor_promotor, ',' , AssentoID, ',', CaixaID, ',' , CartaoID, ',' , PromocaoID, ',' , SessaoID, ',' , VendedorID, ')');
// Showmessage(dm.Qq1.sql.text);
dm.Qq1.execSQL;
// ########### Recording Tickets ###########
// Pegando o Id do Bilhete no PostgreSQL
dm.qQ1.SQL.clear;
dm.qQ1.SQL.text:= Concat('select nextval(',QuotedStr('base.bilhete_id') , ')');
dm.qQ1.open;
BilheteID:= dm.qQ1.fieldByName ('nextval').asString;
dm.qQ1.SQL.clear;
dm.qQ1.SQL.add(Concat('Insert into base.bilhete (id, estado, uso_sequencia, promocao_documento, assento_id, cartao_id, ingresso_id, promocao_id, sessao_id, vendedor_venda_id )'));
dm.qQ1.SQL.add(Concat('values(', BilheteID, ',' , QuotedStr(EstadoID), ',' , '0,0', ',' , QuotedStr(AssentoID), ',' , QuotedStr(CartaoID), ',' , QuotedStr(IngressoID) , ',' , QuotedStr(PromocaoID), ',' , QuotedStr(SessaoID) , ',' , QuotedStr(VendedorID), ')' ) );
dm.qQ1.execSQL;
end;
2007/2/12, Richard Huxton <dev@archonet.com>:
Ezequias Rodrigues da Rocha wrote:
> 2007/2/12, Richard Huxton <dev@archonet.com>:
>>
>> Ezequias Rodrigues da Rocha wrote:
>> > I mean really deadlock. Other transactions can't access the database
>> until
>> > the main transaction is complete. A question:
>> >
>> > PostgreSQL doesn't permit multiple transactions concurrently ?
>>
>> PG has quite good concurrency behaviour. And "can't access the database"
>> isn't anything to do with locking - clients should still be able to log
>> in.
>>
>> We'll need more details:
>> 1. How do you know this is a deadlock? You haven't shown us what's in
>> pg_locks, but I assume you've identified the problem there.
>
>
> I knew becouse my Delphi application reports it to me. If there is anything
> in pg_log I could see it to you.
You still aren't showing us the deadlock in pg_locks. How does your
Delphi application decide there is a deadlock? I can see how it might
know a timeout has occurred, but not how it determines what any other
connections to the database might be doing. You do know the difference
between just waiting for locks and having a deadlock?
http://en.wikipedia.org/wiki/Deadlock
> 2. What data are you loading, to what tables?
>
> Simple data but allways with the same username (does it make any difference
> ?)
>
> 3. Are there any foreign-keys or triggers we would need to know about?
>
> No.
Then you don't have a deadlock. Without multiple processes trying to
take the same locks in different orders you don't get deadlock.
> Ps: When I do not use transactions the connections does not lock to other
> users.
You are always using transactions. There is no out-of-transaction way to
execute a statement in PostgreSQL. You can't lock either a connection or
users, just rows.
OK - I think you'd better give a full example. Can you show us:
1. What query or queries you execute to load the data?
2. What query or queries you think are causing "deadlock"?
3. The order they execute in and what error you get.
--
Richard Huxton
Archonet Ltd
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Atenciosamente (Sincerely)
Ezequias Rodrigues da Rocha
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
A pior das democracias ainda é melhor do que a melhor das ditaduras
The worst of democracies is still better than the better of dictatorships
http://ezequiasrocha.blogspot.com/
On Mon, 2007-02-12 at 12:08, Ezequias Rodrigues da Rocha wrote: > I mean really deadlock. Other transactions can't access the database > until the main transaction is complete. A question: > > PostgreSQL doesn't permit multiple transactions concurrently ? Again, that's not a deadlock. A deadlock would mean that the transaction causing the lock never returned, because it wound up waiting for another transaction that was in turn waiting on it. i.e. begin transaction a begin trasnaction b a does something that waits on b b does something that waits on a neither transaction can complete. What you have is a blocking transaction. In postgresql, very few transactions tend to block other transactions. Have you got a test case that demonstrates your problem with blocking?
On Mon, Feb 12, 2007 at 03:08:27PM -0300, Ezequias Rodrigues da Rocha wrote: > I mean really deadlock. Other transactions can't access the database until > the main transaction is complete. What you describe sounds like a blocking lock: one transaction uses a resource and another has to wait until it is not locked anymore A deadlock is a special case: two transactions both need resources A and B. transaction 1 locks A, and the next moment transaction 2 locks B. Now transaction 1 waits for 2 to release B, but at the same time 2 waits for 1 to release A. This is called a deadlock, or circulair lock. Postgresql automatically detects such a situation and rolls one of them back. The client recieves a 'restartable database error' and that is exactly true: ususally another try will work. One method of fixing deadlocks is to make sure all your code accesses the database in the same order. Alphabetically is easiest to remember... -- __________________________________________________ "Nothing is as subjective as reality" Reinoud van Leeuwen reinoud.v@n.leeuwen.net http://www.xs4all.nl/~reinoud __________________________________________________
Ok you teach me. Thank you and sorry. :(
I just want to know why is it occuring. I am pretty sad.
Ezequias
I just want to know why is it occuring. I am pretty sad.
Ezequias
2007/2/12, Reinoud van Leeuwen < reinoud.v@n.leeuwen.net>:
On Mon, Feb 12, 2007 at 03:08:27PM -0300, Ezequias Rodrigues da Rocha wrote:
> I mean really deadlock. Other transactions can't access the database until
> the main transaction is complete.
What you describe sounds like a blocking lock: one transaction uses a
resource and another has to wait until it is not locked anymore
A deadlock is a special case: two transactions both need resources A and
B. transaction 1 locks A, and the next moment transaction 2 locks B. Now
transaction 1 waits for 2 to release B, but at the same time 2 waits for 1
to release A. This is called a deadlock, or circulair lock.
Postgresql automatically detects such a situation and rolls one of them
back. The client recieves a 'restartable database error' and that is
exactly true: ususally another try will work.
One method of fixing deadlocks is to make sure all your code accesses the
database in the same order. Alphabetically is easiest to remember...
Ezequias Rodrigues da Rocha wrote: > This is delphi. I don't intent you understand but the sql actions are quite > simple (I am reading a list of numbers). Well, let's see - the last Pascal I did was in 1986 I think... > If not dm.database1.InTransaction then > dm.database1.StartTransaction; > > For i:= 0 to memo1.Lines.Count - 1 do Loop through your import. > Begin > // Catching ID > dm.qQ1.SQL.text:= Concat('select id from base.cartao where > numero = ', memo1.lines[i]); > dm.qQ1.open; > cartaoID:= dm.qQ1.fieldByName('id').asString; > // Cathing the ticket ID OK, you're fetching ticket IDs one at a time. > dm.qQ1.SQL.clear; > dm.qQ1.SQL.text:= Concat('select nextval(',QuotedStr(' > base.ingresso_id') , ')'); > dm.qQ1.open; > IngressoID:= dm.qQ1.fieldByName('nextval').asString; And now you've generated a new ID from a sequence. > // $$$$$$$$$$ Recording Tickets $$$$$$$$$$ > dm.Qq1.sql.text:= Concat('Insert into base.ingresso values (' > , QuotedStr(IngressoID), ',' , EstadoID, ',' , 'now()', ',' , valor, ',' , > valor_promotor, ',' , AssentoID, ',', CaixaID, ',' , CartaoID, ',' , > PromocaoID, ',' , SessaoID, ',' , VendedorID, ')'); > // Showmessage(dm.Qq1.sql.text); > dm.Qq1.execSQL; Insert one row into "ingresso". > // ########### Recording Tickets ########### > > // Pegando o Id do Bilhete no PostgreSQL > dm.qQ1.SQL.clear; > dm.qQ1.SQL.text:= Concat('select nextval(',QuotedStr(' > base.bilhete_id') , ')'); > dm.qQ1.open; > BilheteID:= dm.qQ1.fieldByName('nextval').asString; > > dm.qQ1.SQL.clear; > dm.qQ1.SQL.add(Concat('Insert into base.bilhete (id, estado, > uso_sequencia, promocao_documento, assento_id, cartao_id, ingresso_id, > promocao_id, sessao_id, vendedor_venda_id )')); > dm.qQ1.SQL.add(Concat('values(', BilheteID, ',' , > QuotedStr(EstadoID), ',' , '0,0', ',' , QuotedStr(AssentoID), ',' , > QuotedStr(CartaoID), ',' , QuotedStr(IngressoID) , ',' , > QuotedStr(PromocaoID), ',' , QuotedStr(SessaoID) , ',' , > QuotedStr(VendedorID), ')' ) ); > > dm.qQ1.execSQL; Insert a row into "bilhete". > > end; I don't see anything like "dm.database1.CommitTransaction" - are you sure you are committing after the inserts? One other point - if Delphi doesn't offer something like it, you might find it useful to write a function that makes it easier to build queries: my_db_func('INSERT INTO foo (id,a,b,c) VALUES (?,?,?,?)', 'NUM,TEXT,DATE,DATE', variable1,variable2,variable3...); I'd be surprised if there wasn't something like that already though. -- Richard Huxton Archonet Ltd