Thread: MVCC works in serialized mode!
CVS is just updated... Please try concurrent writes/reads... It'll take some time to implement READ COMMITTED mode... Note: 1. Vacuum is not updated yet and so will remove deleted tuples though some xactions would like to see them in selects... 2. LOCK TABLE modes are not yet implemented... 3. As well as SELECT ... FOR UPDATE: I need in advice here! Do Oracle, Informix etc support SELECT FOR UPDATE if there are aggregates, group by, distinct in SELECT ??? Is SELECTFOR UPDATE allowed in subqueries ??? 4. Rtree & gist indices use _relation level_ locking - no wish, no time to change them... Executor locks them in exclusivemode (on relation level) for update/insert and so prevents deadlocks... These locks released after _query_ isdone - not so bad for concurrency... 5. Hash & btree indices use _page level_ locking and so are "opened" for deadlocks -:)) (In the case of concurrent inserts/updateswhen indices are used for scans). I hope to change btree - it's possible, - but not hashes... 6. I'm not happy with current deadlock detection code! It seems that backend does DeadLockCheck each time when timer expired- shouldn't this be done _once_, before backend is going to sleep ?! 7. As I read in Sybase documentation the default standard transaction mode is _chained_ - all queries before explicitCOMMIT/ABORT are run in _single transaction_... But we have to use BEGIN/END to get it! This was not so bad forsystem with relation level locks, but now only the same row writes block one other and so chained mode seems more appropriate... Shouldn't we change default transaction mode now? And use option/SET TRANSACTION MODE to switch to un-chainedmode if one like it? Vadim
> 6. I'm not happy with current deadlock detection code! > It seems that backend does DeadLockCheck each time > when timer expired - shouldn't this be done _once_, > before backend is going to sleep ?! Not sure. Now that I think of it, it makes sense that if I go to sleep, a deadlock is not sudenly going to appear while I am asleep. If a new process causes a deadlock, the new process that causes it will see it. I did not check when I went to sleep because I thought it may be too cpu-intensive to do checking on every sleep, but now that I remember it, it may be very trivial in cpu time to do the check on every sleep. I recommend changing it to do it just before every sleep. Let me know if you want me to make the change. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian wrote: > > > 6. I'm not happy with current deadlock detection code! > > It seems that backend does DeadLockCheck each time > > when timer expired - shouldn't this be done _once_, > > before backend is going to sleep ?! > > Not sure. Now that I think of it, it makes sense that if I go to sleep, > a deadlock is not sudenly going to appear while I am asleep. If a new > process causes a deadlock, the new process that causes it will see it. > > I did not check when I went to sleep because I thought it may be too > cpu-intensive to do checking on every sleep, but now that I remember it, > it may be very trivial in cpu time to do the check on every sleep. > > I recommend changing it to do it just before every sleep. Let me know > if you want me to make the change. May be we could just reduce first sleep time (60 sec is too long), do DeadLockCheck _only once_, after first SIGALARM, and after that just sleep forever ? Why do DeadLockCheck many times ? Let's think more... And please consider when lock conflict occures: 1. One process tries update row being updated by other. 2. When reading/writing hashes (I hope to change btrees to use new buffer context lock code, as heap access methods do,- this is short term locking without deadlocks and so - without using lockmanager). Vadim
> Bruce Momjian wrote: > > > > > 6. I'm not happy with current deadlock detection code! > > > It seems that backend does DeadLockCheck each time > > > when timer expired - shouldn't this be done _once_, > > > before backend is going to sleep ?! > > > > Not sure. Now that I think of it, it makes sense that if I go to sleep, > > a deadlock is not sudenly going to appear while I am asleep. If a new > > process causes a deadlock, the new process that causes it will see it. > > > > I did not check when I went to sleep because I thought it may be too > > cpu-intensive to do checking on every sleep, but now that I remember it, > > it may be very trivial in cpu time to do the check on every sleep. > > > > I recommend changing it to do it just before every sleep. Let me know > > if you want me to make the change. > > May be we could just reduce first sleep time (60 sec is too long), Yes, much too long. 5 or 15 seconds sounds good. > do DeadLockCheck _only once_, after first SIGALARM, and after that > just sleep forever ? > Why do DeadLockCheck many times ? Yes, no need to do many times. OK. Should I make the change? > > Let's think more... > > And please consider when lock conflict occures: > > 1. One process tries update row being updated by other. > 2. When reading/writing hashes (I hope to change btrees to > use new buffer context lock code, as heap access methods > do, - this is short term locking without deadlocks and so - > without using lockmanager). Oh. I would think even a 1 second deadlock detection would work well. Most short-term locks don't last that long. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
> Bruce Momjian wrote: > > > > > 6. I'm not happy with current deadlock detection code! > > > It seems that backend does DeadLockCheck each time > > > when timer expired - shouldn't this be done _once_, > > > before backend is going to sleep ?! > > > > Not sure. Now that I think of it, it makes sense that if I go to sleep, > > a deadlock is not sudenly going to appear while I am asleep. If a new > > process causes a deadlock, the new process that causes it will see it. > > > > I did not check when I went to sleep because I thought it may be too > > cpu-intensive to do checking on every sleep, but now that I remember it, > > it may be very trivial in cpu time to do the check on every sleep. > > > > I recommend changing it to do it just before every sleep. Let me know > > if you want me to make the change. > > May be we could just reduce first sleep time (60 sec is too long), > do DeadLockCheck _only once_, after first SIGALARM, and after that > just sleep forever ? > Why do DeadLockCheck many times ? > > Let's think more... > > And please consider when lock conflict occures: > > 1. One process tries update row being updated by other. > 2. When reading/writing hashes (I hope to change btrees to > use new buffer context lock code, as heap access methods > do, - this is short term locking without deadlocks and so - > without using lockmanager). > > Vadim > I have applied the following patch. It causes only one deadlock check after a sleep of one second. Applied only to the CURRENT tree. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026 Index: src/backend/parser/scan.c =================================================================== RCS file: /usr/local/cvsroot/pgsql/src/backend/parser/scan.c,v retrieving revision 1.31 diff -c -r1.31 scan.c *** scan.c 1998/10/13 17:26:50 1.31 --- scan.c 1998/12/18 19:30:54 *************** *** 1,7 **** /* A lexical scanner generated by flex */ /* Scanner skeleton version: ! * $Header: /usr/local/cvsroot/pgsql/src/backend/parser/scan.c,v 1.31 1998/10/13 17:26:50 scrappy Exp $ */ #define FLEX_SCANNER --- 1,7 ---- /* A lexical scanner generated by flex */ /* Scanner skeleton version: ! * /master/usr.bin/lex/skel.c,v 1.3 1997/09/25 00:10:23 jch Exp */ #define FLEX_SCANNER *************** *** 556,562 **** * * * IDENTIFICATION ! * $Header: /usr/local/cvsroot/pgsql/src/backend/parser/scan.c,v 1.31 1998/10/13 17:26:50 scrappy Exp $ * *------------------------------------------------------------------------- */ --- 556,562 ---- * * * IDENTIFICATION ! * $Header: /usr/local/cvsroot/pgsql/src/backend/parser/scan.l,v 1.44 1998/10/08 18:29:51 momjian Exp $ * *------------------------------------------------------------------------- */ Index: src/backend/storage/lmgr/proc.c =================================================================== RCS file: /usr/local/cvsroot/pgsql/src/backend/storage/lmgr/proc.c,v retrieving revision 1.43 diff -c -r1.43 proc.c *** proc.c 1998/09/01 04:32:02 1.43 --- proc.c 1998/12/18 19:30:59 *************** *** 77,83 **** #include "storage/proc.h" #include "utils/trace.h" ! static void HandleDeadLock(int sig); static PROC *ProcWakeup(PROC *proc, int errType); #define DeadlockCheckTimer pg_options[OPT_DEADLOCKTIMEOUT] --- 77,83 ---- #include "storage/proc.h" #include "utils/trace.h" ! static void HandleDeadLock(void); static PROC *ProcWakeup(PROC *proc, int errType); #define DeadlockCheckTimer pg_options[OPT_DEADLOCKTIMEOUT] *************** *** 154,161 **** * Routine called if deadlock timer goes off. See ProcSleep() * ------------------ */ - pqsignal(SIGALRM, HandleDeadLock); - SpinAcquire(ProcStructLock); /* attach to the free list */ --- 154,159 ---- *************** *** 449,457 **** TransactionId xid) /* needed by user locks, see below */ { int i; PROC *proc; ! struct itimerval timeval, ! dummy; /* * If the first entries in the waitQueue have a greater priority than --- 447,455 ---- TransactionId xid) /* needed by user locks, see below */ { int i; + bool deadlock_checked = false; PROC *proc; ! struct timeval timeval; /* * If the first entries in the waitQueue have a greater priority than *************** *** 523,539 **** * to 0. * -------------- */ ! MemSet(&timeval, 0, sizeof(struct itimerval)); ! timeval.it_value.tv_sec = \ (DeadlockCheckTimer ? DeadlockCheckTimer : DEADLOCK_CHECK_TIMER); do { MyProc->errType = NO_ERROR; /* reset flag after deadlock check */ ! if (setitimer(ITIMER_REAL, &timeval, &dummy)) elog(FATAL, "ProcSleep: Unable to set timer for process wakeup"); /* -------------- * if someone wakes us between SpinRelease and IpcSemaphoreLock, * IpcSemaphoreLock will not block. The wakeup is "saved" by --- 521,546 ---- * to 0. * -------------- */ ! MemSet(&timeval, 0, sizeof(struct timeval)); ! timeval.tv_sec = \ (DeadlockCheckTimer ? DeadlockCheckTimer : DEADLOCK_CHECK_TIMER); do { + int expire; + MyProc->errType = NO_ERROR; /* reset flag after deadlock check */ ! if ((expire = select(0, NULL, NULL, NULL, ! (deadlock_checked == false) ? &timeval : NULL)) == -1) elog(FATAL, "ProcSleep: Unable to set timer for process wakeup"); + if (expire == 0 /* timeout reached */ && deadlock_checked == false) + { + HandleDeadLock(); + deadlock_checked = true; + } + /* -------------- * if someone wakes us between SpinRelease and IpcSemaphoreLock, * IpcSemaphoreLock will not block. The wakeup is "saved" by *************** *** 545,558 **** } while (MyProc->errType == STATUS_NOT_FOUND); /* sleep after deadlock * check */ - /* --------------- - * We were awoken before a timeout - now disable the timer - * --------------- - */ - timeval.it_value.tv_sec = 0; - if (setitimer(ITIMER_REAL, &timeval, &dummy)) - elog(FATAL, "ProcSleep: Unable to diable timer for process wakeup"); - /* ---------------- * We were assumed to be in a critical section when we went * to sleep. --- 552,557 ---- *************** *** 695,701 **** * -------------------- */ static void ! HandleDeadLock(int sig) { LOCK *mywaitlock; --- 694,700 ---- * -------------------- */ static void ! HandleDeadLock() { LOCK *mywaitlock; Index: src/pl/plpgsql/src/gram.c =================================================================== RCS file: /usr/local/cvsroot/pgsql/src/pl/plpgsql/src/gram.c,v retrieving revision 1.1 diff -c -r1.1 gram.c *** gram.c 1998/10/28 17:07:17 1.1 --- gram.c 1998/12/18 19:31:12 *************** *** 65,71 **** * procedural language * * IDENTIFICATION ! * $Header: /usr/local/cvsroot/pgsql/src/pl/plpgsql/src/gram.c,v 1.1 1998/10/28 17:07:17 momjian Exp $ * * This software is copyrighted by Jan Wieck - Hamburg. * --- 65,71 ---- * procedural language * * IDENTIFICATION ! * $Header: /usr/local/cvsroot/pgsql/src/pl/plpgsql/src/gram.y,v 1.1 1998/08/24 19:14:47 momjian Exp $ * * This software is copyrighted by Jan Wieck - Hamburg. * Index: src/pl/plpgsql/src/scan.c =================================================================== RCS file: /usr/local/cvsroot/pgsql/src/pl/plpgsql/src/scan.c,v retrieving revision 1.1 diff -c -r1.1 scan.c *** scan.c 1998/10/28 17:07:17 1.1 --- scan.c 1998/12/18 19:31:21 *************** *** 635,641 **** * procedural language * * IDENTIFICATION ! * $Header: /usr/local/cvsroot/pgsql/src/pl/plpgsql/src/scan.c,v 1.1 1998/10/28 17:07:17 momjian Exp $ * * This software is copyrighted by Jan Wieck - Hamburg. * --- 635,641 ---- * procedural language * * IDENTIFICATION ! * $Header: /usr/local/cvsroot/pgsql/src/pl/plpgsql/src/scan.l,v 1.1 1998/08/24 19:14:49 momjian Exp $ * * This software is copyrighted by Jan Wieck - Hamburg. *
Bruce Momjian wrote: > > I have applied the following patch. It causes only one deadlock check > after a sleep of one second. Applied only to the CURRENT tree. Nice, at least for now. Vadim
Hi all, > -----Original Message----- > From: owner-pgsql-hackers@postgreSQL.org > [mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Vadim Mikheev > Sent: Wednesday, December 16, 1998 10:01 PM > To: hackers@postgreSQL.org > Subject: [HACKERS] MVCC works in serialized mode! > > > CVS is just updated... > > Please try concurrent writes/reads... > I happend to enjoy MVCC a little in (v6.4.1) . Readers are never blocked and writers are blocked only by same row writers as I expected. It's so comfortable. But I have a question. Once transactions are blocked,it takes so long time to resume after blocks were removed. Why ? Currently blocked transactions resume immediately after blocks were removed. Thanks. Hiroshi Inoue Inoue@tpf.co.jp
[Charset iso-8859-1 unsupported, filtering to ASCII...] > Hi all, > > > -----Original Message----- > > From: owner-pgsql-hackers@postgreSQL.org > > [mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Vadim Mikheev > > Sent: Wednesday, December 16, 1998 10:01 PM > > To: hackers@postgreSQL.org > > Subject: [HACKERS] MVCC works in serialized mode! > > > > > > CVS is just updated... > > > > Please try concurrent writes/reads... > > > > I happend to enjoy MVCC a little in (v6.4.1) . That's funny. > > Readers are never blocked and writers are blocked only by > same row writers as I expected. > It's so comfortable. Wow, that is cool. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Hiroshi Inoue wrote: > > > > > CVS is just updated... > > > > Please try concurrent writes/reads... > > > > I happend to enjoy MVCC a little in (v6.4.1) . > > Readers are never blocked and writers are blocked only by > same row writers as I expected. > It's so comfortable. > > But I have a question. > Once transactions are blocked,it takes so long time to resume > after blocks were removed. > Why ? > Currently blocked transactions resume immediately after blocks > were removed. What do you mean? Example please... Vadim
> -----Original Message-----
> From: root@dune.krs.ru [mailto:root@dune.krs.ru]On Behalf Of Vadim
> Mikheev
> Sent: Friday, December 25, 1998 2:40 PM
> To: Hiroshi Inoue
> Cc: hackers@postgreSQL.org
> Subject: Re: [HACKERS] MVCC works in serialized mode!
>
>
> Hiroshi Inoue wrote:
> >
> > >
> > > CVS is just updated...
> > >
> > > Please try concurrent writes/reads...
> > >
> >
> > I happend to enjoy MVCC a little in (v6.4.1) .
> >
> > Readers are never blocked and writers are blocked only by
> > same row writers as I expected.
> > It's so comfortable.
> >
> > But I have a question.
> > Once transactions are blocked,it takes so long time to resume
> > after blocks were removed.
> > Why ?
> > Currently blocked transactions resume immediately after blocks
> > were removed.
>
> What do you mean?
> Example please...
>
inoue=> create table t1 (key int,a int);
inoue=> insert into t1 values (1,0);
<Session-1> <Session-2>
inoue=> begin; inoue=> begin;
BEGIN BEGIN
inoue=> update t1 set a=1 where key=1;
UPDATE 1 inoue=> update
t1 set a=2 where key=1;
[ blocked ]
inoue=> end/abort;
END/ABORT [ after
long time .... ] ERROR: Can't
serialize access due to concurrent update / UPDATE 1
Thanks.
Hiroshi Inoue
Inoue@tpf.co.jp
Hiroshi Inoue wrote: > > > > > > > But I have a question. > > > Once transactions are blocked,it takes so long time to resume > > > after blocks were removed. > > > Why ? > > > Currently blocked transactions resume immediately after blocks > > > were removed. > > > > What do you mean? > > Example please... > > > [ after long time .... ] > ERROR: Can't serialize access due to concurrent update Just tried (with CURRENT) - no problems... Vadim
> > 2. LOCK TABLE modes are not yet implemented... Implemented, as well as SET TRANSACTION ISOLATION LEVEL... > > 3. As well as SELECT ... FOR UPDATE: I need in advice here! > > Do Oracle, Informix etc support SELECT FOR UPDATE if > there are aggregates, group by, distinct in SELECT ??? As I see in Oracle documentation FOR UPDATE is disallowed in these cases... No objections ? > Is SELECT FOR UPDATE allowed in subqueries ??? Could someone test this? > > 7. As I read in Sybase documentation the default > standard transaction mode is _chained_ - all > queries before explicit COMMIT/ABORT are run in _single > transaction_... But we have to use BEGIN/END to get it! > This was not so bad for system with relation level locks, > but now only the same row writes block one other and so > chained mode seems more appropriate... > > Shouldn't we change default transaction mode now? > And use option/SET TRANSACTION MODE to switch to > un-chained mode if one like it? No comments on this? I would like to make BEGIN implicit... Objections? Vadim
> > > > 7. As I read in Sybase documentation the default > > standard transaction mode is _chained_ - all > > queries before explicit COMMIT/ABORT are run in _single > > transaction_... But we have to use BEGIN/END to get it! > > This was not so bad for system with relation level locks, > > but now only the same row writes block one other and so > > chained mode seems more appropriate... > > > > Shouldn't we change default transaction mode now? > > And use option/SET TRANSACTION MODE to switch to > > un-chained mode if one like it? > > No comments on this? > I would like to make BEGIN implicit... > Objections? Informix requires BEGIN. Ingres does not, but has 'set autocommit' mode, so each SQL statement is its own transaction, like we have now. I always felt that if I go in to psql and run a query, I want it applied. I don't want to have to commit every query I type. I know I can use SET TRANSACTION MODE to change this. What do others think? -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian wrote: > > > 7. As I read in Sybase documentation the default > > > standard transaction mode is _chained_ - all > > > queries before explicit COMMIT/ABORT are run in _single > > > transaction_... But we have to use BEGIN/END to get it! > > > This was not so bad for system with relation level locks, > > > but now only the same row writes block one other and so > > > chained mode seems more appropriate... > > > > > > Shouldn't we change default transaction mode now? > > > And use option/SET TRANSACTION MODE to switch to > > > un-chained mode if one like it? > > > > No comments on this? > > I would like to make BEGIN implicit... > > Objections? > > Informix requires BEGIN. Ingres does not, but has 'set autocommit' > mode, so each SQL statement is its own transaction, like we have now. > > I always felt that if I go in to psql and run a query, I want it > applied. I don't want to have to commit every query I type. > > I know I can use SET TRANSACTION MODE to change this. > > What do others think? As long as we can switch. Interactive applications typically update more than one objects per transaction. -------- Regards Theo
> > No comments on this?
> > I would like to make BEGIN implicit...
> > Objections?
> Informix requires BEGIN. Ingres does not, but has 'set autocommit'
> mode, so each SQL statement is its own transaction, like we have now.
imho we should have an Ingres-like feature for "autocommit". We can have
the default be "set autocommit on" (probably with an equals sign like
our other "set" variables) and we can have it be a run-time option like
DATESTYLE and other settable parameters. So you can configure your
server or your client environment to always behave the way you prefer.
I'd be happy to implement the syntax and the environment variable parts;
just need a global boolean value to check and set...
- Tom
Hello! I want to make a filtered selection on a table. This can return with many tuples. The download of the query results may take a few minutes. I'd like to show the progress status. How can i do this? If i run this in two query (first i count the number of lines, than the real query), than the backend calculates the query two times. If i use cursor, i can not get the line count before i fetch the whole table. --So long and thanx for all the fish.NeKo@(kva.hu|kornel.szif.hu)
"Thomas G. Lockhart" wrote: > > > > No comments on this? > > > I would like to make BEGIN implicit... > > > Objections? > > Informix requires BEGIN. Ingres does not, but has 'set autocommit' > > mode, so each SQL statement is its own transaction, like we have now. > > imho we should have an Ingres-like feature for "autocommit". We can have > the default be "set autocommit on" (probably with an equals sign like > our other "set" variables) and we can have it be a run-time option like > DATESTYLE and other settable parameters. So you can configure your > server or your client environment to always behave the way you prefer. > > I'd be happy to implement the syntax and the environment variable parts; > just need a global boolean value to check and set... Nice! TIA -:) Could you also change configure stuff, to choose default mode before compiling? Vadim
> > imho we should have an Ingres-like feature for "autocommit".
> Nice! TIA -:)
Sure, but if I do too much work on this I'll need you to help on outer
joins :) (Actually, that's just an excuse; I'm flailing on it trying to
understand things you already know.)
Is there a global boolean "autocommit flag" already there to work with,
or could you add one? I'd like to (if possible) keep from getting sucked
into non-parser/non-docs work until I've worked out some of the outer
join stuff. But I can do the parser and "set autocommit" stuff. Also,
I'd like to go through the gram.y code for the new MVCC support commands
and have it use parser keywords rather than "Ident" entities for the
syntax. OK?
> Could you also change configure stuff, to choose default mode
> before compiling?
I'll set things up for it in the code, and perhaps Tom Lane or someone
can help with the configure support. We can ask when it's time...
- Tom