Thread: ARC buffer strategy committed

I have just committed the implementation of ARC into the 7.5devel tree.

This doesn't contain the vacuum page delay, or the test implementation 
of the background writer based on the checkpoint process. These two need 
some more discussion and bouncing around.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Jan Wieck <JanWieck@Yahoo.com> writes:
> I have just committed the implementation of ARC into the 7.5devel tree.

I'm seeing a whole bunch of regression test failures that weren't there
half an hour ago ...
        regards, tom lane

Tom Lane wrote:

> Jan Wieck <JanWieck@Yahoo.com> writes:
>> I have just committed the implementation of ARC into the 7.5devel tree.
> 
> I'm seeing a whole bunch of regression test failures that weren't there
> half an hour ago ...
> 
>             regards, tom lane

Oh ... hmmm ... will check


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Jan Wieck wrote:

> Tom Lane wrote:
> 
>> Jan Wieck <JanWieck@Yahoo.com> writes:
>>> I have just committed the implementation of ARC into the 7.5devel tree.
>> 
>> I'm seeing a whole bunch of regression test failures that weren't there
>> half an hour ago ...
>> 
>>             regards, tom lane
> 
> Oh ... hmmm ... will check

That's strange, if I reverse apply my patch I have buffer leak's and all 
kinds of crap. Can't even initdb. Who else applied patches tonight?


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Jan Wieck <JanWieck@Yahoo.com> writes:
> That's strange, if I reverse apply my patch I have buffer leak's and all 
> kinds of crap. Can't even initdb. Who else applied patches tonight?

Well, there was my operator-class patch (which went nowhere near the
buffer stuff) and also the initdb-in-C code (new at least to me).
But on my HP machine all that stuff seems to play nice.  On my Linux
machine, which I cvs updated just a little bit later, problems :-(
        regards, tom lane

Jan Wieck <JanWieck@Yahoo.com> writes:
> That's strange, if I reverse apply my patch I have buffer leak's and all 
> kinds of crap. Can't even initdb. Who else applied patches tonight?

FWIW, I can initdb --- that seems fine --- but the regression tests spew
lots of small fragments.  What I suspect is that the bufmgr is
occasionally returning the wrong buffer :-(

regression.diffs attached.  Also, "grep TRAP postmaster.log" shows
this is the cause of the coredump midway through:

TRAP: FailedAssertion("!(( (&(cdb_replace->buf_tag))->rnode.tblNode == (&(buf->tag))->rnode.tblNode &&
(&(cdb_replace->buf_tag))->rnode.relNode== (&(buf->tag))->rnode.relNode && (&(cdb_replace->buf_tag))->blockNum ==
(&(buf->tag))->blockNum))", File: "freelist.c", Line: 610)
 

Note this is with a serial test (make installcheck).  Chris' very
different-looking results may be because he was doing parallel tests;
or there might be something else involved.
        regards, tom lane


*** ./expected/name.out    Sun May 25 20:11:28 2003
--- ./results/name.out    Wed Nov 12 22:16:27 2003
***************
*** 90,126 **** (6 rows)  SELECT '' AS seven, c.f1 FROM NAME_TBL c WHERE c.f1 ~ '.*';
!  seven |                               f1                                
! -------+-----------------------------------------------------------------
!        | 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQ
!        | 1234567890abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopq
!        | asdfghjkl;
!        | 343f%2a
!        | d34aaasdf
!        | 
!        | 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQ
! (7 rows)
!  SELECT '' AS zero, c.f1 FROM NAME_TBL c WHERE c.f1 !~ '.*';
!  zero | f1 
! ------+----
! (0 rows)
!  SELECT '' AS three, c.f1 FROM NAME_TBL c WHERE c.f1 ~ '[0-9]';
!  three |                               f1                                
! -------+-----------------------------------------------------------------
!        | 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQ
!        | 1234567890abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopq
!        | 343f%2a
!        | d34aaasdf
!        | 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890ABCDEFGHIJKLMNOPQ
! (5 rows)
!  SELECT '' AS two, c.f1 FROM NAME_TBL c WHERE c.f1 ~ '.*asdf.*';
!  two |     f1     
! -----+------------
!      | asdfghjkl;
!      | d34aaasdf
! (2 rows)
!  DROP TABLE NAME_TBL;
--- 90,104 ---- (6 rows)  SELECT '' AS seven, c.f1 FROM NAME_TBL c WHERE c.f1 ~ '.*';
! ERROR:  could not access status of transaction 2752512
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS zero,
c.f1FROM NAME_TBL c WHERE c.f1 !~ '.*';
 
! ERROR:  could not access status of transaction 2752512
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS three,
c.f1FROM NAME_TBL c WHERE c.f1 ~ '[0-9]';
 
! ERROR:  could not access status of transaction 2752512
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS two,
c.f1FROM NAME_TBL c WHERE c.f1 ~ '.*asdf.*';
 
! ERROR:  could not access status of transaction 2752512
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory DROP TABLE
NAME_TBL;

======================================================================

*** ./expected/varchar.out    Mon May 21 12:54:46 2001
--- ./results/varchar.out    Wed Nov 12 22:16:27 2003
***************
*** 2,111 **** -- VARCHAR -- CREATE TABLE VARCHAR_TBL(f1 varchar(1)); INSERT INTO VARCHAR_TBL (f1) VALUES ('a'); INSERT
INTOVARCHAR_TBL (f1) VALUES ('A'); -- any of the following three input formats are acceptable  INSERT INTO VARCHAR_TBL
(f1)VALUES ('1'); INSERT INTO VARCHAR_TBL (f1) VALUES (2); INSERT INTO VARCHAR_TBL (f1) VALUES ('3'); -- zero-length
char INSERT INTO VARCHAR_TBL (f1) VALUES (''); -- try varchar's of greater than 1 length  INSERT INTO VARCHAR_TBL (f1)
VALUES('cd');
 
! ERROR:  value too long for type character varying(1) INSERT INTO VARCHAR_TBL (f1) VALUES ('c     '); SELECT '' AS
seven,VARCHAR_TBL.*;
 
!  seven | f1 
! -------+----
!        | a
!        | A
!        | 1
!        | 2
!        | 3
!        | 
!        | c
! (7 rows)
!  SELECT '' AS six, c.*    FROM VARCHAR_TBL c    WHERE c.f1 <> 'a';
!  six | f1 
! -----+----
!      | A
!      | 1
!      | 2
!      | 3
!      | 
!      | c
! (6 rows)
!  SELECT '' AS one, c.*    FROM VARCHAR_TBL c    WHERE c.f1 = 'a';
!  one | f1 
! -----+----
!      | a
! (1 row)
!  SELECT '' AS five, c.*    FROM VARCHAR_TBL c    WHERE c.f1 < 'a';
!  five | f1 
! ------+----
!       | A
!       | 1
!       | 2
!       | 3
!       | 
! (5 rows)
!  SELECT '' AS six, c.*    FROM VARCHAR_TBL c    WHERE c.f1 <= 'a';
!  six | f1 
! -----+----
!      | a
!      | A
!      | 1
!      | 2
!      | 3
!      | 
! (6 rows)
!  SELECT '' AS one, c.*    FROM VARCHAR_TBL c    WHERE c.f1 > 'a';
!  one | f1 
! -----+----
!      | c
! (1 row)
!  SELECT '' AS two, c.*    FROM VARCHAR_TBL c    WHERE c.f1 >= 'a';
!  two | f1 
! -----+----
!      | a
!      | c
! (2 rows)
!  DROP TABLE VARCHAR_TBL; -- -- Now test longer arrays of char -- CREATE TABLE VARCHAR_TBL(f1 varchar(4)); INSERT INTO
VARCHAR_TBL(f1) VALUES ('a'); INSERT INTO VARCHAR_TBL (f1) VALUES ('ab'); INSERT INTO VARCHAR_TBL (f1) VALUES ('abcd');
INSERTINTO VARCHAR_TBL (f1) VALUES ('abcde');
 
! ERROR:  value too long for type character varying(4) INSERT INTO VARCHAR_TBL (f1) VALUES ('abcd    '); SELECT '' AS
four,VARCHAR_TBL.*;
 
!  four |  f1  
! ------+------
!       | a
!       | ab
!       | abcd
!       | abcd
! (4 rows)
! 
--- 2,70 ---- -- VARCHAR -- CREATE TABLE VARCHAR_TBL(f1 varchar(1));
+ ERROR:  could not access status of transaction 2752512
+ DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory INSERT INTO
VARCHAR_TBL(f1) VALUES ('a');
 
+ ERROR:  relation "varchar_tbl" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES ('A');
+ ERROR:  relation "varchar_tbl" does not exist -- any of the following three input formats are acceptable  INSERT INTO
VARCHAR_TBL(f1) VALUES ('1');
 
+ ERROR:  relation "varchar_tbl" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES (2);
+ ERROR:  relation "varchar_tbl" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES ('3');
+ ERROR:  relation "varchar_tbl" does not exist -- zero-length char  INSERT INTO VARCHAR_TBL (f1) VALUES ('');
+ ERROR:  relation "varchar_tbl" does not exist -- try varchar's of greater than 1 length  INSERT INTO VARCHAR_TBL (f1)
VALUES('cd');
 
! ERROR:  relation "varchar_tbl" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES ('c     ');
+ ERROR:  relation "varchar_tbl" does not exist SELECT '' AS seven, VARCHAR_TBL.*;
! ERROR:  relation "varchar_tbl" does not exist SELECT '' AS six, c.*    FROM VARCHAR_TBL c    WHERE c.f1 <> 'a';
! ERROR:  relation "varchar_tbl" does not exist SELECT '' AS one, c.*    FROM VARCHAR_TBL c    WHERE c.f1 = 'a';
! ERROR:  relation "varchar_tbl" does not exist SELECT '' AS five, c.*    FROM VARCHAR_TBL c    WHERE c.f1 < 'a';
! ERROR:  relation "varchar_tbl" does not exist SELECT '' AS six, c.*    FROM VARCHAR_TBL c    WHERE c.f1 <= 'a';
! ERROR:  relation "varchar_tbl" does not exist SELECT '' AS one, c.*    FROM VARCHAR_TBL c    WHERE c.f1 > 'a';
! ERROR:  relation "varchar_tbl" does not exist SELECT '' AS two, c.*    FROM VARCHAR_TBL c    WHERE c.f1 >= 'a';
! ERROR:  relation "varchar_tbl" does not exist DROP TABLE VARCHAR_TBL;
+ ERROR:  table "varchar_tbl" does not exist -- -- Now test longer arrays of char -- CREATE TABLE VARCHAR_TBL(f1
varchar(4));
+ ERROR:  type "pg_catalog.varchar" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES ('a');
+ ERROR:  relation "varchar_tbl" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES ('ab');
+ ERROR:  relation "varchar_tbl" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES ('abcd');
+ ERROR:  relation "varchar_tbl" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES ('abcde');
! ERROR:  relation "varchar_tbl" does not exist INSERT INTO VARCHAR_TBL (f1) VALUES ('abcd    ');
+ ERROR:  relation "varchar_tbl" does not exist SELECT '' AS four, VARCHAR_TBL.*;
! ERROR:  relation "varchar_tbl" does not exist

======================================================================

*** ./expected/text.out    Tue Jan  4 11:19:34 2000
--- ./results/text.out    Wed Nov 12 22:16:27 2003
***************
*** 2,25 **** -- TEXT -- SELECT text 'this is a text string' = text 'this is a text string' AS true;
!  true 
! ------
!  t
! (1 row)
!  SELECT text 'this is a text string' = text 'this is a text strin' AS false;
!  false 
! -------
!  f
! (1 row)
!  CREATE TABLE TEXT_TBL (f1 text); INSERT INTO TEXT_TBL VALUES ('doh!'); INSERT INTO TEXT_TBL VALUES ('hi de ho
neighbor');SELECT '' AS two, * FROM TEXT_TBL;
 
!  two |        f1         
! -----+-------------------
!      | doh!
!      | hi de ho neighbor
! (2 rows)
! 
--- 2,15 ---- -- TEXT -- SELECT text 'this is a text string' = text 'this is a text string' AS true;
! ERROR:  type "text" does not exist SELECT text 'this is a text string' = text 'this is a text strin' AS false;
! ERROR:  type "text" does not exist CREATE TABLE TEXT_TBL (f1 text);
+ ERROR:  type "text" does not exist INSERT INTO TEXT_TBL VALUES ('doh!');
+ ERROR:  relation "text_tbl" does not exist INSERT INTO TEXT_TBL VALUES ('hi de ho neighbor');
+ ERROR:  relation "text_tbl" does not exist SELECT '' AS two, * FROM TEXT_TBL;
! ERROR:  relation "text_tbl" does not exist

======================================================================

*** ./expected/int8.out    Thu Mar 27 11:35:31 2003
--- ./results/int8.out    Wed Nov 12 22:16:27 2003
***************
*** 111,285 **** -- SELECT '' AS to_char_1, to_char(q1, '9G999G999G999G999G999'), to_char(q2, '9,999,999,999,999,999')
   FROM INT8_TBL;
 
!  to_char_1 |        to_char         |        to_char         
! -----------+------------------------+------------------------
!            |                    123 |                    456
!            |                    123 |  4,567,890,123,456,789
!            |  4,567,890,123,456,789 |                    123
!            |  4,567,890,123,456,789 |  4,567,890,123,456,789
!            |  4,567,890,123,456,789 | -4,567,890,123,456,789
! (5 rows)
!  SELECT '' AS to_char_2, to_char(q1, '9G999G999G999G999G999D999G999'), to_char(q2, '9,999,999,999,999,999.999,999')
  FROM INT8_TBL;    
 
!  to_char_2 |            to_char             |            to_char             
! -----------+--------------------------------+--------------------------------
!            |                    123.000,000 |                    456.000,000
!            |                    123.000,000 |  4,567,890,123,456,789.000,000
!            |  4,567,890,123,456,789.000,000 |                    123.000,000
!            |  4,567,890,123,456,789.000,000 |  4,567,890,123,456,789.000,000
!            |  4,567,890,123,456,789.000,000 | -4,567,890,123,456,789.000,000
! (5 rows)
!  SELECT '' AS to_char_3, to_char( (q1 * -1), '9999999999999999PR'), to_char( (q2 * -1), '9999999999999999.999PR')
FROM INT8_TBL;
 
!  to_char_3 |      to_char       |        to_char         
! -----------+--------------------+------------------------
!            |              <123> |              <456.000>
!            |              <123> | <4567890123456789.000>
!            | <4567890123456789> |              <123.000>
!            | <4567890123456789> | <4567890123456789.000>
!            | <4567890123456789> |  4567890123456789.000 
! (5 rows)
!  SELECT '' AS to_char_4, to_char( (q1 * -1), '9999999999999999S'), to_char( (q2 * -1), 'S9999999999999999')      FROM
INT8_TBL;
!  to_char_4 |      to_char      |      to_char      
! -----------+-------------------+-------------------
!            |              123- |              -456
!            |              123- | -4567890123456789
!            | 4567890123456789- |              -123
!            | 4567890123456789- | -4567890123456789
!            | 4567890123456789- | +4567890123456789
! (5 rows)
!  SELECT '' AS to_char_5,  to_char(q2, 'MI9999999999999999')     FROM INT8_TBL;    
!  to_char_5 |      to_char      
! -----------+-------------------
!            |               456
!            |  4567890123456789
!            |               123
!            |  4567890123456789
!            | -4567890123456789
! (5 rows)
!  SELECT '' AS to_char_6,  to_char(q2, 'FMS9999999999999999')    FROM INT8_TBL;
!  to_char_6 |      to_char      
! -----------+-------------------
!            | +456
!            | +4567890123456789
!            | +123
!            | +4567890123456789
!            | -4567890123456789
! (5 rows)
!  SELECT '' AS to_char_7,  to_char(q2, 'FM9999999999999999THPR') FROM INT8_TBL;
!  to_char_7 |      to_char       
! -----------+--------------------
!            | 456TH
!            | 4567890123456789TH
!            | 123RD
!            | 4567890123456789TH
!            | <4567890123456789>
! (5 rows)
!  SELECT '' AS to_char_8,  to_char(q2, 'SG9999999999999999th')   FROM INT8_TBL;    
!  to_char_8 |       to_char       
! -----------+---------------------
!            | +             456th
!            | +4567890123456789th
!            | +             123rd
!            | +4567890123456789th
!            | -4567890123456789
! (5 rows)
!  SELECT '' AS to_char_9,  to_char(q2, '0999999999999999')       FROM INT8_TBL;    
!  to_char_9 |      to_char      
! -----------+-------------------
!            |  0000000000000456
!            |  4567890123456789
!            |  0000000000000123
!            |  4567890123456789
!            | -4567890123456789
! (5 rows)
!  SELECT '' AS to_char_10, to_char(q2, 'S0999999999999999')      FROM INT8_TBL;    
!  to_char_10 |      to_char      
! ------------+-------------------
!             | +0000000000000456
!             | +4567890123456789
!             | +0000000000000123
!             | +4567890123456789
!             | -4567890123456789
! (5 rows)
!  SELECT '' AS to_char_11, to_char(q2, 'FM0999999999999999')     FROM INT8_TBL;    
!  to_char_11 |      to_char      
! ------------+-------------------
!             | 0000000000000456
!             | 4567890123456789
!             | 0000000000000123
!             | 4567890123456789
!             | -4567890123456789
! (5 rows)
!  SELECT '' AS to_char_12, to_char(q2, 'FM9999999999999999.000') FROM INT8_TBL;
!  to_char_12 |        to_char        
! ------------+-----------------------
!             | 456.000
!             | 4567890123456789.000
!             | 123.000
!             | 4567890123456789.000
!             | -4567890123456789.000
! (5 rows)
!  SELECT '' AS to_char_13, to_char(q2, 'L9999999999999999.000')  FROM INT8_TBL;    
!  to_char_13 |        to_char         
! ------------+------------------------
!             |                456.000
!             |   4567890123456789.000
!             |                123.000
!             |   4567890123456789.000
!             |  -4567890123456789.000
! (5 rows)
!  SELECT '' AS to_char_14, to_char(q2, 'FM9999999999999999.999') FROM INT8_TBL;
!  to_char_14 |      to_char       
! ------------+--------------------
!             | 456.
!             | 4567890123456789.
!             | 123.
!             | 4567890123456789.
!             | -4567890123456789.
! (5 rows)
!  SELECT '' AS to_char_15, to_char(q2, 'S 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 . 9 9 9') FROM INT8_TBL;
!  to_char_15 |                  to_char                  
! ------------+-------------------------------------------
!             |                            +4 5 6 . 0 0 0
!             |  +4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 . 0 0 0
!             |                            +1 2 3 . 0 0 0
!             |  +4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 . 0 0 0
!             |  -4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 . 0 0 0
! (5 rows)
!  SELECT '' AS to_char_16, to_char(q2, '99999 "text" 9999 "9999" 999 "\\"text between quote marks\\"" 9999') FROM
INT8_TBL;
!  to_char_16 |                          to_char                          
! ------------+-----------------------------------------------------------
!             |       text      9999     "text between quote marks"   456
!             |  45678 text 9012 9999 345 "text between quote marks" 6789
!             |       text      9999     "text between quote marks"   123
!             |  45678 text 9012 9999 345 "text between quote marks" 6789
!             | -45678 text 9012 9999 345 "text between quote marks" 6789
! (5 rows)
!  SELECT '' AS to_char_17, to_char(q2, '999999SG9999999999')     FROM INT8_TBL;
!  to_char_17 |      to_char      
! ------------+-------------------
!             |       +       456
!             | 456789+0123456789
!             |       +       123
!             | 456789+0123456789
!             | 456789-0123456789
! (5 rows)
! 
--- 111,166 ---- -- SELECT '' AS to_char_1, to_char(q1, '9G999G999G999G999G999'), to_char(q2, '9,999,999,999,999,999')
   FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_2,to_char(q1, '9G999G999G999G999G999D999G999'), to_char(q2, '9,999,999,999,999,999.999,999')      FROM
INT8_TBL;   
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_3,to_char( (q1 * -1), '9999999999999999PR'), to_char( (q2 * -1), '9999999999999999.999PR')      FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_4,to_char( (q1 * -1), '9999999999999999S'), to_char( (q2 * -1), 'S9999999999999999')      FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_5, to_char(q2, 'MI9999999999999999')     FROM INT8_TBL;    
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_6, to_char(q2, 'FMS9999999999999999')    FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_7, to_char(q2, 'FM9999999999999999THPR') FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_8, to_char(q2, 'SG9999999999999999th')   FROM INT8_TBL;    
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_9, to_char(q2, '0999999999999999')       FROM INT8_TBL;    
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_10,to_char(q2, 'S0999999999999999')      FROM INT8_TBL;    
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_11,to_char(q2, 'FM0999999999999999')     FROM INT8_TBL;    
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_12,to_char(q2, 'FM9999999999999999.000') FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_13,to_char(q2, 'L9999999999999999.000')  FROM INT8_TBL;    
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_14,to_char(q2, 'FM9999999999999999.999') FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_15,to_char(q2, 'S 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 . 9 9 9') FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_16,to_char(q2, '99999 "text" 9999 "9999" 999 "\\"text between quote marks\\"" 9999') FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory SELECT '' AS
to_char_17,to_char(q2, '999999SG9999999999')     FROM INT8_TBL;
 
! ERROR:  could not access status of transaction 2686976
! DETAIL:  could not open file "/home/tgl/testversion/data/pg_clog/0002": No such file or directory

======================================================================

*** ./expected/oid.out    Thu Sep 25 02:58:06 2003
--- ./results/oid.out    Wed Nov 12 22:16:27 2003
***************
*** 41,75 **** (5 rows)  SELECT '' AS three, o.* FROM OID_TBL o WHERE o.f1 <= '1234';
!  three |  f1  
! -------+------
!        | 1234
!        |  987
!        |    0
! (3 rows)
! 
! SELECT '' AS two, o.* FROM OID_TBL o WHERE o.f1 < '1234';
!  two | f1  
! -----+-----
!      | 987
!      |   0
! (2 rows)
! 
! SELECT '' AS four, o.* FROM OID_TBL o WHERE o.f1 >= '1234';
!  four |     f1     
! ------+------------
!       |       1234
!       |       1235
!       | 4294966256
!       |   99999999
! (4 rows)
! 
! SELECT '' AS three, o.* FROM OID_TBL o WHERE o.f1 > '1234';
!  three |     f1     
! -------+------------
!        |       1235
!        | 4294966256
!        |   99999999
! (3 rows)
! 
! DROP TABLE OID_TBL;
--- 41,47 ---- (5 rows)  SELECT '' AS three, o.* FROM OID_TBL o WHERE o.f1 <= '1234';
! server closed the connection unexpectedly
!     This probably means the server terminated abnormally
!     before or while processing the request.
! connection to server was lost

... from here down they all fail with
! psql: FATAL:  the database system is starting up

Jan Wieck <JanWieck@Yahoo.com> writes:
> Can't even initdb. Who else applied patches tonight?

FWIW, I did "make distclean", full rebuild, initdb on both HP and Linux
machines.  The HP passes both serial and parallel regression tests.
The Linux machine initdb's, and simple manual queries seem to work,
but regression tests fail hard.  "diff -r" confirms that the only
difference in the source trees is your ARC patch.  Configuration in
both cases is --enable-cassert --enable-debug ...
        regards, tom lane

Tom Lane wrote:
> Jan Wieck <JanWieck@Yahoo.com> writes:
>> That's strange, if I reverse apply my patch I have buffer leak's and all 
>> kinds of crap. Can't even initdb. Who else applied patches tonight?
> 
> FWIW, I can initdb --- that seems fine --- but the regression tests spew
> lots of small fragments.  What I suspect is that the bufmgr is
> occasionally returning the wrong buffer :-(

It does, somehow and magically ... I have not found the exact reason 
yet. I have backed out my patch for the moment, more tomorrow.


Sorry,
Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Jan Wieck wrote:

> Tom Lane wrote:
>> Jan Wieck <JanWieck@Yahoo.com> writes:
>>> That's strange, if I reverse apply my patch I have buffer leak's and all 
>>> kinds of crap. Can't even initdb. Who else applied patches tonight?
>> 
>> FWIW, I can initdb --- that seems fine --- but the regression tests spew
>> lots of small fragments.  What I suspect is that the bufmgr is
>> occasionally returning the wrong buffer :-(
> 
> It does, somehow and magically ... I have not found the exact reason 
> yet. I have backed out my patch for the moment, more tomorrow.

Yeah, adding a buffer multiple times to the list of unused buffers 
ensures that it later on gets used for multiple contents simultaneously. 
Would be cool if that actually worked, it would give the discussion 
about recommended shared buffer size a totally new twist ;-)


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Jan Wieck <JanWieck@Yahoo.com> writes:
> This all has changed. The freelist in ARC really contains clean and 
> unused buffers. I think I should write up a README.freelist explaining 
> the new bufmgr<->strategy interface and how the algorithm works.

Yes, I was going to complain about the lack of any README updates, too ...
        regards, tom lane

Tom Lane wrote:
> Jan Wieck <JanWieck@Yahoo.com> writes:
>> Yeah, adding a buffer multiple times to the list of unused buffers 
>> ensures that it later on gets used for multiple contents simultaneously. 
> 
> Hm.  Looking at the patch last night, I was wondering why you had
> removed all the guard logic from BufTableInsert and BufTableDelete.
> Was that indeed a bad idea?  In particular, the removal of this bit
> from BufTableDelete

I had to, because that BufTable does not contain references to page 
buffers any more, it contains references to cache directory entries 
(CDB), which can reference a buffer (T1 and T2 CDBs) or can just 
remember the block-tag of a page that once was in memory (B1 and B2 
CDBs). When a CDB gets removed from that hash-table, its page is long 
gone from the memory. As a matter of fact since it is a part of the 
replacement strategy, it should reside in freelist.c ... and freelist.c 
should be renamed to buf_strategy.c or so.

Also, the whole freelist thing was totally misnamed. GetFreeBuffer() did 
not return a free buffer, but an unpinned one that might be free, used 
or even dirty. So the old "freelist" simply contained all unpinned 
buffers. PinBuffer() removed it from there when it changed refcount from 
zero, and UnpinBuffer() added it back when the refcount dropped to zero.

This all has changed. The freelist in ARC really contains clean and 
unused buffers. I think I should write up a README.freelist explaining 
the new bufmgr<->strategy interface and how the algorithm works.


Jan

> 
>     /*
>      * Clear the buffer's tag.  This doesn't matter for the hash table,
>      * since the buffer is already removed from it, but it ensures that
>      * sequential searches through the buffer table won't think the buffer
>      * is still valid for its old page.
>      */
>     buf->tag.rnode.relNode = InvalidOid;
>     buf->tag.rnode.tblNode = InvalidOid;
> 
> worries me quite a lot, because I *know* that was necessary before.
> Have you really changed the search algorithms to the point where it's not?
> 
>             regards, tom lane


-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Jan Wieck <JanWieck@Yahoo.com> writes:
> Yeah, adding a buffer multiple times to the list of unused buffers 
> ensures that it later on gets used for multiple contents simultaneously. 

Hm.  Looking at the patch last night, I was wondering why you had
removed all the guard logic from BufTableInsert and BufTableDelete.
Was that indeed a bad idea?  In particular, the removal of this bit
from BufTableDelete
   /*    * Clear the buffer's tag.  This doesn't matter for the hash table,    * since the buffer is already removed
fromit, but it ensures that    * sequential searches through the buffer table won't think the buffer    * is still
validfor its old page.    */   buf->tag.rnode.relNode = InvalidOid;   buf->tag.rnode.tblNode = InvalidOid;
 

worries me quite a lot, because I *know* that was necessary before.
Have you really changed the search algorithms to the point where it's not?
        regards, tom lane