Thread: 64-bit CommandIds
Hi, attached is our patch against HEAD which enables extending CommandIds to 64-bit. This is for enabling long transactions that really do that much non-read-only work in one transaction. The feature is off by default, you need to --enable-huge-commandid. It fails only one regression test (without_oid) that measures the saved space in 8.3. Also, modifying FirstCommandId to be (1<<32ULL - 4) to early overflow the 32-bit limit) doesn't show any real problem besides the combocid regression failure that explicitly lists cmin/cmax values, which is expected. It was written by Zoltán Böszörményi <zb@cybertec.at> and Hans-Jürgen Schönig <hs@cybertec.at> Best regards, Zoltán Böszörményi -- ---------------------------------- Zoltán Böszörményi Cybertec Schönig & Schönig GmbH http://www.postgresql.at/ diff -dcrpN pgsql.orig/configure pgsql-cid64/configure *** pgsql.orig/configure 2008-03-02 13:44:42.000000000 +0100 --- pgsql-cid64/configure 2008-03-04 16:53:46.000000000 +0100 *************** if test -n "$ac_init_help"; then *** 1349,1354 **** --- 1349,1355 ---- Optional Features: --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] + --enable-huge-commandid enable 64-bit CommandId support --enable-integer-datetimes enable 64-bit integer date/time support --enable-nls[=LANGUAGES] enable Native Language Support --disable-shared do not build shared libraries *************** fi *** 2175,2180 **** --- 2176,2219 ---- # + # 64-bit CommandId + # + echo "$as_me:$LINENO: checking whether to build with 64-bit CommandId support" >&5 + echo $ECHO_N "checking whether to build with 64-bit CommandId support... $ECHO_C" >&6 + + pgac_args="$pgac_args enable_huge_commandid" + + # Check whether --enable-huge-commandid or --disable-huge-commandid was given. + if test "${enable_huge_commandid+set}" = set; then + enableval="$enable_huge_commandid" + + case $enableval in + yes) + + cat >>confdefs.h <<\_ACEOF + #define USE_64BIT_COMMANDID 1 + _ACEOF + + ;; + no) + : + ;; + *) + { { echo "$as_me:$LINENO: error: no argument expected for --enable-huge-commandid option" >&5 + echo "$as_me: error: no argument expected for --enable-huge-commandid option" >&2;} + { (exit 1); exit 1; }; } + ;; + esac + + else + enable_huge_commandid=no + + fi; + + echo "$as_me:$LINENO: result: $enable_huge_commandid" >&5 + echo "${ECHO_T}$enable_huge_commandid" >&6 + + # # 64-bit integer date/time storage (--enable-integer-datetimes) # { echo "$as_me:$LINENO: checking whether to build with 64-bit integer date/time support" >&5 diff -dcrpN pgsql.orig/configure.in pgsql-cid64/configure.in *** pgsql.orig/configure.in 2008-03-02 13:44:43.000000000 +0100 --- pgsql-cid64/configure.in 2008-03-04 16:53:46.000000000 +0100 *************** PGAC_ARG_REQ(with, libs, [ --with- *** 128,133 **** --- 128,142 ---- # + # 64-bit CommandId + # + AC_MSG_CHECKING([whether to build with 64-bit CommandId support]) + PGAC_ARG_BOOL(enable, huge-commandid, no, [ --enable-huge-commandid enable 64-bit CommandId support], + [AC_DEFINE([USE_64BIT_COMMANDID], 1, + [Define to 1 if you want 64-bit CommandId support. (--enable-huge-commandid)])]) + AC_MSG_RESULT([$enable_huge_commandid]) + + # # 64-bit integer date/time storage (--enable-integer-datetimes) # AC_MSG_CHECKING([whether to build with 64-bit integer date/time support]) diff -dcrpN pgsql.orig/doc/src/sgml/installation.sgml pgsql-cid64/doc/src/sgml/installation.sgml *** pgsql.orig/doc/src/sgml/installation.sgml 2008-02-18 13:49:58.000000000 +0100 --- pgsql-cid64/doc/src/sgml/installation.sgml 2008-03-04 17:16:14.000000000 +0100 *************** su - postgres *** 1011,1016 **** --- 1011,1027 ---- </varlistentry> <varlistentry> + <term><option>--enable-huge-commandid</option></term> + <listitem> + <para> + Use 64-bit CommandIds if you are planning to run transactions + consisting of more than 4 billion commands. This is off by default + to save disk space. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><option>--enable-integer-datetimes</option></term> <listitem> <para> diff -dcrpN pgsql.orig/src/backend/access/transam/xact.c pgsql-cid64/src/backend/access/transam/xact.c *** pgsql.orig/src/backend/access/transam/xact.c 2008-01-15 19:56:59.000000000 +0100 --- pgsql-cid64/src/backend/access/transam/xact.c 2008-03-04 16:57:54.000000000 +0100 *************** CommandCounterIncrement(void) *** 592,598 **** --- 592,602 ---- currentCommandId -= 1; ereport(ERROR, (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + #ifdef USE_64BIT_COMMANDID + errmsg("cannot have more than 2^64-1 commands in a transaction"))); + #else errmsg("cannot have more than 2^32-1 commands in a transaction"))); + #endif } currentCommandIdUsed = false; diff -dcrpN pgsql.orig/src/backend/bootstrap/bootstrap.c pgsql-cid64/src/backend/bootstrap/bootstrap.c *** pgsql.orig/src/backend/bootstrap/bootstrap.c 2008-02-18 13:50:00.000000000 +0100 --- pgsql-cid64/src/backend/bootstrap/bootstrap.c 2008-03-04 16:53:46.000000000 +0100 *************** static const struct typinfo TypInfo[] = *** 139,145 **** --- 139,149 ---- F_TIDIN, F_TIDOUT}, {"xid", XIDOID, 0, 4, true, 'i', 'p', F_XIDIN, F_XIDOUT}, + #ifdef USE_64BIT_COMMANDID + {"cid", CIDOID, 0, 8, false, 'd', 'p', + #else {"cid", CIDOID, 0, 4, true, 'i', 'p', + #endif F_CIDIN, F_CIDOUT}, {"int2vector", INT2VECTOROID, INT2OID, -1, false, 'i', 'p', F_INT2VECTORIN, F_INT2VECTOROUT}, diff -dcrpN pgsql.orig/src/backend/catalog/genbki.sh pgsql-cid64/src/backend/catalog/genbki.sh *** pgsql.orig/src/backend/catalog/genbki.sh 2008-01-01 20:45:48.000000000 +0100 --- pgsql-cid64/src/backend/catalog/genbki.sh 2008-03-04 16:53:46.000000000 +0100 *************** for dir in $INCLUDE_DIRS; do *** 114,119 **** --- 114,136 ---- fi done + # Deduce CIDSTORAGELEN from pg_config.h / USE_64BIT_COMMANDID + for dir in $INCLUDE_DIRS; do + if [ -f "$dir/pg_config.h" ]; then + HUGECID=`grep '^#define[ ]*USE_64BIT_COMMANDID' $dir/pg_config.h | $AWK '{ print $3 }'` + break + fi + done + if [ "$HUGECID" == "1" ]; then + CIDSTORAGELEN="8" + CIDSTORAGEALIGN="d" + CIDPASSBYVAL="f" + else + CIDSTORAGELEN="4" + CIDSTORAGEALIGN="i" + CIDPASSBYVAL="t" + fi + # Get BOOTSTRAP_SUPERUSERID from catalog/pg_authid.h for dir in $INCLUDE_DIRS; do if [ -f "$dir/catalog/pg_authid.h" ]; then *************** sed -e "s/;[ ]*$//g" \ *** 164,169 **** --- 181,189 ---- -e "s/(TransactionId/(xid/g" \ -e "s/PGUID/$BOOTSTRAP_SUPERUSERID/g" \ -e "s/NAMEDATALEN/$NAMEDATALEN/g" \ + -e "s/CIDSTORAGELEN/$CIDSTORAGELEN/g" \ + -e "s/CIDSTORAGEALIGN/$CIDSTORAGEALIGN/g" \ + -e "s/CIDPASSBYVAL/$CIDPASSBYVAL/g" \ -e "s/PGNSP/$PG_CATALOG_NAMESPACE/g" \ | $AWK ' # ---------------- diff -dcrpN pgsql.orig/src/backend/catalog/heap.c pgsql-cid64/src/backend/catalog/heap.c *** pgsql.orig/src/backend/catalog/heap.c 2008-01-01 20:45:48.000000000 +0100 --- pgsql-cid64/src/backend/catalog/heap.c 2008-03-04 16:53:46.000000000 +0100 *************** static FormData_pg_attribute a3 = { *** 117,123 **** --- 117,127 ---- static FormData_pg_attribute a4 = { 0, {"cmin"}, CIDOID, 0, sizeof(CommandId), MinCommandIdAttributeNumber, 0, -1, -1, + #ifdef USE_64BIT_COMMANDID + false, 'p', 'd', true, false, false, true, 0 + #else true, 'p', 'i', true, false, false, true, 0 + #endif }; static FormData_pg_attribute a5 = { *************** static FormData_pg_attribute a5 = { *** 129,135 **** --- 133,143 ---- static FormData_pg_attribute a6 = { 0, {"cmax"}, CIDOID, 0, sizeof(CommandId), MaxCommandIdAttributeNumber, 0, -1, -1, + #ifdef USE_64BIT_COMMANDID + false, 'p', 'd', true, false, false, true, 0 + #else true, 'p', 'i', true, false, false, true, 0 + #endif }; /* diff -dcrpN pgsql.orig/src/backend/catalog/Makefile pgsql-cid64/src/backend/catalog/Makefile *** pgsql.orig/src/backend/catalog/Makefile 2008-03-02 13:44:44.000000000 +0100 --- pgsql-cid64/src/backend/catalog/Makefile 2008-03-04 16:55:51.000000000 +0100 *************** postgres.description: postgres.bki ; *** 46,52 **** postgres.shdescription: postgres.bki ; ! postgres.bki: genbki.sh $(POSTGRES_BKI_SRCS) $(top_builddir)/src/include/pg_config_manual.h AWK='$(AWK)' $(SHELL) $< $(pg_includes) --set-version=$(VERSION) -o postgres $(POSTGRES_BKI_SRCS) .PHONY: install-data --- 46,52 ---- postgres.shdescription: postgres.bki ; ! postgres.bki: genbki.sh $(POSTGRES_BKI_SRCS) $(top_builddir)/src/include/pg_config_manual.h $(top_builddir)/src/include/pg_config.h AWK='$(AWK)' $(SHELL) $< $(pg_includes) --set-version=$(VERSION) -o postgres $(POSTGRES_BKI_SRCS) .PHONY: install-data diff -dcrpN pgsql.orig/src/include/catalog/pg_attribute.h pgsql-cid64/src/include/catalog/pg_attribute.h *** pgsql.orig/src/include/catalog/pg_attribute.h 2008-01-01 20:45:56.000000000 +0100 --- pgsql-cid64/src/include/catalog/pg_attribute.h 2008-03-04 16:53:46.000000000 +0100 *************** DATA(insert ( 1247 typdefault 25 -1 -1 *** 278,286 **** DATA(insert ( 1247 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1247 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1247 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1247 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1247 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1247 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1247 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- --- 278,286 ---- DATA(insert ( 1247 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1247 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1247 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1247 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1247 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1247 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1247 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- *************** DATA(insert ( 1255 proacl 1034 -1 -1 *** 334,342 **** DATA(insert ( 1255 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1255 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1255 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1255 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1255 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1255 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1255 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- --- 334,342 ---- DATA(insert ( 1255 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1255 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1255 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1255 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1255 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1255 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1255 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- *************** DATA(insert ( 1249 attinhcount 23 -1 4 *** 382,390 **** DATA(insert ( 1249 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); /* no OIDs in pg_attribute */ DATA(insert ( 1249 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1249 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1249 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1249 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1249 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- --- 382,390 ---- DATA(insert ( 1249 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); /* no OIDs in pg_attribute */ DATA(insert ( 1249 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1249 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1249 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1249 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1249 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- *************** DATA(insert ( 1259 reloptions 1009 -1 *** 450,458 **** DATA(insert ( 1259 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1259 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1259 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1259 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1259 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1259 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1259 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- --- 450,458 ---- DATA(insert ( 1259 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1259 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1259 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1259 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1259 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1259 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1259 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- diff -dcrpN pgsql.orig/src/include/catalog/pg_type.h pgsql-cid64/src/include/catalog/pg_type.h *** pgsql.orig/src/include/catalog/pg_type.h 2008-01-01 20:45:57.000000000 +0100 --- pgsql-cid64/src/include/catalog/pg_type.h 2008-03-04 16:59:27.000000000 +0100 *************** DATA(insert OID = 28 ( xid PGNSP PGU *** 314,320 **** DESCR("transaction id"); #define XIDOID 28 ! DATA(insert OID = 29 ( cid PGNSP PGUID 4 t b t \054 0 0 1012 cidin cidout cidrecv cidsend - - - i pf 0 -1 0 _null_ _null_ )); DESCR("command identifier type, sequence in transaction id"); #define CIDOID 29 --- 314,320 ---- DESCR("transaction id"); #define XIDOID 28 ! DATA(insert OID = 29 ( cid PGNSP PGUID CIDSTORAGELEN CIDPASSBYVAL b t \054 0 0 1012 cidin cidout cidrecvcidsend - - - CIDSTORAGEALIGN p f 0 -1 0 _null_ _null_ )); DESCR("command identifier type, sequence in transaction id"); #define CIDOID 29 diff -dcrpN pgsql.orig/src/include/c.h pgsql-cid64/src/include/c.h *** pgsql.orig/src/include/c.h 2008-03-02 13:44:45.000000000 +0100 --- pgsql-cid64/src/include/c.h 2008-03-04 17:42:26.000000000 +0100 *************** typedef TransactionId MultiXactId; *** 382,388 **** --- 382,392 ---- typedef uint32 MultiXactOffset; + #ifdef USE_64BIT_COMMANDID + typedef uint64 CommandId; + #else typedef uint32 CommandId; + #endif #define FirstCommandId ((CommandId) 0) diff -dcrpN pgsql.orig/src/include/pg_config.h.in pgsql-cid64/src/include/pg_config.h.in *** pgsql.orig/src/include/pg_config.h.in 2008-02-18 13:50:12.000000000 +0100 --- pgsql-cid64/src/include/pg_config.h.in 2008-03-04 16:53:46.000000000 +0100 *************** *** 656,661 **** --- 656,665 ---- */ #undef UINT64_FORMAT + /* Define to 1 if you want 64-bit CommandId support. (--enable-huge-commandid) + */ + #undef USE_64BIT_COMMANDID + /* Define to 1 to build with assertion checks. (--enable-cassert) */ #undef USE_ASSERT_CHECKING diff -dcrpN pgsql.orig/src/include/postgres.h pgsql-cid64/src/include/postgres.h *** pgsql.orig/src/include/postgres.h 2008-01-01 20:45:56.000000000 +0100 --- pgsql-cid64/src/include/postgres.h 2008-03-04 16:53:46.000000000 +0100 *************** typedef Datum *DatumPtr; *** 462,475 **** --- 462,483 ---- * Returns command identifier value of a datum. */ + #ifdef USE_64BIT_COMMANDID + #define DatumGetCommandId(X) ((CommandId) DatumGetInt64(X)) + #else #define DatumGetCommandId(X) ((CommandId) GET_4_BYTES(X)) + #endif /* * CommandIdGetDatum * Returns datum representation for a command identifier. */ + #ifdef USE_64BIT_COMMANDID + #define CommandIdGetDatum(X) ((Datum) Int64GetDatum(X)) + #else #define CommandIdGetDatum(X) ((Datum) SET_4_BYTES(X)) + #endif /* * DatumGetPointer
Zoltan Boszormenyi wrote: > attached is our patch against HEAD which enables extending CommandIds > to 64-bit. This is for enabling long transactions that really do that much > non-read-only work in one transaction. I think you should add a pg_control field and corresponding check, to avoid a 64bit-Cid postmaster to start on a 32bit-Cid data area and vice versa. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera írta: > Zoltan Boszormenyi wrote: > > >> attached is our patch against HEAD which enables extending CommandIds >> to 64-bit. This is for enabling long transactions that really do that much >> non-read-only work in one transaction. >> > > I think you should add a pg_control field and corresponding check, to > avoid a 64bit-Cid postmaster to start on a 32bit-Cid data area and vice > versa. > You're right, thanks. -- ---------------------------------- Zoltán Böszörményi Cybertec Schönig & Schönig GmbH http://www.postgresql.at/
Alvaro Herrera írta: > Zoltan Boszormenyi wrote: > > >> attached is our patch against HEAD which enables extending CommandIds >> to 64-bit. This is for enabling long transactions that really do that much >> non-read-only work in one transaction. >> > > I think you should add a pg_control field and corresponding check, to > avoid a 64bit-Cid postmaster to start on a 32bit-Cid data area and vice > versa. > I added the check but I needed to add it BEFORE checking for toast_max_chunk_size otherwise it complained about this more cryptic problem. I think it's cleaner to report this failure to know why toast_max_chunk_size != TOAST_MAX_CHUNK_SIZE. Best regards, Zoltán Böszörményi -- ---------------------------------- Zoltán Böszörményi Cybertec Schönig & Schönig GmbH http://www.postgresql.at/ diff -dcrpN pgsql.orig/configure pgsql-cid64/configure *** pgsql.orig/configure 2008-03-02 13:44:42.000000000 +0100 --- pgsql-cid64/configure 2008-03-04 16:53:46.000000000 +0100 *************** if test -n "$ac_init_help"; then *** 1349,1354 **** --- 1349,1355 ---- Optional Features: --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] + --enable-huge-commandid enable 64-bit CommandId support --enable-integer-datetimes enable 64-bit integer date/time support --enable-nls[=LANGUAGES] enable Native Language Support --disable-shared do not build shared libraries *************** fi *** 2175,2180 **** --- 2176,2219 ---- # + # 64-bit CommandId + # + echo "$as_me:$LINENO: checking whether to build with 64-bit CommandId support" >&5 + echo $ECHO_N "checking whether to build with 64-bit CommandId support... $ECHO_C" >&6 + + pgac_args="$pgac_args enable_huge_commandid" + + # Check whether --enable-huge-commandid or --disable-huge-commandid was given. + if test "${enable_huge_commandid+set}" = set; then + enableval="$enable_huge_commandid" + + case $enableval in + yes) + + cat >>confdefs.h <<\_ACEOF + #define USE_64BIT_COMMANDID 1 + _ACEOF + + ;; + no) + : + ;; + *) + { { echo "$as_me:$LINENO: error: no argument expected for --enable-huge-commandid option" >&5 + echo "$as_me: error: no argument expected for --enable-huge-commandid option" >&2;} + { (exit 1); exit 1; }; } + ;; + esac + + else + enable_huge_commandid=no + + fi; + + echo "$as_me:$LINENO: result: $enable_huge_commandid" >&5 + echo "${ECHO_T}$enable_huge_commandid" >&6 + + # # 64-bit integer date/time storage (--enable-integer-datetimes) # { echo "$as_me:$LINENO: checking whether to build with 64-bit integer date/time support" >&5 diff -dcrpN pgsql.orig/configure.in pgsql-cid64/configure.in *** pgsql.orig/configure.in 2008-03-02 13:44:43.000000000 +0100 --- pgsql-cid64/configure.in 2008-03-04 16:53:46.000000000 +0100 *************** PGAC_ARG_REQ(with, libs, [ --with- *** 128,133 **** --- 128,142 ---- # + # 64-bit CommandId + # + AC_MSG_CHECKING([whether to build with 64-bit CommandId support]) + PGAC_ARG_BOOL(enable, huge-commandid, no, [ --enable-huge-commandid enable 64-bit CommandId support], + [AC_DEFINE([USE_64BIT_COMMANDID], 1, + [Define to 1 if you want 64-bit CommandId support. (--enable-huge-commandid)])]) + AC_MSG_RESULT([$enable_huge_commandid]) + + # # 64-bit integer date/time storage (--enable-integer-datetimes) # AC_MSG_CHECKING([whether to build with 64-bit integer date/time support]) diff -dcrpN pgsql.orig/doc/src/sgml/installation.sgml pgsql-cid64/doc/src/sgml/installation.sgml *** pgsql.orig/doc/src/sgml/installation.sgml 2008-02-18 13:49:58.000000000 +0100 --- pgsql-cid64/doc/src/sgml/installation.sgml 2008-03-04 17:16:14.000000000 +0100 *************** su - postgres *** 1011,1016 **** --- 1011,1027 ---- </varlistentry> <varlistentry> + <term><option>--enable-huge-commandid</option></term> + <listitem> + <para> + Use 64-bit CommandIds if you are planning to run transactions + consisting of more than 4 billion commands. This is off by default + to save disk space. + </para> + </listitem> + </varlistentry> + + <varlistentry> <term><option>--enable-integer-datetimes</option></term> <listitem> <para> diff -dcrpN pgsql.orig/src/backend/access/transam/xact.c pgsql-cid64/src/backend/access/transam/xact.c *** pgsql.orig/src/backend/access/transam/xact.c 2008-01-15 19:56:59.000000000 +0100 --- pgsql-cid64/src/backend/access/transam/xact.c 2008-03-04 16:57:54.000000000 +0100 *************** CommandCounterIncrement(void) *** 592,598 **** --- 592,602 ---- currentCommandId -= 1; ereport(ERROR, (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED), + #ifdef USE_64BIT_COMMANDID + errmsg("cannot have more than 2^64-1 commands in a transaction"))); + #else errmsg("cannot have more than 2^32-1 commands in a transaction"))); + #endif } currentCommandIdUsed = false; diff -dcrpN pgsql.orig/src/backend/access/transam/xlog.c pgsql-cid64/src/backend/access/transam/xlog.c *** pgsql.orig/src/backend/access/transam/xlog.c 2008-02-18 13:50:00.000000000 +0100 --- pgsql-cid64/src/backend/access/transam/xlog.c 2008-03-04 21:39:30.000000000 +0100 *************** WriteControlFile(void) *** 3794,3799 **** --- 3794,3805 ---- ControlFile->enableIntTimes = FALSE; #endif + #ifdef USE_64BIT_COMMANDID + ControlFile->enable64bitCommandId = TRUE; + #else + ControlFile->enable64bitCommandId = FALSE; + #endif + ControlFile->localeBuflen = LOCALE_NAME_BUFLEN; localeptr = setlocale(LC_COLLATE, NULL); if (!localeptr) *************** ReadControlFile(void) *** 3989,3994 **** --- 3995,4017 ---- " but the server was compiled with INDEX_MAX_KEYS %d.", ControlFile->indexMaxKeys, INDEX_MAX_KEYS), errhint("It looks like you need to recompile or initdb."))); + + #ifdef USE_64BIT_COMMANDID + if (ControlFile->enable64bitCommandId != TRUE) + ereport(FATAL, + (errmsg("database files are incompatible with server"), + errdetail("The database cluster was initialized without USE_64BIT_COMMANDID" + " but the server was compiled with USE_64BIT_COMMANDID."), + errhint("It looks like you need to recompile or initdb."))); + #else + if (ControlFile->enable64bitCommandId != FALSE) + ereport(FATAL, + (errmsg("database files are incompatible with server"), + errdetail("The database cluster was initialized with USE_64BIT_COMMANDID" + " but the server was compiled without USE_64BIT_COMMANDID."), + errhint("It looks like you need to recompile or initdb."))); + #endif + if (ControlFile->toast_max_chunk_size != TOAST_MAX_CHUNK_SIZE) ereport(FATAL, (errmsg("database files are incompatible with server"), diff -dcrpN pgsql.orig/src/backend/bootstrap/bootstrap.c pgsql-cid64/src/backend/bootstrap/bootstrap.c *** pgsql.orig/src/backend/bootstrap/bootstrap.c 2008-02-18 13:50:00.000000000 +0100 --- pgsql-cid64/src/backend/bootstrap/bootstrap.c 2008-03-04 16:53:46.000000000 +0100 *************** static const struct typinfo TypInfo[] = *** 139,145 **** --- 139,149 ---- F_TIDIN, F_TIDOUT}, {"xid", XIDOID, 0, 4, true, 'i', 'p', F_XIDIN, F_XIDOUT}, + #ifdef USE_64BIT_COMMANDID + {"cid", CIDOID, 0, 8, false, 'd', 'p', + #else {"cid", CIDOID, 0, 4, true, 'i', 'p', + #endif F_CIDIN, F_CIDOUT}, {"int2vector", INT2VECTOROID, INT2OID, -1, false, 'i', 'p', F_INT2VECTORIN, F_INT2VECTOROUT}, diff -dcrpN pgsql.orig/src/backend/catalog/genbki.sh pgsql-cid64/src/backend/catalog/genbki.sh *** pgsql.orig/src/backend/catalog/genbki.sh 2008-01-01 20:45:48.000000000 +0100 --- pgsql-cid64/src/backend/catalog/genbki.sh 2008-03-04 16:53:46.000000000 +0100 *************** for dir in $INCLUDE_DIRS; do *** 114,119 **** --- 114,136 ---- fi done + # Deduce CIDSTORAGELEN from pg_config.h / USE_64BIT_COMMANDID + for dir in $INCLUDE_DIRS; do + if [ -f "$dir/pg_config.h" ]; then + HUGECID=`grep '^#define[ ]*USE_64BIT_COMMANDID' $dir/pg_config.h | $AWK '{ print $3 }'` + break + fi + done + if [ "$HUGECID" == "1" ]; then + CIDSTORAGELEN="8" + CIDSTORAGEALIGN="d" + CIDPASSBYVAL="f" + else + CIDSTORAGELEN="4" + CIDSTORAGEALIGN="i" + CIDPASSBYVAL="t" + fi + # Get BOOTSTRAP_SUPERUSERID from catalog/pg_authid.h for dir in $INCLUDE_DIRS; do if [ -f "$dir/catalog/pg_authid.h" ]; then *************** sed -e "s/;[ ]*$//g" \ *** 164,169 **** --- 181,189 ---- -e "s/(TransactionId/(xid/g" \ -e "s/PGUID/$BOOTSTRAP_SUPERUSERID/g" \ -e "s/NAMEDATALEN/$NAMEDATALEN/g" \ + -e "s/CIDSTORAGELEN/$CIDSTORAGELEN/g" \ + -e "s/CIDSTORAGEALIGN/$CIDSTORAGEALIGN/g" \ + -e "s/CIDPASSBYVAL/$CIDPASSBYVAL/g" \ -e "s/PGNSP/$PG_CATALOG_NAMESPACE/g" \ | $AWK ' # ---------------- diff -dcrpN pgsql.orig/src/backend/catalog/heap.c pgsql-cid64/src/backend/catalog/heap.c *** pgsql.orig/src/backend/catalog/heap.c 2008-01-01 20:45:48.000000000 +0100 --- pgsql-cid64/src/backend/catalog/heap.c 2008-03-04 16:53:46.000000000 +0100 *************** static FormData_pg_attribute a3 = { *** 117,123 **** --- 117,127 ---- static FormData_pg_attribute a4 = { 0, {"cmin"}, CIDOID, 0, sizeof(CommandId), MinCommandIdAttributeNumber, 0, -1, -1, + #ifdef USE_64BIT_COMMANDID + false, 'p', 'd', true, false, false, true, 0 + #else true, 'p', 'i', true, false, false, true, 0 + #endif }; static FormData_pg_attribute a5 = { *************** static FormData_pg_attribute a5 = { *** 129,135 **** --- 133,143 ---- static FormData_pg_attribute a6 = { 0, {"cmax"}, CIDOID, 0, sizeof(CommandId), MaxCommandIdAttributeNumber, 0, -1, -1, + #ifdef USE_64BIT_COMMANDID + false, 'p', 'd', true, false, false, true, 0 + #else true, 'p', 'i', true, false, false, true, 0 + #endif }; /* diff -dcrpN pgsql.orig/src/backend/catalog/Makefile pgsql-cid64/src/backend/catalog/Makefile *** pgsql.orig/src/backend/catalog/Makefile 2008-03-02 13:44:44.000000000 +0100 --- pgsql-cid64/src/backend/catalog/Makefile 2008-03-04 16:55:51.000000000 +0100 *************** postgres.description: postgres.bki ; *** 46,52 **** postgres.shdescription: postgres.bki ; ! postgres.bki: genbki.sh $(POSTGRES_BKI_SRCS) $(top_builddir)/src/include/pg_config_manual.h AWK='$(AWK)' $(SHELL) $< $(pg_includes) --set-version=$(VERSION) -o postgres $(POSTGRES_BKI_SRCS) .PHONY: install-data --- 46,52 ---- postgres.shdescription: postgres.bki ; ! postgres.bki: genbki.sh $(POSTGRES_BKI_SRCS) $(top_builddir)/src/include/pg_config_manual.h $(top_builddir)/src/include/pg_config.h AWK='$(AWK)' $(SHELL) $< $(pg_includes) --set-version=$(VERSION) -o postgres $(POSTGRES_BKI_SRCS) .PHONY: install-data diff -dcrpN pgsql.orig/src/include/catalog/pg_attribute.h pgsql-cid64/src/include/catalog/pg_attribute.h *** pgsql.orig/src/include/catalog/pg_attribute.h 2008-01-01 20:45:56.000000000 +0100 --- pgsql-cid64/src/include/catalog/pg_attribute.h 2008-03-04 16:53:46.000000000 +0100 *************** DATA(insert ( 1247 typdefault 25 -1 -1 *** 278,286 **** DATA(insert ( 1247 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1247 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1247 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1247 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1247 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1247 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1247 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- --- 278,286 ---- DATA(insert ( 1247 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1247 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1247 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1247 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1247 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1247 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1247 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- *************** DATA(insert ( 1255 proacl 1034 -1 -1 *** 334,342 **** DATA(insert ( 1255 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1255 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1255 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1255 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1255 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1255 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1255 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- --- 334,342 ---- DATA(insert ( 1255 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1255 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1255 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1255 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1255 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1255 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1255 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- *************** DATA(insert ( 1249 attinhcount 23 -1 4 *** 382,390 **** DATA(insert ( 1249 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); /* no OIDs in pg_attribute */ DATA(insert ( 1249 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1249 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1249 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1249 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1249 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- --- 382,390 ---- DATA(insert ( 1249 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); /* no OIDs in pg_attribute */ DATA(insert ( 1249 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1249 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1249 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1249 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1249 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- *************** DATA(insert ( 1259 reloptions 1009 -1 *** 450,458 **** DATA(insert ( 1259 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1259 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1259 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1259 cmin 29 0 4 -4 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1259 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1259 cmax 29 0 4 -6 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1259 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- --- 450,458 ---- DATA(insert ( 1259 ctid 27 0 6 -1 0 -1 -1 f p s t f f t 0)); DATA(insert ( 1259 oid 26 0 4 -2 0 -1 -1 t p i t f f t 0)); DATA(insert ( 1259 xmin 28 0 4 -3 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1259 cmin 29 0 CIDSTORAGELEN -4 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1259 xmax 28 0 4 -5 0 -1 -1 t p i t f f t 0)); ! DATA(insert ( 1259 cmax 29 0 CIDSTORAGELEN -6 0 -1 -1 CIDPASSBYVAL p CIDSTORAGEALIGN t f f t 0)); DATA(insert ( 1259 tableoid 26 0 4 -7 0 -1 -1 t p i t f f t 0)); /* ---------------- diff -dcrpN pgsql.orig/src/include/catalog/pg_control.h pgsql-cid64/src/include/catalog/pg_control.h *** pgsql.orig/src/include/catalog/pg_control.h 2008-02-18 13:50:12.000000000 +0100 --- pgsql-cid64/src/include/catalog/pg_control.h 2008-03-04 21:14:04.000000000 +0100 *************** typedef struct ControlFileData *** 145,150 **** --- 145,152 ---- char lc_collate[LOCALE_NAME_BUFLEN]; char lc_ctype[LOCALE_NAME_BUFLEN]; + uint32 enable64bitCommandId; + /* CRC of all above ... MUST BE LAST! */ pg_crc32 crc; } ControlFileData; diff -dcrpN pgsql.orig/src/include/catalog/pg_type.h pgsql-cid64/src/include/catalog/pg_type.h *** pgsql.orig/src/include/catalog/pg_type.h 2008-01-01 20:45:57.000000000 +0100 --- pgsql-cid64/src/include/catalog/pg_type.h 2008-03-04 16:59:27.000000000 +0100 *************** DATA(insert OID = 28 ( xid PGNSP PGU *** 314,320 **** DESCR("transaction id"); #define XIDOID 28 ! DATA(insert OID = 29 ( cid PGNSP PGUID 4 t b t \054 0 0 1012 cidin cidout cidrecv cidsend - - - i pf 0 -1 0 _null_ _null_ )); DESCR("command identifier type, sequence in transaction id"); #define CIDOID 29 --- 314,320 ---- DESCR("transaction id"); #define XIDOID 28 ! DATA(insert OID = 29 ( cid PGNSP PGUID CIDSTORAGELEN CIDPASSBYVAL b t \054 0 0 1012 cidin cidout cidrecvcidsend - - - CIDSTORAGEALIGN p f 0 -1 0 _null_ _null_ )); DESCR("command identifier type, sequence in transaction id"); #define CIDOID 29 diff -dcrpN pgsql.orig/src/include/c.h pgsql-cid64/src/include/c.h *** pgsql.orig/src/include/c.h 2008-03-02 13:44:45.000000000 +0100 --- pgsql-cid64/src/include/c.h 2008-03-04 21:05:23.000000000 +0100 *************** typedef TransactionId MultiXactId; *** 382,388 **** --- 382,392 ---- typedef uint32 MultiXactOffset; + #ifdef USE_64BIT_COMMANDID + typedef uint64 CommandId; + #else typedef uint32 CommandId; + #endif #define FirstCommandId ((CommandId) 0) diff -dcrpN pgsql.orig/src/include/pg_config.h.in pgsql-cid64/src/include/pg_config.h.in *** pgsql.orig/src/include/pg_config.h.in 2008-02-18 13:50:12.000000000 +0100 --- pgsql-cid64/src/include/pg_config.h.in 2008-03-04 16:53:46.000000000 +0100 *************** *** 656,661 **** --- 656,665 ---- */ #undef UINT64_FORMAT + /* Define to 1 if you want 64-bit CommandId support. (--enable-huge-commandid) + */ + #undef USE_64BIT_COMMANDID + /* Define to 1 to build with assertion checks. (--enable-cassert) */ #undef USE_ASSERT_CHECKING diff -dcrpN pgsql.orig/src/include/postgres.h pgsql-cid64/src/include/postgres.h *** pgsql.orig/src/include/postgres.h 2008-01-01 20:45:56.000000000 +0100 --- pgsql-cid64/src/include/postgres.h 2008-03-04 16:53:46.000000000 +0100 *************** typedef Datum *DatumPtr; *** 462,475 **** --- 462,483 ---- * Returns command identifier value of a datum. */ + #ifdef USE_64BIT_COMMANDID + #define DatumGetCommandId(X) ((CommandId) DatumGetInt64(X)) + #else #define DatumGetCommandId(X) ((CommandId) GET_4_BYTES(X)) + #endif /* * CommandIdGetDatum * Returns datum representation for a command identifier. */ + #ifdef USE_64BIT_COMMANDID + #define CommandIdGetDatum(X) ((Datum) Int64GetDatum(X)) + #else #define CommandIdGetDatum(X) ((Datum) SET_4_BYTES(X)) + #endif /* * DatumGetPointer
So, is this an option we want for configure? --------------------------------------------------------------------------- Zoltan Boszormenyi wrote: > Alvaro Herrera ?rta: > > Zoltan Boszormenyi wrote: > > > > > >> attached is our patch against HEAD which enables extending CommandIds > >> to 64-bit. This is for enabling long transactions that really do that much > >> non-read-only work in one transaction. > >> > > > > I think you should add a pg_control field and corresponding check, to > > avoid a 64bit-Cid postmaster to start on a 32bit-Cid data area and vice > > versa. > > > > I added the check but I needed to add it BEFORE checking for > toast_max_chunk_size otherwise it complained about this more > cryptic problem. I think it's cleaner to report this failure to know > why toast_max_chunk_size != TOAST_MAX_CHUNK_SIZE. > > Best regards, > Zolt?n B?sz?rm?nyi > > -- > ---------------------------------- > Zolt?n B?sz?rm?nyi > Cybertec Sch?nig & Sch?nig GmbH > http://www.postgresql.at/ > > > -- > Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) > To make changes to your Subscription: > http://mail.postgresql.org/mj/mj_wwwusr?domain=postgresql.org&extra=pgsql-patches -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> So, is this an option we want for configure?
I think the case for it got a whole lot weaker in 8.3, with lazy
consumption of CIDs. If someone had tables big enough to make the
32-bit-CID limit still be a problem despite that fix, I'd think they'd
not be very happy about adding another 4 bytes of tuple header overhead
(or more likely 8 bytes, on the kind of machine where this patch would
make some sense). I don't foresee many people paying that cost rather
than breaking up their transactions.
My feeling is we should avoid the extra complexity, at least till such
time as we see whether there are still any real field complaints about
this with 8.3.
In any case the patch is broken by --enable-float8-byval, and would
need some adjustments to play nice with that.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > So, is this an option we want for configure? > > I think the case for it got a whole lot weaker in 8.3, with lazy > consumption of CIDs. If someone had tables big enough to make the > 32-bit-CID limit still be a problem despite that fix, I'd think they'd > not be very happy about adding another 4 bytes of tuple header overhead > (or more likely 8 bytes, on the kind of machine where this patch would > make some sense). I don't foresee many people paying that cost rather > than breaking up their transactions. > > My feeling is we should avoid the extra complexity, at least till such > time as we see whether there are still any real field complaints about > this with 8.3. > > In any case the patch is broken by --enable-float8-byval, and would > need some adjustments to play nice with that. Agreed. Let's see if we get requests for it in >= 8.3 releases. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > > I think the case for it got a whole lot weaker in 8.3, with lazy > > consumption of CIDs. > > Agreed. Let's see if we get requests for it in >= 8.3 releases. In the original submission message you find this text: : attached is our patch against HEAD which enables extending CommandIds : to 64-bit. This is for enabling long transactions that really do that : much non-read-only work in one transaction. Question for Hans-Juergen and Zoltan: have you tested 8.3 and do you still see the need for this? -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera wrote:
> Bruce Momjian wrote:
>
>
>>> I think the case for it got a whole lot weaker in 8.3, with lazy
>>> consumption of CIDs.
>>>
>> Agreed. Let's see if we get requests for it in >= 8.3 releases.
>>
>
> In the original submission message you find this text:
>
> : attached is our patch against HEAD which enables extending CommandIds
> : to 64-bit. This is for enabling long transactions that really do that
> : much non-read-only work in one transaction.
>
> Question for Hans-Juergen and Zoltan: have you tested 8.3 and do you
> still see the need for this?
>
>
good morning,
i have seen this problem two or three times within the past 2-3 years or
so. so, it can basically happen in the field for some special purpose
applications but i don't see this as an every day problem. it would be
nice to have it in.
we could also go for some special contrib module which could contain a
patch along with some documentation but i am not quite sure how this
fits in there.
we would of course maintain the patch.
many thanks,
hans
--
Cybertec Schönig & Schönig GmbH
PostgreSQL Solutions and Support
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Tel: +43/1/205 10 35 / 340
www.postgresql-support.de, www.postgresql-support.com
Hans-Juergen Schoenig <hs@cybertec.at> writes:
> Alvaro Herrera wrote:
>> Question for Hans-Juergen and Zoltan: have you tested 8.3 and do you
>> still see the need for this?
> i have seen this problem two or three times within the past 2-3 years or
> so. so, it can basically happen in the field for some special purpose
> applications but i don't see this as an every day problem. it would be
> nice to have it in.
So these experiences were pre-8.3, right?
The reason that I'm harping on that is that plpgsql does a
CommandCounterIncrement for each expression it evaluates, whether
or not there's any visible database access. As of 8.3 that won't
cause consumption of CIDs, but before it did. I suspect that in a
lot of real-world scenarios, CID consumption from triggers will be
down by an order of magnitude.
regards, tom lane
Tom Lane wrote:
> Hans-Juergen Schoenig <hs@cybertec.at> writes:
>
>> Alvaro Herrera wrote:
>>
>>> Question for Hans-Juergen and Zoltan: have you tested 8.3 and do you
>>> still see the need for this?
>>>
>
>
>> i have seen this problem two or three times within the past 2-3 years or
>> so. so, it can basically happen in the field for some special purpose
>> applications but i don't see this as an every day problem. it would be
>> nice to have it in.
>>
>
> So these experiences were pre-8.3, right?
>
> The reason that I'm harping on that is that plpgsql does a
> CommandCounterIncrement for each expression it evaluates, whether
> or not there's any visible database access. As of 8.3 that won't
> cause consumption of CIDs, but before it did. I suspect that in a
> lot of real-world scenarios, CID consumption from triggers will be
> down by an order of magnitude.
>
> regards, tom lane
>
we found those problems when we dealt with stored procedures basically
(during huge analysis and data modification transactions - rollup and
materialize stuff, basically). i would think as well that it should have
improved a lot.
i have not tested with 8.3, however.
on 8.2 it took something like 18 hours to reach that threshold - just to
give you an impression.
many thanks,
hans
--
Cybertec Schönig & Schönig GmbH
PostgreSQL Solutions and Support
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Tel: +43/1/205 10 35 / 340
www.postgresql-support.de, www.postgresql-support.com