Thread: pgbulkload - pgbulkload: Add parallel loader for pg_bulkload 3.0.
pgbulkload - pgbulkload: Add parallel loader for pg_bulkload 3.0.
From
itagaki@pgfoundry.org (User Itagaki)
Date:
Log Message: ----------- Add parallel loader for pg_bulkload 3.0. If LOADER=PARALLEL is specified, input file parse and data load are executed in different backends. Parsed tuples are passed through a shared memory queue. Loading would be improved at best twice as faster as loading with single process. Modified Files: -------------- pgbulkload/bin: Makefile (r1.5 -> r1.6) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/bin/Makefile?r1=1.5&r2=1.6) pg_bulkload.c (r1.9 -> r1.10) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/bin/pg_bulkload.c?r1=1.9&r2=1.10) postgresql (r1.3 -> r1.4) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/bin/postgresql?r1=1.3&r2=1.4) pgbulkload/bin/expected: init.out (r1.2 -> r1.3) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/bin/expected/init.out?r1=1.2&r2=1.3) pgbulkload/bin/sql: init.sql (r1.2 -> r1.3) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/bin/sql/init.sql?r1=1.2&r2=1.3) pgbulkload/include: pg_btree.h (r1.3 -> r1.4) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/include/pg_btree.h?r1=1.3&r2=1.4) pg_bulkload.h (r1.7 -> r1.8) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/include/pg_bulkload.h?r1=1.7&r2=1.8) reader.h (r1.2 -> r1.3) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/include/reader.h?r1=1.2&r2=1.3) writer.h (r1.3 -> r1.4) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/include/writer.h?r1=1.3&r2=1.4) pgbulkload/lib: Makefile (r1.6 -> r1.7) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/Makefile?r1=1.6&r2=1.7) pg_btree.c (r1.8 -> r1.9) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pg_btree.c?r1=1.8&r2=1.9) pg_bulkload.c (r1.10 -> r1.11) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pg_bulkload.c?r1=1.10&r2=1.11) pg_bulkload.sql.in (r1.5 -> r1.6) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pg_bulkload.sql.in?r1=1.5&r2=1.6) pg_strutil.c (r1.3 -> r1.4) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pg_strutil.c?r1=1.3&r2=1.4) reader.c (r1.3 -> r1.4) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/reader.c?r1=1.3&r2=1.4) uninstall_pg_bulkload.sql (r1.4 -> r1.5) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/uninstall_pg_bulkload.sql?r1=1.4&r2=1.5) Added Files: ----------- pgbulkload/bin/expected: load_parallel.out (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/bin/expected/load_parallel.out?rev=1.1&content-type=text/x-cvsweb-markup) pgbulkload/bin/sql: load_parallel.sql (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/bin/sql/load_parallel.sql?rev=1.1&content-type=text/x-cvsweb-markup) pgbulkload/lib: parser_binary.c (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/parser_binary.c?rev=1.1&content-type=text/x-cvsweb-markup) parser_csv.c (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/parser_csv.c?rev=1.1&content-type=text/x-cvsweb-markup) parser_tuple.c (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/parser_tuple.c?rev=1.1&content-type=text/x-cvsweb-markup) source.c (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/source.c?rev=1.1&content-type=text/x-cvsweb-markup) writer_buffered.c (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/writer_buffered.c?rev=1.1&content-type=text/x-cvsweb-markup) writer_direct.c (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/writer_direct.c?rev=1.1&content-type=text/x-cvsweb-markup) writer_parallel.c (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/writer_parallel.c?rev=1.1&content-type=text/x-cvsweb-markup) pgbulkload/lib/pgut: pgut-ipc.c (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pgut/pgut-ipc.c?rev=1.1&content-type=text/x-cvsweb-markup) pgut-ipc.h (r1.1) (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pgut/pgut-ipc.h?rev=1.1&content-type=text/x-cvsweb-markup) Removed Files: ------------- pgbulkload/lib: pg_heap_buffered.c (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pg_heap_buffered.c) pg_heap_direct.c (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pg_heap_direct.c) pg_input_csv.c (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pg_input_csv.c) pg_input_fixed.c (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/pg_input_fixed.c) writer.c (http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbulkload/pgbulkload/lib/writer.c)