Thread: pg_migrator alpha 5 - truncates at 10 M rows
2009.05.09
pg_migrator alpha 5 results
from PostgreSQL 8.3.7 to 8.4cvs
Centos 5.2 x86_64 GNU/Linux
pg_migrator ran without errors.
Of 120 tables, all smaller tables
have the correct rowcount, but all
larger tables are 'truncated' at
around 10 million rows. I haven't
looked at table content.
8.3 8.4cvs| rowcount | rowcount |+-----------+----------+| 7708808 | 7708808 || 10189986 | 9826926 ||
2598808 | 2598808 || 6074119 | 6074119 || 13563559 | 9830128 || 13597969 | 9830114 || 23507096 | 9829964 ||
15517779 | 9805163 || 12618858 | 9829974 || 11127521 | 9829902 || 4728763 | 4728763 || 4728763 | 4728763 ||
61193699 | 9173790 || 1968328 | 1968328 || 53213115 | 9823165 || 31601320 | 9830060 |
One complication: I hadn't noticed
that there were 2 tables with a not yet
installed datatype. These tables were
simply not created by the pg_migrator-run.
I don't know how this influenced the results,
but I'll repeat it in the coming days.
Erik Rijkers
Erik Rijkers wrote: > 2009.05.09 > > pg_migrator alpha 5 results > from PostgreSQL 8.3.7 to 8.4cvs > > Centos 5.2 x86_64 GNU/Linux > > pg_migrator ran without errors. > > Of 120 tables, all smaller tables > have the correct rowcount, but all > larger tables are 'truncated' at > around 10 million rows. I haven't > looked at table content. Can you please show SELECT relname, pg_relation_size(oid) FROM pg_class for your tables? I'm wondering that it could have forgotten to migrate the later table segments ... -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
On Sun, May 10, 2009 02:05, Alvaro Herrera wrote: > Erik Rijkers wrote: >> 2009.05.09 >> >> pg_migrator alpha 5 results >> from PostgreSQL 8.3.7 to 8.4cvs >> >> Centos 5.2 x86_64 GNU/Linux >> >> pg_migrator ran without errors. >> >> Of 120 tables, all smaller tables >> have the correct rowcount, but all >> larger tables are 'truncated' at >> around 10 million rows. I haven't >> looked at table content. > > Can you please show > SELECT relname, pg_relation_size(oid) FROM pg_class > for your tables? > > I'm wondering that it could have forgotten to migrate the later table > segments ... > In the following list, I have put count(*) of each table next to the pg_relation_size(oid) from pg_class: It seems al 'truncated' tables give pg_relation_size(oid) = 1073741824 (not the same subset as in previous mail) | v 8.3.7 | 8.4 cvs head | count(*) | pg_relation_size(oid) | count(*) | pg_relation_size(oid) +----------+----------------------- ---+----------+---------------------- | 15 | 8192 | 15 | 8192 | 7708808 | 842334208 | 7708808 | 842334208 | 10189986 | 1113432064 | 9826926 | 1073741824 | 2598808 | 284360704 | 2598808 | 284360704 | 6074119 | 663576576 | 6074119 | 663576576 | 13563559 | 1481555968 | 9830128 | 1073741824 | 13597969 | 1485307904 | 9830114 | 1073741824 | 23507096 | 2567831552 | 9829964 | 1073741824 | 15517779 | 1699192832 | 9805163 | 1073741824 | 12618858 | 1378377728 | 9829974 | 1073741824 | 11127521 | 1215488000 | 9829902 | 1073741824 | 56 | 16384 | 56 | 16384 | 12667465 | 1278066688 | 10647510 | 1073741824 | 4728763 | 1623310336 | 4728763 | 553402368 | 4728763 | 553402368 | 4728763 | 553402368 | 10 | 8192 | 10 | 8192 | 100 | 16384 | 100 | 16384 | 13843420 | 1620082688 | 9175040 | 1073741824 | 1523927 | 215244800 | 1523927 | 215244800 | 1614576 | 219340800 | 1614576 | 219340800 | 61193699 | 7162437632 | 9173790 | 1073741824 | 1968328 | 215367680 | 1968328 | 215367680 | 53213115 | 5817516032 | 9823165 | 1073741824 | 31462058 | 3440214016 | 9821425 | 1073741824 | 31601320 | 3451863040 | 9830060 | 1073741824 hth, Erik Rijkers
"Erik Rijkers" <er@xs4all.nl> writes:
> On Sun, May 10, 2009 02:05, Alvaro Herrera wrote:
>> I'm wondering that it could have forgotten to migrate the later table
>> segments ...
> It seems al 'truncated' tables give
> pg_relation_size(oid) = 1073741824
Looks like Alvaro nailed it. Bruce, you reading?
regards, tom lane
Tom Lane wrote: > "Erik Rijkers" <er@xs4all.nl> writes: > > On Sun, May 10, 2009 02:05, Alvaro Herrera wrote: > >> I'm wondering that it could have forgotten to migrate the later table > >> segments ... > > > It seems al 'truncated' tables give > > pg_relation_size(oid) = 1073741824 > > Looks like Alvaro nailed it. Bruce, you reading? Yes, will research; thanks. I know we have multi-segment code, but obviously it is not working reliably. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Tom Lane wrote: > > "Erik Rijkers" <er@xs4all.nl> writes: > > > On Sun, May 10, 2009 02:05, Alvaro Herrera wrote: > > >> I'm wondering that it could have forgotten to migrate the later table > > >> segments ... > > > > > It seems al 'truncated' tables give > > > pg_relation_size(oid) = 1073741824 > > > > Looks like Alvaro nailed it. Bruce, you reading? > > Yes, will research; thanks. I know we have multi-segment code, but > obviously it is not working reliably. I have fixed pg_migrator's multi-segment code and have released an alpha-6 that fixes all known bugs: http://pgfoundry.org/frs/?group_id=1000235 -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Erik Rijkers wrote: > One complication: I hadn't noticed > that there were 2 tables with a not yet > installed datatype. These tables were > simply not created by the pg_migrator-run. > > I don't know how this influenced the results, > but I'll repeat it in the coming days. I have modified pg_migrator so that an error on schema restore will be displayed to the user and pg_migrator will exit with an error. FYI, the multi-segment bug you reported is fixed in a new alpha-6 release too. Thanks for the testing and the report. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +