bizgres - bizgres: Enhancements and Fixes 1. - Mailing list pgsql-committers

From nwakefield@pgfoundry.org (User Nwakefield)
Subject bizgres - bizgres: Enhancements and Fixes 1.
Date
Msg-id 20060424061425.C5E1986C5FB@pgfoundry.org
Whole thread Raw
List pgsql-committers
Log Message:
-----------
Enhancements and Fixes
1. NIOFileReader has been reworked behind the scenes to solve a severe problem with record delimiters existing within a
field.From now on the character stream is read field by field, with an unsynchronized buffer existing between the file
channeland the reading class. Performance should be comparable, with complete support for multi character delimiters. 
2. FastSimpleDateFormat, had a problem with its internal cache that resulted in fields with the same value matching
eventhough they were not in the same position 
3. Parameter write back from the SQL JOb executor has been migrated from MPP. This allow for SQL jobs to writeback  to
parameterlist. See the IncrementalFetch.xml for an example of its usage. 
4. Added RunNIOFileReaderTestjobs.java, this is a simple JUnit test suite that tests the reader.

Modified Files:
--------------
    bizgres/KETL/src/com/kni/etl/dbutils:
        Upsert.java (r1.3 -> r1.4)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/Upsert.java.diff?r1=1.3&r2=1.4)
        DatabaseColumnDefinition.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/DatabaseColumnDefinition.java.diff?r1=1.2&r2=1.3)
        ColumnDefinition.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/ColumnDefinition.java.diff?r1=1.2&r2=1.3)
    bizgres/KETL/src/com/kni/etl:
        BaseDataItem.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/BaseDataItem.java.diff?r1=1.2&r2=1.3)
        DataItem.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/DataItem.java.diff?r1=1.2&r2=1.3)
        SourceFieldDefinition.java (r1.1 -> r1.2)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SourceFieldDefinition.java.diff?r1=1.1&r2=1.2)
        EngineConstants.java (r1.6 -> r1.7)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/EngineConstants.java.diff?r1=1.6&r2=1.7)
        SQLJob.java (r1.4 -> r1.5)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SQLJob.java.diff?r1=1.4&r2=1.5)
        SQLJobExecutor.java (r1.3 -> r1.4)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SQLJobExecutor.java.diff?r1=1.3&r2=1.4)
        FastInputChannel.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/FastInputChannel.java.diff?r1=1.2&r2=1.3)
    bizgres/KETL/src/com/kni/etl/ketl/writer:
        SubComponentParallelBatchWriter.java (r1.1 -> r1.2)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/SubComponentParallelBatchWriter.java.diff?r1=1.1&r2=1.2)
        JDBCWriter.java (r1.6 -> r1.7)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/JDBCWriter.java.diff?r1=1.6&r2=1.7)
        PGBulkWriter.java (r1.4 -> r1.5)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/PGBulkWriter.java.diff?r1=1.4&r2=1.5)
        ETLWriter.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/ETLWriter.java.diff?r1=1.2&r2=1.3)
        SubComponentBatchRunnerThread.java (r1.1 -> r1.2)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/SubComponentBatchRunnerThread.java.diff?r1=1.1&r2=1.2)
    bizgres/KETL/src/com/kni/etl/stringtools:
        FastSimpleDateFormat.java (r1.1 -> r1.2)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/stringtools/FastSimpleDateFormat.java.diff?r1=1.1&r2=1.2)
    bizgres/KETL/src/com/kni/etl/ketl:
        ETLStep.java (r1.6 -> r1.7)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/ETLStep.java.diff?r1=1.6&r2=1.7)
    bizgres/KETL/src/com/kni/etl/ketl/reader:
        NIOFileReader.java (r1.5 -> r1.6)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/reader/NIOFileReader.java.diff?r1=1.5&r2=1.6)

Added Files:
-----------
    bizgres/KETL/src/com/kni/etl:
        FieldLevelFastInputChannel.java (r1.1)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/FieldLevelFastInputChannel.java?rev=1.1&content-type=text/x-cvsweb-markup)
    bizgres/KETL/src/com/kni/etl/ketl/test:
        RunNIOFileReaderTestJobs.java (r1.1)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/test/RunNIOFileReaderTestJobs.java?rev=1.1&content-type=text/x-cvsweb-markup)

pgsql-committers by date:

Previous
From: nwakefield@pgfoundry.org (User Nwakefield)
Date:
Subject: bizgres - bizgres: New Directory
Next
From: nwakefield@pgfoundry.org (User Nwakefield)
Date:
Subject: bizgres - bizgres: New Directory