Thread: Importing data into the database using JDBC
Hi folks, Anyone knows about any Java based tools to import data into the database from CSV, Excel, and other formats? I know about the following resources for reading CSV and Excel: http://jakarta.apache.org/poi/ http://ostermiller.org/utils/CSVLexer.html http://sourceforge.net/projects/csvjdbc http:// www.andykhan.com/jexcelapi/index.html Before writing the tool, I'm wondering if someone knows of higher level tool to grab the data and import it into the database. -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.zapatec.com
Dror, no, I'm not aware of any; but am interested. Do you have some specs for what you want to do? Dave On Fri, 2002-11-01 at 13:40, Dror Matalon wrote: > Hi folks, > > Anyone knows about any Java based tools to import data into the > database from CSV, Excel, and other formats? > > I know about the following resources for reading CSV and Excel: > http://jakarta.apache.org/poi/ > http://ostermiller.org/utils/CSVLexer.html > http://sourceforge.net/projects/csvjdbc > http:// www.andykhan.com/jexcelapi/index.html > > Before writing the tool, I'm wondering if someone knows of higher level > tool to grab the data and import it into the database. > > -- > Dror Matalon > Zapatec Inc > 1700 MLK Way > Berkeley, CA 94709 > http://www.zapatec.com > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html > >
Dave, There's a short description of this in our marketplace at https://www.zapatec.com/website/marketplace/showOneProject.jsp?oid=145072 We're a little bit in hurry with this, so we're probably going to start soon, but would be interested in working with other people once we got some functionality in place. I'm open to the idea of contributing at least some of the code we'll write. Error handling for this stuff is going to be a pain. There are a bunch of error issues that we need to think of. I guess we can look and see what other tools do. Can you think of any Postgres specific functionality that would be need to go into this, or do you think it can be a generic SQL/JDBC solution? Dror On Sun, Nov 03, 2002 at 11:14:19PM -0500, Dave Cramer wrote: > Dror, > > no, I'm not aware of any; but am interested. > > Do you have some specs for what you want to do? > > Dave > On Fri, 2002-11-01 at 13:40, Dror Matalon wrote: > > Hi folks, > > > > Anyone knows about any Java based tools to import data into the > > database from CSV, Excel, and other formats? > > > > I know about the following resources for reading CSV and Excel: > > http://jakarta.apache.org/poi/ > > http://ostermiller.org/utils/CSVLexer.html > > http://sourceforge.net/projects/csvjdbc > > http:// www.andykhan.com/jexcelapi/index.html > > > > Before writing the tool, I'm wondering if someone knows of higher level > > tool to grab the data and import it into the database. > > > > -- > > Dror Matalon > > Zapatec Inc > > 1700 MLK Way > > Berkeley, CA 94709 > > http://www.zapatec.com > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 5: Have you checked our extensive FAQ? > > > > http://www.postgresql.org/users-lounge/docs/faq.html > > > > > > > -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.zapatec.com
Dror, Interesting marketplace! The bid's look low? Do you actually get good work at those prices? my 2 cents worth, I would probably write an XML -> JDBC connector, and then have everything else write to XML. There is one postgres option that invalidates the above statement. I think someone hacked in the ability to do a SQL COPY, so if you wanted it to be really fast you would format for the copy command. Dave On Sun, 2002-11-03 at 23:30, Dror Matalon wrote: > Dave, > > There's a short description of this in our marketplace at > https://www.zapatec.com/website/marketplace/showOneProject.jsp?oid=145072 > > We're a little bit in hurry with this, so we're probably going to start > soon, but would be interested in working with other people once we got > some functionality in place. I'm open to the idea of contributing > at least some of the code we'll write. > > Error handling for this stuff is going to be a pain. There are a bunch > of error issues that we need to think of. I guess we can look and see > what other tools do. > > Can you think of any Postgres specific functionality that would be > need to go into this, or do you think it can be a generic SQL/JDBC > solution? > > > Dror > > > > On Sun, Nov 03, 2002 at 11:14:19PM -0500, Dave Cramer wrote: > > Dror, > > > > no, I'm not aware of any; but am interested. > > > > Do you have some specs for what you want to do? > > > > Dave > > On Fri, 2002-11-01 at 13:40, Dror Matalon wrote: > > > Hi folks, > > > > > > Anyone knows about any Java based tools to import data into the > > > database from CSV, Excel, and other formats? > > > > > > I know about the following resources for reading CSV and Excel: > > > http://jakarta.apache.org/poi/ > > > http://ostermiller.org/utils/CSVLexer.html > > > http://sourceforge.net/projects/csvjdbc > > > http:// www.andykhan.com/jexcelapi/index.html > > > > > > Before writing the tool, I'm wondering if someone knows of higher level > > > tool to grab the data and import it into the database. > > > > > > -- > > > Dror Matalon > > > Zapatec Inc > > > 1700 MLK Way > > > Berkeley, CA 94709 > > > http://www.zapatec.com > > > > > > ---------------------------(end of broadcast)--------------------------- > > > TIP 5: Have you checked our extensive FAQ? > > > > > > http://www.postgresql.org/users-lounge/docs/faq.html > > > > > > > > > > > > > > -- > Dror Matalon > Zapatec Inc > 1700 MLK Way > Berkeley, CA 94709 > http://www.zapatec.com > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > >
On Sun, Nov 03, 2002 at 11:43:24PM -0500, Dave Cramer wrote: > Dror, > > Interesting marketplace! The bid's look low? Do you actually get good > work at those prices? Yes they are low, but they've been climbing a bit lately which is something that we've been encouraging. We've just started a month ago with the marketplace and it's a work in progress. Part of the challenge is that new people need to get up to speed with our platform. Still, we are getting some good work. > > my 2 cents worth, I would probably write an XML -> JDBC connector, and > then have everything else write to XML. Yeah, that's a good idea. > > There is one postgres option that invalidates the above statement. I > think someone hacked in the ability to do a SQL COPY, so if you wanted > it to be really fast you would format for the copy command. If you wanted it to be really fast, and you're handling large number of rows, you should probably use something other than JDBC. I suspect that for large scale stuff, JDBC would be too slow because of all the overhead. By the way, where do you need this? Dror > > Dave > On Sun, 2002-11-03 at 23:30, Dror Matalon wrote: > > Dave, > > > > There's a short description of this in our marketplace at > > https://www.zapatec.com/website/marketplace/showOneProject.jsp?oid=145072 > > > > We're a little bit in hurry with this, so we're probably going to start > > soon, but would be interested in working with other people once we got > > some functionality in place. I'm open to the idea of contributing > > at least some of the code we'll write. > > > > Error handling for this stuff is going to be a pain. There are a bunch > > of error issues that we need to think of. I guess we can look and see > > what other tools do. > > > > Can you think of any Postgres specific functionality that would be > > need to go into this, or do you think it can be a generic SQL/JDBC > > solution? > > > > > > Dror > > > > > > > > On Sun, Nov 03, 2002 at 11:14:19PM -0500, Dave Cramer wrote: > > > Dror, > > > > > > no, I'm not aware of any; but am interested. > > > > > > Do you have some specs for what you want to do? > > > > > > Dave > > > On Fri, 2002-11-01 at 13:40, Dror Matalon wrote: > > > > Hi folks, > > > > > > > > Anyone knows about any Java based tools to import data into the > > > > database from CSV, Excel, and other formats? > > > > > > > > I know about the following resources for reading CSV and Excel: > > > > http://jakarta.apache.org/poi/ > > > > http://ostermiller.org/utils/CSVLexer.html > > > > http://sourceforge.net/projects/csvjdbc > > > > http:// www.andykhan.com/jexcelapi/index.html > > > > > > > > Before writing the tool, I'm wondering if someone knows of higher level > > > > tool to grab the data and import it into the database. > > > > > > > > -- > > > > Dror Matalon > > > > Zapatec Inc > > > > 1700 MLK Way > > > > Berkeley, CA 94709 > > > > http://www.zapatec.com > > > > > > > > ---------------------------(end of broadcast)--------------------------- > > > > TIP 5: Have you checked our extensive FAQ? > > > > > > > > http://www.postgresql.org/users-lounge/docs/faq.html > > > > > > > > > > > > > > > > > > > > > -- > > Dror Matalon > > Zapatec Inc > > 1700 MLK Way > > Berkeley, CA 94709 > > http://www.zapatec.com > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > > > > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.zapatec.com