Thread: Postgresql-fdw

You can build a multicorn fdw http://multicorn.org/

Regards

On May 23, 2016 7:54 AM, "aluka raju" <alukaraju2894@gmail.com> wrote:
>
> I have data storage in flat files (structured and unstructured) . I want to run sql queries on that , so i am looking in to postgresql how to use fdw on the data that i have. I want to prepare an api for running sql queries for the data in files,so i am trying to have separate postgresql engine and write an wrappper for connecting the data that are in files.
>
> Help me out how can i proceed.
>
> Thanks
> aluka
>
> Sent with MailTrack

I have data storage in flat files (structured and unstructured) . I want to run sql queries on that , so i am looking in to postgresql how to use fdw on the data that i have. I want to prepare an api for running sql queries for the data in files,so i am trying to have separate postgresql engine and write an wrappper for connecting the data that are in files.

Help me out how can i proceed.

I'm not sure what you're expecting postgres to do for you here...  Flat unstructured files have to be read sequentially to do /anything/.      even a simple 'select * from someflattable where id=115', it will have to read the whole file to find any and all records with id=115.  

If you want to use postgres to query this data efficiently, you really should import this data into postgres tables, properly indexed for the sorts of queries you wish to do.

-- 
john r pierce, recycling bits in santa cruz

On Sun, May 22, 2016 at 23:38:43 -0700,
  John R Pierce <pierce@hogranch.com> wrote:
>
>If you want to use postgres to query this data efficiently, you really
>should import this data into postgres tables, properly indexed for the
>sorts of queries you wish to do.

And it isn't that hard to script this kind of thing. Postgres' copy command
makes it easy to read csv files. You could trigger the scripts by hand (or
as part of the script that runs the queries) just before running queries, run
them scheduled at what are normally good times to pick up updates or trigger
off file changes.