Thread: Some additions to pg_filedump...

Some additions to pg_filedump...

From
"Darren Reed"
Date:
I did a bit of hacking on pg_filedump to make it slightly more
useful...but it wasn't clear where or to whom the patches should
be sent, so I'm sending an email to pgsql-admins 'cause they're
the kind of folks that probably care about them...

The changes I added are two command line options:
-F <filter> to filter out which items get dumped.
   presently only works on the infomask field.
-T <types> to dump out each item according to either
   int,char,varchar,bool types (yes, I know, that's
   not all of the postgres types.)
-q to supress lots of additional output (quiet)

Now, combined they allow data to be recovered from a situation
where the index is fubar'd but you've still got good data and
you don't want to recover from backup...that said, there is
great danger because pg_filedump doesn't know which item from
the dataset is the most recent/alive one.

If there's further interest, I can add more types but the
warning above holds.  pg_filedump shouldn't be used for what
the above makes easy but I can't see why that shouldn't be
a decision we (and not the db) gets to make...

Cheers,
Darren

http://coombs.anu.edu.au/~avalon/pg_fd_FT.patch.gz

Re: Some additions to pg_filedump...

From
Devrim GÜNDÜZ
Date:
Hi,

On Mon, 2008-04-21 at 13:51 +0200, Darren Reed wrote:
> I did a bit of hacking on pg_filedump to make it slightly more
> useful...but it wasn't clear where or to whom the patches should
> be sent,

pg_filedump is maintained by Red Hat:

http://sources.redhat.com/rhdb/

Current maintainer is Tom, and I CC'ed him, to make sure that he does
not miss this e-mail.

Regards,
--
Devrim GÜNDÜZ , RHCE
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/

Attachment

Re: Some additions to pg_filedump...

From
"Darren Reed"
Date:
On Mon, 21 Apr 2008 23:19:48 +0300, "Devrim GÜNDÜZ"
<devrim@CommandPrompt.com> said:
> Hi,
>
> On Mon, 2008-04-21 at 13:51 +0200, Darren Reed wrote:
> > I did a bit of hacking on pg_filedump to make it slightly more
> > useful...but it wasn't clear where or to whom the patches should
> > be sent,
>
> pg_filedump is maintained by Red Hat:
>
> http://sources.redhat.com/rhdb/
>
> Current maintainer is Tom, and I CC'ed him, to make sure that he does
> not miss this e-mail.

I saw this, but when I went in search of activity, there was nothing.

Look in:
http://sources.redhat.com/ml/rhdb/
http://sources.redhat.com/ml/rhdb-announce/
http://sources.redhat.com/ml/rhdb-cvs/

All of the archives for 2008 are empty...

It looks like "PostgreSQL - Red Hat Edition Project" is a defunct
project.

Darren

Re: Some additions to pg_filedump...

From
Alvaro Herrera
Date:
Darren Reed wrote:

> I saw this, but when I went in search of activity, there was nothing.
>
> Look in:
> http://sources.redhat.com/ml/rhdb/
> http://sources.redhat.com/ml/rhdb-announce/
> http://sources.redhat.com/ml/rhdb-cvs/
>
> All of the archives for 2008 are empty...
>
> It looks like "PostgreSQL - Red Hat Edition Project" is a defunct
> project.

Perhaps it would be apropos for RedHat to donate pg_filedump under BSD
license so we could include it in contrib?  It's not like they're
getting much benefit of it being under GPL.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Re: Some additions to pg_filedump...

From
Devrim GÜNDÜZ
Date:
Hi,

On Wed, 2008-04-23 at 11:55 +0200, Darren Reed wrote:
> I saw this, but when I went in search of activity, there was nothing.

See ChangeLog:

2008-02-08  Tom Lane <tgl@redhat.com>

---

Regards,
--
Devrim GÜNDÜZ , RHCE
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/

Attachment

Re: Some additions to pg_filedump...

From
"Darren Reed"
Date:
On Wed, 23 Apr 2008 09:20:49 -0400, "Alvaro Herrera"
<alvherre@CommandPrompt.com> said:
> Darren Reed wrote:
>
> > I saw this, but when I went in search of activity, there was nothing.
> >
> > Look in:
> > http://sources.redhat.com/ml/rhdb/
> > http://sources.redhat.com/ml/rhdb-announce/
> > http://sources.redhat.com/ml/rhdb-cvs/
> >
> > All of the archives for 2008 are empty...
> >
> > It looks like "PostgreSQL - Red Hat Edition Project" is a defunct
> > project.
>
> Perhaps it would be apropos for RedHat to donate pg_filedump under BSD
> license so we could include it in contrib?  It's not like they're
> getting much benefit of it being under GPL.


If there's interest, why not just write a new version of said
program from scratch so that it can be included in contrib?

And yes, if others are strapped for time, I'm willing to look
into doing this more, soon, now that I realise that GPL is a
problem for wider use.

Darren

Re: Some additions to pg_filedump...

From
Darren Reed
Date:
  Alvaro Herrera wrote:
> Darren Reed wrote:
>
>
>> I saw this, but when I went in search of activity, there was nothing.
>>
>> Look in:
>> http://sources.redhat.com/ml/rhdb/
>> http://sources.redhat.com/ml/rhdb-announce/
>> http://sources.redhat.com/ml/rhdb-cvs/
>>
>> All of the archives for 2008 are empty...
>>
>> It looks like "PostgreSQL - Red Hat Edition Project" is a defunct
>> project.
>>
>
> Perhaps it would be apropos for RedHat to donate pg_filedump under BSD
> license so we could include it in contrib?  It's not like they're
> getting much benefit of it being under GPL.
>

Whether or not that was a hint, I sat down and wrote a program from scratch
to try and dump out only the data files.  I suppose this might be a
version 1.0.

http://coombs.anu.edu.au/~avalon/pg_dumpdata.tgz

Alas, there is a catch...

I couldn't work out the binary format for all of the data types - as an
exercise
I tried to reverse engineer their behaviour from examining what got
stored and
for most of the simple types, I think I've got it right.  I couldn't
fathom "time";
the geometric ones I couldn't easily find examples of how to insert data
- I am
assuming the web documentation is self explanatory to those who use it;
"double"
wasn't a straight fit of "double' from unix (huh?) and "bit" wants a
length to work
which was against what I was trying to achieve.

It might be interesting to write something that tries to deconstruct the
index files,
but to me there's no value in that - the value is in the data.

Cheers,
Darren