Re: logical changeset generation v6.2 - Mailing list pgsql-hackers

From ktm@rice.edu
Subject Re: logical changeset generation v6.2
Date
Msg-id 20131015151411.GR26805@aart.rice.edu
Whole thread Raw
In response to Re: logical changeset generation v6.2  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On Tue, Oct 15, 2013 at 11:02:39AM -0400, Robert Haas wrote:
> >> goals may be in conflict; we'll have to pick something.
> >
> > Note that parsing COPYs is a major PITA from most languages...
> >
> > Perhaps we should make the default output json instead? With every
> > action terminated by a nullbyte?
> > That's probably easier to parse from various scripting languages than
> > anything else.
> 
> I could go for that.  It's not quite as compact as I might hope, but
> JSON does seem to make people awfully happy.
> 
> -- 
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
> 

Feeding such a JSON stream into a compression algorithm like lz4 or
snappy should result in a pretty compact stream. The latest lz4 updates
also have ability to use a pre-existing dictionary which would really
help remove the redundant pieces.

Regards,
Ken



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: INSERT...ON DUPLICATE KEY LOCK FOR UPDATE
Next
From: Andres Freund
Date:
Subject: Re: INSERT...ON DUPLICATE KEY LOCK FOR UPDATE