Re: Variable length varlena headers redux - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: Variable length varlena headers redux
Date
Msg-id 200702100244.l1A2iaI10632@momjian.us
Whole thread Raw
In response to Re: Variable length varlena headers redux  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Variable length varlena headers redux  (Bruce Momjian <bruce@momjian.us>)
List pgsql-hackers
Tom Lane wrote:
> Gregory Stark <stark@enterprisedb.com> writes:
> > That seems like an awful lot of copying and pallocs that aren't there
> > currently though. And it'll make us reluctant to change over frequently used
> > data types like text -- which are precisely the ones that would gain us the
> > most.
> 
> > It seems to me that it might be better to change to storing varlena lengths in
> > network byte order instead. That way we can dedicate the leading bits to toast
> > flags and read more bytes as necessary.
> 
> This'll add its own overhead ... but probably less than pallocs and
> data-copying would.  And I agree we can find (pretty much) all the
> places that need changing by the expedient of deliberately renaming
> the macros and struct fields.

I think we should go with the pallocs and see how it performs.  That is
certainly going to be easier to do, and we can test it pretty easily.

One palloc optimization idea would be to split out the representation so
the length is stored seprately from the data in memory, and we could use
an int32 for the length, and point to the shared buffer for the data. 
However I don't think our macros can handle that so it might be a
non-starter.

However, I think we should find out of the palloc is a problem before
avoiding it.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: HOT for PostgreSQL 8.3
Next
From: Tom Lane
Date:
Subject: Foreign keys for non-default datatypes, redux