Re: [HACKERS] LONG varsize - how to go on - Mailing list pgsql-hackers

> Bruce Momjian wrote:
> 
> > >     I  know  that  I  can  deal with a bunch of deferred patches,
> > >     staying in sync with CURRENT and having new features only  as
> > >     patches.  But  that  worx only as long as I have most catalog
> > >     changes in CURRENT. One single concurrent catalog change  can
> > >     cost me days of work in the worst case otherwise.
> >
> > The Feb 1 date is not set in stone.  If you would prefer March 1, we can
> > look at that option.
> 
>     Let's see how far I can get it in the next 3-4 weeks. Then it
>     should have turned out if it's worth to delay the release for
>     another couple of weeks or not.

Yes, let's see how it goes.

> 
>     Had  an  Idea  just  as  I  wrote the (now deleted) text that
>     appeared here :-)
> 
>     The problem I wanted to write about are  sections  (possible,
>     even  if  I don't know about one I haven't written myself :-)
>     in the code,  where  a  Datum  is  explicitly  or  implicitly
>     casted, to get access to vl_len and vl_dat.

You will find only a few files that access vl_len/vl_dat, and the rest
all use macros.  mkid or whatever indexing you use on the source tree
will do nicely.  The bigger question is going to be handling the VARDATA
entries properly when the relate to system tables.  The scope of how
long that data has to exist may be an issue, and textout() may be the
trick in all those cases.  The only issue there is pfree'ing the string
once your are done with it.


> 
>     Well,  I intend to redefine the varlena struct and rename any
>     macro that deals with it. This way I'll catch any location in
>     the code, that does anything with variable size attributes in
>     a usual way.

Yes, but again, just using mkid or something else will find all of them
quickly.

Setting the compress bit to catch any unusual cases may be interesting,
though I can't see how any routine could get to the varlena data without
accessing the field name or macros.

> 
>     We wanted to use the top 2 bits of vl_len for flags,  leaving
>     us  a theoretical maximum size of 1GB for one single extended
>     attribute value.  Since I want to leave out  the  compression
>     part  for  now, I could set the compression info bit ALLWAYS.
>     That would force any  part  of  the  code,  where  the  above
>     casting  (abuse)  occurs,  to  immediately CRASH at first hit
>     (would allocate or access >=1G of memory and I think this  is
>     enough to trigger a crash somewhere).  If such a setup passes
>     BETA, I'll feel comfortable with it.

Yes, makes sense.  Good thing user applications will never see our long
pointers.  That would be very confusing for them.

I am concerned about your trying to continue development on a snapshot
while we release the 7.0 release.  A single pgindent run will mess you
up terribly.  I can prevent that, but other work will affect you.

I don't want a release to cause you any additional work.  Marc is very
clear on that.

--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026