Re: Wanted: jsonb on-disk representation documentation - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Wanted: jsonb on-disk representation documentation
Date
Msg-id 20140507114041.GE22053@awork2.anarazel.de
Whole thread Raw
In response to Re: Wanted: jsonb on-disk representation documentation  (Heikki Linnakangas <hlinnakangas@vmware.com>)
Responses Re: Wanted: jsonb on-disk representation documentation  (Robert Haas <robertmhaas@gmail.com>)
Re: Wanted: jsonb on-disk representation documentation  (Peter Geoghegan <pg@heroku.com>)
Re: Wanted: jsonb on-disk representation documentation  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Hi,

On 2014-05-07 14:20:19 +0300, Heikki Linnakangas wrote:
> So, apart from cleaning up the code, we really need to take a close look at
> the on-disk format now. The code can be cleaned up later, too, but we're
> going to be stuck with the on-disk format forever, so it's critical to get
> that right.

+1

> First, a few observations:

Agreed.

I'd like to add that:
* Imo we need space in jsonb ondisk values to indicate a format version. We won't fully get this right.
* The jentry representation should be changed so it's possible to get the type of a entry without checking individual
types.That'll make code like findJsonbValueFromSuperHeader() (well, whatever you've renamed it to) much easier to read.
Preferrablyso it an have the same values (after shifting/masking) ask the JBE variants. And it needs space for futher
types(or representations thereof).
 
* I wonder if the hash/object pair representation is wise and if it shouldn't be keys combined with offsets first, and
thenthe values. That will make access much faster. And that's what jsonb essentially is about.
 
* I think both arrays and hashes should grow individual structs. With space for additional flags.

* I have doubts of the wisdom of allowing to embed jbvBinary values in JsonbValues. Although that can be changed later
sinceit's not on disk.
 

> I'm going to proceed refactoring those things, which will change the on-disk
> format. It's late in the release cycle - these things really should've been
> cleaned up earlier - but it's important to get the on-disk format right.
> Shout if you have any objections.

I don't think it's likely that beta1 will be binary compatible with the
final version at this point.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Wanted: jsonb on-disk representation documentation
Next
From: Robert Haas
Date:
Subject: Re: proposal: Set effective_cache_size to greater of .conf value, shared_buffers