Re: update on TOAST status' - Mailing list pgsql-hackers
From | JanWieck@t-online.de (Jan Wieck) |
---|---|
Subject | Re: update on TOAST status' |
Date | |
Msg-id | 200007111202.OAA17389@hot.jw.home Whole thread Raw |
In response to | Re: update on TOAST status' (Tom Lane <tgl@sss.pgh.pa.us>) |
Responses |
Re: update on TOAST status'
Re: update on TOAST status' |
List | pgsql-hackers |
Tom Lane wrote: > JanWieck@t-online.de (Jan Wieck) writes: > > ... So do we want to have indices storing plain > > values allways and limit them in the index-tuple size or not? > > I think not: it will be seen as a robustness failure, even (or > especially) if it doesn't happen often. I can see the bug reports now: > "Hey! I tried to insert a long value in my field, and it didn't work! > I thought you'd fixed this bug?" > > You make good arguments that we shouldn't be too concerned about the > speed of access to toasted index values, and I'm willing to accept > that point of view (at least till we have hard evidence about it). > But when I say "it should be bulletproof" I mean it should *work*, > without imposing arbitrary limits on the user. Arbitrary limits are > exactly what we are trying to eliminate. After debugging something I thought was a bug in the toaster, I've found something really causing headaches. TOAST AS IS IS NOT CAPABLE OF HOLDING INDEXED VALUES! It appears that brtee indices (at least) can keep references to old toast values that survive a VACUUM! Seemsthese references live in nodes actually not referring to a heap tuple any more, but used during tree traversal in comparisions. As if an index tuple delete from a btree not necessarily causes the index value to disappear from the btree completely. It'll never be returned by an index scan, but the value is still theresomewhere. Everything is OK with this up to a VACUUM run. The toaster uses SnapShotAny to fetch toast values. So an external value can be fetched by the toaster even if it is already deleted and committed. If he has a reference somewhere, he has allways a share or higher lock on the main relation preventing VACUUM to mangleup the toast relation (I moved toast relation vacuuming into the lock time of the main table recently). But in the above case it is already vacuumed and not present any more. Now the btree traversal needs to compare againsta value, long gone to the bit heaven, and that cannot work with the toast architecture. Seems the designs of btree and toast are colliding. As soon as "text" is toastable, this'll hurt - be warned. AFAICS, we need to detoast values for index inserts allways and have another toaster inside the index access methodsat some day. In the meantime we should decide a safe maximum index tuple size and emit an explanative errormessage on the attempt to insert oversized index entries instead of possibly corrupting the index. Comment! Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com #
pgsql-hackers by date: