Thread: cast bytea to/from bit strings
Dear PostgreSQL developers,
Please find attached a small patch to convert bytea to bit strings and
vice versa.
I used it in order to be able xor md5 results so as to checksum bundles of
tuples together. The MD5 result is an hexa text convertible to bytea with
decode, but then I was stuck...
ISTM that having these types explicitely castable may be useful to others,
hence this small contribution. The cast allows to work on a bytea at the
bit level and to perform bitwise operations.
./src/backend/utils/adt/varbit.c
- add two conversion functions
./src/include/catalog/pg_proc.h
- declare the above functions in the catalog
./src/include/catalog/pg_cast.h
- declare the 4 explicit casts
./src/test/regress/sql/bit.sql
- test all those new casts
./src/test/regress/expected/bit.out
- new regression results
./src/test/regress/expected/opr_sanity.out
- pg figures out that bit and varbit are binary compatible,
which is the case (well, at least I assumed it).
--
Fabien.
Attachment
I am not sure this is of general enough usefulness to be in the backend. Can you add it as a pgfoundry project? --------------------------------------------------------------------------- Fabien COELHO wrote: > > Dear PostgreSQL developers, > > Please find attached a small patch to convert bytea to bit strings and > vice versa. > > I used it in order to be able xor md5 results so as to checksum bundles of > tuples together. The MD5 result is an hexa text convertible to bytea with > decode, but then I was stuck... > > ISTM that having these types explicitely castable may be useful to others, > hence this small contribution. The cast allows to work on a bytea at the > bit level and to perform bitwise operations. > > ./src/backend/utils/adt/varbit.c > - add two conversion functions > > ./src/include/catalog/pg_proc.h > - declare the above functions in the catalog > > ./src/include/catalog/pg_cast.h > - declare the 4 explicit casts > > ./src/test/regress/sql/bit.sql > - test all those new casts > > ./src/test/regress/expected/bit.out > - new regression results > > ./src/test/regress/expected/opr_sanity.out > - pg figures out that bit and varbit are binary compatible, > which is the case (well, at least I assumed it). > > -- > Fabien. Content-Description: [ Attachment, skipping... ] > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org -- Bruce Momjian http://candle.pha.pa.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Dear Bruce, > I am not sure this is of general enough usefulness to be in the backend. Hmm... I think that the inability to convert nearly binary compatible standard types one to the other is a postgresql issue. Even if it is not often useful, the point is completeness and soundness of the type provided by the core. More over I haven't found any work around with decode/encode and other casts functions. Bytea is somehow an isolated type, which makes it not so useful from within the database. > Can you add it as a pgfoundry project? I could do it, but ISTM it is really overkill for two stupid 10 lines functions that deal with internal core types. -- Fabien.
Fabien COELHO wrote: > > Dear Bruce, > > > I am not sure this is of general enough usefulness to be in the backend. > > Hmm... > > I think that the inability to convert nearly binary compatible standard > types one to the other is a postgresql issue. Even if it is not often > useful, the point is completeness and soundness of the type provided by > the core. More over I haven't found any work around with decode/encode and > other casts functions. Bytea is somehow an isolated type, which makes it > not so useful from within the database. > > > Can you add it as a pgfoundry project? > > I could do it, but ISTM it is really overkill for two stupid 10 lines > functions that deal with internal core types. OK, can I get some feedback from others about this patch? -- Bruce Momjian http://candle.pha.pa.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Fabien COELHO wrote:
>> I think that the inability to convert nearly binary compatible standard
>> types one to the other is a postgresql issue. Even if it is not often
>> useful, the point is completeness and soundness of the type provided by
>> the core.
> OK, can I get some feedback from others about this patch?
I think Fabien is way overstating his case here. It's not immediately
obvious that there should be a cast between bit(n) and bytea, and it's
even less obvious that it should be done in a way that exposes the
internal representation of bit(n) as this does. There's no principled
reason for one bit ordering over the other, for example, nor any very
clean way to handle coercions where N isn't a multiple of 8.
I think this request has more to do with a lack of adequate operators
for one type or the other. If we're missing, say, bitwise logical
operators for bytea, then let's add those rather than create a bogus
equivalence between the types.
regards, tom lane
Dear Tom, >>> I think that the inability to convert nearly binary compatible standard >>> types one to the other is a postgresql issue. Even if it is not often >>> useful, the point is completeness and soundness of the type provided by >>> the core. > >> OK, can I get some feedback from others about this patch? > > I think Fabien is way overstating his case here. Maybe. > It's not immediately obvious that there should be a cast between bit(n) > and bytea, and it's even less obvious that it should be done in a way > that exposes the internal representation of bit(n) as this does. Hmmm... I think people guessed it anyway;-) Well, if there is a big/little endian issue, I agree that it is not a good idea. As I cast at the byte level, it seems to me that it should be okay. If so, I see no real harm in having an *explicit* cast allowed, which by nature may be a little destructive, as long as it is reasonnable. > There's no principled reason for one bit ordering over the other, for > example, nor any very clean way to handle coercions where N isn't a > multiple of 8. It could be rejected instead. > I think this request has more to do with a lack of adequate operators > for one type or the other. If we're missing, say, bitwise logical > operators for bytea, then let's add those rather than create a bogus > equivalence between the types. Indeed, what triggers my development for this cast was that I needed a xor on md5 results, which can only be converted to bytea with convert. I could develop a bunch of bitwise operators for bytea, but casting to varbit where they are already available was the quickest path. -- Fabien.