Thread: Base64 decode/encode performance

Base64 decode/encode performance

From
Gaetano Mendola
Date:
Hi,
I have been experimenting with some base64 encoding/decoding implementation.

I find out that the one at http://code.google.com/p/stringencoders is the best
obtaining a 1.3 speedup vs the postgres one.

Do you think is worth to submit a patch that replaces the postgres base64 implementation
with this one?


Regards
Gaetano Mendola


Re: Base64 decode/encode performance

From
"Marko Kreen"
Date:
On 9/10/08, Gaetano Mendola <mendola@gmail.com> wrote:
>  I have been experimenting with some base64 encoding/decoding implementation.
>
>  I find out that the one at http://code.google.com/p/stringencoders is the best
>  obtaining a 1.3 speedup vs the postgres one.
>
>  Do you think is worth to submit a patch that replaces the postgres base64 implementation
>  with this one?

(Note: the b64encode there reads 3 chars at a time, b64decode int32
at a time.)

There are 2 killer problems:

- decode does not seem to handle architectures that segfault on unaligned int32 accesses.
- decode does not allow whitespace in input string.

If those are fixed it's question of if the 1.3x speed if worth more
complex code with big lookup tables.

If you want to optimize, it seems more worthwhile to add additional
loop to current code that reads 3 or 4 chars at a time, before the
current single-char loop.  The decode loop may thus even optimize
to int32 fetching on x86/64 with reasonable compiler.  Handling
whitespace with such code is doable, but will the code be clear enough?

-- 
marko


Re: Base64 decode/encode performance

From
Mark Mielke
Date:
Marko Kreen wrote:
> (Note: the b64encode there reads 3 chars at a time, b64decode int32
> at a time.)
>
> There are 2 killer problems:
>
> - decode does not seem to handle architectures that segfault
>   on unaligned int32 accesses.

Out of curiosity - does this problem exist on any platform for which 
PostgreSQL is currently ported and supported?

Cheers,
mark

-- 
Mark Mielke <mark@mielke.cc>



Re: Base64 decode/encode performance

From
tomas@tuxteam.de
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Sep 10, 2008 at 10:44:00AM -0400, Mark Mielke wrote:
> Marko Kreen wrote:
[...]
>> - decode does not seem to handle architectures that segfault
>>   on unaligned int32 accesses.
>
> Out of curiosity - does this problem exist on any platform for which 
> PostgreSQL is currently ported and supported?

HP PA is one of them. Besides, some others hide that behind a painful
(more than a factor of 1.3 -- we are talking software emulation here)
performance hit. More modeern architectures (Alpha, Itanium, IA-64) take
this route.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIx+ggBcgs9XrR2kYRAha9AJ0Xy6Zg/m76H2H4Uzta3pSXJh/D2gCfS8PF
vpDQMU8gg2BahURgSI97GSk=
=KnIm
-----END PGP SIGNATURE-----


Re: Base64 decode/encode performance

From
Martijn van Oosterhout
Date:
On Wed, Sep 10, 2008 at 10:44:00AM -0400, Mark Mielke wrote:
> >There are 2 killer problems:
> >
> >- decode does not seem to handle architectures that segfault
> >  on unaligned int32 accesses.
>
> Out of curiosity - does this problem exist on any platform for which
> PostgreSQL is currently ported and supported?

It exists on most CPUs actually like Alpha/Sparc/MIPS, just not on
Intel chips, which is why you don't see them very often. Unaligned
accesses do take twice as long to execute though, even on Intel chips.
On some OSes the unaligned access is trapped and emulated by the OS,
which doesn't do much for performance.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.