Re: bytea memory improvement - Mailing list pgsql-jdbc

Your explanation is very simple and correctly explains both methods.<br /><br /> Here are some more comments ...<br
/><br/> till toenges wrote: <blockquote cite="mid45192600.5050704@kyon.de" type="cite"><pre wrap="">Kris Jurka wrote:
</pre><blockquotetype="cite"><pre wrap="">I'm not super impressed with these timing results.  They are certainly 
 
showing some effects due to GC, consider the rise in time here at 10.5MB.   </pre></blockquote><pre wrap="">
The method isn't neccessarily much faster, especially when there are
only a few megabytes involved. This is very difficult to benchmark in
the presence of a garbage collector.
 </pre><blockquote type="cite"><pre wrap="">I've committed this to CVS HEAD with a rather arbitrarily set 
MAX_3_BUFF_SIZE value of 2MB.  Note that this is also the escaped size, so 
we may actually be dealing with output data a quarter of that size.  If 
anyone could do some more testing of what a good crossover point would be 
that would be a good thing.   </pre></blockquote><pre wrap="">
AFAIK the MAX_3_BUFF_SIZE entry was a debug artifact. Not needed any </pre></blockquote> It's almost correct, I would
likethe new code to be more tested before it fully replaces the old - in large arrays there's a big memory advantage,
soit makes sense to replace, in small array it's almost the same, so the old code can stay for a while ...<br
/><blockquotecite="mid45192600.5050704@kyon.de" type="cite"><pre wrap="">more. The new method is always faster or at
leastas fast as the old
 
method, because it requires fewer memory accesses.

3 Buffers:

Buffer1 zeroing (vm intern)
Buffer1 filling

Buffer2 zeroing (vm intern)
Buffer1 reading
Buffer2 writing

Buffer3 zeroing (vm intern)
Buffer2 reading
Buffer3 writing

Total: 8 memory accesses.

Eventually Buffer3 reading, but that's not part of the driver.


2 Buffers:

Buffer1 zeroing (vm intern)
Buffer1 filling

Buffer1 reading (the new pass)

Buffer2 zeroing (vm intern)
Buffer1 reading
Buffer2 writing

Total: 6 memory accesses.


Conclusion: The new method uses less memory. It must be faster as well,
since everything else is fast in comparison to memory access.

Additionally, it requires only 2 allocations, and memory allocation have
some overhead as well, and mean more work for the garbage collector in
the end. Even if the VM can do some magic to avoid zeroing the buffers,
the newer method has one less memory access. It is always the winner.
 </pre></blockquote> Not all memory accesses are created equal :-), the Buffer1 is the biggest buffer, and the new code
passone more time through it. The last copy from Buffer3 to Buffer2 in the old method is done through System.arraycopy,
whichI think is very, very fast (hardware based), so the methods are more balanced ...<br /> For large arrays the new
isALWAYS much faster off course - due to memory access.<br /><br /><blockquote cite="mid45192600.5050704@kyon.de"
type="cite"><prewrap="">---------------------------(end of broadcast)---------------------------
 
TIP 9: In versions below 8.0, the planner will ignore your desire to      choose an index scan if your joining column's
datatypesdo not      match
 
 </pre></blockquote><br /><br /><div class="moz-signature">-- <br /></div><p><font color="#7da647"><font face="Verdana,
sans-serif"><fontsize="2" style="font-size: 10pt;"> Luis Flores </font></font></font><p><font color="#7da647"><font
face="Verdana,sans-serif"><font size="2" style="font-size: 8pt;"> Analista de Sistemas</font></font></font><p><a
href="http://www.evolute.pt"><fontface="Verdana, sans-serif"><font size="2" style="font-size: 8pt;"><b>Evolute</b> -
ConsultoriaInformática<br /><br /></font></font></a> <font color="#7da647"><font face="Verdana, sans-serif"><font
size="2"style="font-size: 8pt;"> Email: </font></font></font> <a href="mailto:lflores@evolute.pt"><font face="Verdana,
sans-serif"><fontsize="2" style="font-size: 8pt;">lflores@evolute.pt </font></font></a><p><font color="#7da647"><font
face="Verdana,sans-serif"><font size="2" style="font-size: 8pt;"> Tel: (+351) 212949689</font></font></font><div
style="text-align:justify;"><font color="#7d7d7d"><font face="Verdana, sans-serif"><font size="1" style="font-size:
7pt;"><br/> AVISO DE CONFIDENCIALIDADE</font></font></font><br /><font color="#7d7d7d"><font face="Verdana,
sans-serif"><fontsize="1" style="font-size: 7pt;"> Esta mensagem de correio electrónico e eventuais ficheiros anexos
sãoconfidenciais e destinados apenas à(s) pessoa(s) ou entidade(s) acima referida(s), podendo conter informação
privilegiadae confidencial, a qual não poderá ser divulgada, copiada, gravada ou distribuída nos termos da lei vigente.
Casonão seja o destinatário da mensagem, ou se ela lhe foi enviada por engano, agradecemos que não faça uso ou
divulgaçãoda mesma. A distribuição ou utilização da informação nela contida é interdita. Se recebeu esta mensagem por
engano,por favor notifique o remetente e apague este e-mail do seu sistema. Obrigado. <br /></font></font></font><font
color="#7d7d7d"><fontface="Verdana, sans-serif"><font size="1" style="font-size: 7pt;"> </font></font></font><br
/><fontcolor="#7d7d7d"><font face="Verdana, sans-serif"><font size="1" style="font-size: 7pt;"> CONFIDENTIALITY
NOTICE</font></font></font><br/><font color="#7d7d7d"><font face="Verdana, sans-serif"><font size="1" style="font-size:
7pt;">This e-mail transmission and eventual attached files are intended only for the use of the individual(s) or
entity(ies)named above and may contain information that is both privileged and confidential and is exempt from
disclosureunder applicable law. If you are not the intended recipient, you are hereby notified that any disclosure,
copying,distribution or use of any of the information contained in this transmission is strictly restricted. If by any
meansyou have received this transmission in error, please immediately notify the sender and delete this e-mail from
yoursystem. Thank you. </font></font></font></div>