Thread: pgsql: Add general purpose hasing functions to pgbench.

pgsql: Add general purpose hasing functions to pgbench.

From
Teodor Sigaev
Date:
Add general purpose hasing functions to pgbench.

Hashing function is useful for simulating real-world workload in test like
WEB workload, as an example - YCSB benchmarks.

Author: Ildar Musin with minor editorization by me
Reviewed by: Fabien Coelho, me
Discussion: https://www.postgresql.org/message-id/flat/0e8bd39e-dfcd-2879-f88f-272799ad7ef2@postgrespro.ru

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e51a04840a1c45db101686bef0b7025d5014c74b

Modified Files
--------------
doc/src/sgml/ref/pgbench.sgml                |  59 ++++++++++++++--
src/bin/pgbench/exprparse.y                  |  98 +++++++++++++++++++-------
src/bin/pgbench/pgbench.c                    | 101 +++++++++++++++++++++++++--
src/bin/pgbench/pgbench.h                    |   4 +-
src/bin/pgbench/t/001_pgbench_with_server.pl |  11 +++
5 files changed, 239 insertions(+), 34 deletions(-)


Re: pgsql: Add general purpose hasing functions to pgbench.

From
Tom Lane
Date:
Teodor Sigaev <teodor@sigaev.ru> writes:
> Add general purpose hasing functions to pgbench.

The 32-bit members of the buildfarm aren't very happy with this,
eg on dromedary:

cache gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -g -O2 -ansi -I. -I.
-I../../../src/interfaces/libpq-I../../../src/include  -DCOPY_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST   -c -o
command.ocommand.c 
pgbench.c: In function 'getHashFnv1a':
pgbench.c:945: warning: integer constant is too large for 'long' type
pgbench.c:952: warning: integer constant is too large for 'long' type
pgbench.c: In function 'getHashMurmur2':
pgbench.c:967: warning: integer constant is too large for 'long' type
pgbench.c:970: warning: integer constant is too large for 'long' type
pgbench.c:972: warning: integer constant is too large for 'long' type
pgbench.c:975: warning: integer constant is too large for 'long' type
pgbench.c:978: warning: integer constant is too large for 'long' type

Looks to me like the constants need to be written with INT64CONST().
Also, the fact that the added regression test is passing makes me
wonder whether it's actually exercising these functions meaningfully.

            regards, tom lane


Re: pgsql: Add general purpose hasing functions to pgbench.

From
Teodor Sigaev
Date:
will fix,

Tom Lane wrote:
> Teodor Sigaev <teodor@sigaev.ru> writes:
>> Add general purpose hasing functions to pgbench.
> 
> The 32-bit members of the buildfarm aren't very happy with this,
> eg on dromedary:
> 
> cache gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -g -O2 -ansi -I. -I.
-I../../../src/interfaces/libpq-I../../../src/include  -DCOPY_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST   -c -o
command.ocommand.c
 
> pgbench.c: In function 'getHashFnv1a':
> pgbench.c:945: warning: integer constant is too large for 'long' type
> pgbench.c:952: warning: integer constant is too large for 'long' type
> pgbench.c: In function 'getHashMurmur2':
> pgbench.c:967: warning: integer constant is too large for 'long' type
> pgbench.c:970: warning: integer constant is too large for 'long' type
> pgbench.c:972: warning: integer constant is too large for 'long' type
> pgbench.c:975: warning: integer constant is too large for 'long' type
> pgbench.c:978: warning: integer constant is too large for 'long' type
> 
> Looks to me like the constants need to be written with INT64CONST().
> Also, the fact that the added regression test is passing makes me
> wonder whether it's actually exercising these functions meaningfully.
> 
>             regards, tom lane
> 

-- 
Teodor Sigaev                                   E-mail: teodor@sigaev.ru
                                                    WWW: http://www.sigaev.ru/


Re: pgsql: Add general purpose hasing functions to pgbench.

From
Fabien COELHO
Date:
> will fix,

A blind attempt, if it helps.

>> The 32-bit members of the buildfarm aren't very happy with this,
>> eg on dromedary:
>> 
>> cache gcc -Wall -Wmissing-prototypes -Wpointer-arith 
>> -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
>> -Wformat-security -fno-strict-aliasing -fwrapv -g -O2 -ansi -I. -I. 
>> -I../../../src/interfaces/libpq -I../../../src/include 
>> -DCOPY_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST   -c -o command.o 
>> command.c
>> pgbench.c: In function 'getHashFnv1a':
>> pgbench.c:945: warning: integer constant is too large for 'long' type
>> pgbench.c:952: warning: integer constant is too large for 'long' type
>> pgbench.c: In function 'getHashMurmur2':
>> pgbench.c:967: warning: integer constant is too large for 'long' type
>> pgbench.c:970: warning: integer constant is too large for 'long' type
>> pgbench.c:972: warning: integer constant is too large for 'long' type
>> pgbench.c:975: warning: integer constant is too large for 'long' type
>> pgbench.c:978: warning: integer constant is too large for 'long' type
>> 
>> Looks to me like the constants need to be written with INT64CONST().
>> Also, the fact that the added regression test is passing makes me
>> wonder whether it's actually exercising these functions meaningfully.

I think that there is at least some coverage 
(https://coverage.postgresql.org/src/bin/pgbench/pgbench.c.gcov.html)
and the function results are also checked for some values in the perl 
script output.

-- 
Fabien.
Attachment

Re: pgsql: Add general purpose hasing functions to pgbench.

From
Ildar Musin
Date:
Hello all,


22/03/2018 19:31, Fabien COELHO пишет:
>>>
>>> Looks to me like the constants need to be written with INT64CONST().
>>> Also, the fact that the added regression test is passing makes me
>>> wonder whether it's actually exercising these functions meaningfully.
>
> I think that there is at least some coverage
> (https://coverage.postgresql.org/src/bin/pgbench/pgbench.c.gcov.html)
> and the function results are also checked for some values in the perl
> script output.
>
As Fabien said output values for some particular inputs are checked in
tests. The fact that test passed apparently shows that compiler is smart
enough to recognize those constants as 'long long' and use them
accordingly in expressions.

-- 
Ildar Musin
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company