Thread: PATCH: pgbench / int64 instead of int for xact count
Hi, I've been running a few longer pgbench tests (~week), and I've run into this: transaction type: SELECT only scaling factor: 1250 query mode: simple number of clients: 32 number of threads: 4 duration: 605000 s number of transactions actually processed: -1785047856 latency average: -10.846 ms tps = -2950.492090 (including connections establishing) tps = -2950.492325 (excluding connections establishing) The instance was doing ~10k tps for a week, which caused overflow of the int counter, used to track number of transactions. Hence the negative values. I think we've reached the time when hardeare capable of doing this is pretty common (SSDs, ...), so I think it's time to switch the counter to int64. Tomas
Attachment
Hi,
On 2014-05-25 18:05:03 +0200, Tomas Vondra wrote:
> I've been running a few longer pgbench tests (~week), and I've run into
> this:
> number of transactions actually processed: -1785047856
> latency average: -10.846 ms
> tps = -2950.492090 (including connections establishing)
> tps = -2950.492325 (excluding connections establishing)
>
> The instance was doing ~10k tps for a week, which caused overflow of the
> int counter, used to track number of transactions. Hence the negative
> values.
> I think we've reached the time when hardeare capable of doing this is
> pretty common (SSDs, ...), so I think it's time to switch the counter to
> int64.
Especially when it's perfectly possible to do 500k read only
transactions a second...
> printf("number of transactions per client: %d\n", nxacts);
> - printf("number of transactions actually processed: %d/%d\n",
> + printf("number of transactions actually processed: %ld/%d\n",
> normal_xacts, nxacts * nclients);
That's not right though. On windows a long (indicated by the %l) is only
4 bytes wide. Check INT64_FORMAT. That's generated by configure/platform
template files and should always be correct.
Greetings,
Andres Freund
-- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services
On 25.5.2014 19:05, Andres Freund wrote:
>> printf("number of transactions per client: %d\n", nxacts);
>> - printf("number of transactions actually processed: %d/%d\n",
>> + printf("number of transactions actually processed: %ld/%d\n",
>> normal_xacts, nxacts * nclients);
>
> That's not right though. On windows a long (indicated by the %l) is only
> 4 bytes wide. Check INT64_FORMAT. That's generated by configure/platform
> template files and should always be correct.
Oh, right. v2 of the patch attached.
Tomas
Attachment
Tomas Vondra <tv@fuzzy.cz> writes:
> On 25.5.2014 19:05, Andres Freund wrote:
>> That's not right though. On windows a long (indicated by the %l) is only
>> 4 bytes wide. Check INT64_FORMAT. That's generated by configure/platform
>> template files and should always be correct.
> Oh, right. v2 of the patch attached.
Hm ... if normal_xacts can overflow an int, shouldn't we expect that
the product nxacts * nclients could?
regards, tom lane
On 25.5.2014 20:32, Tom Lane wrote: > Tomas Vondra <tv@fuzzy.cz> writes: >> On 25.5.2014 19:05, Andres Freund wrote: >>> That's not right though. On windows a long (indicated by the %l) is only >>> 4 bytes wide. Check INT64_FORMAT. That's generated by configure/platform >>> template files and should always be correct. > >> Oh, right. v2 of the patch attached. > > Hm ... if normal_xacts can overflow an int, shouldn't we expect that > the product nxacts * nclients could? Maybe, but I saw that as a separate thing, mostly unrelated to the 'hardware got so fast it can overflow' problem. Because nxacts is the parameter we use to define duration, so if it din't overflow in the past, it won't overflow today. OTOH, thanks to hardware improvements we tend to use more clients / higher number of transactions, so fixing this seems like a good idea. Should we change the type of nxacts to int64 (thus allowing '-t' with 64-bit integers), or just the overflow in the printf call? I don't find it very practical to use -t with values not fitting into 32-bits (the -T seems better to do that), so I'm inclined to just fix the printf. Patch v3 attached. Tomas
Attachment
Tomas Vondra <tv@fuzzy.cz> writes:
> Should we change the type of nxacts to int64 (thus allowing '-t' with
> 64-bit integers), or just the overflow in the printf call? I don't find
> it very practical to use -t with values not fitting into 32-bits (the -T
> seems better to do that), so I'm inclined to just fix the printf.
I concur: I don't see people setting -t higher than INT_MAX, but a -t
times -c product exceeding INT_MAX seems more plausible.
> Patch v3 attached.
Looks good, pushed.
regards, tom lane