Re: pgbench throttling latency limit - Mailing list pgsql-hackers

Hello Heikki,

> I find the definition of the latency limit a bit strange. It's a limit on how 
> late a transaction can *start* compared to it's scheduled starting time, not 
> how long a query is allowed to last.

Yes. This is what can be done easily with pgbench under throttling. Note 
that if transactions take long it is recorded (average, stddev...) so it 
appears elsewhere.

> How do figure out what it should be set to?

It is really just a simple tool to measure unresponsiveness under 
throttling, which I'm testing and complaining about in another thread.

The underlying model I have in mind would some timeout from an 
application, say a web server, or a pooling process which is handling a 
queue of requests...

Now, if I describe that as "lag limit" instead if "latency limit", maybe
it is clearer and better?

> That model might make some sense if you think e.g. of a web application,
> [...]

Yep, that is what I had in mind, but the primary objective is really to 
check whether pg is responsive or not.

> So I think a more useful model is that new queries arrive at a given 
> rate, and each query is expected to finish in X milliseconds from its 
> arrival time (i.e the time the query is scheduled to begin, not the time 
> it was sent to the server) or it's counted as failed. If a transaction 
> cannot even be started by that deadline, because the connection is still 
> busy with the previous query, it's counted as failed without even 
> sending it to the server. With that definition, it makes sense to 
> specify the latency limit even without --rate.

Yep. But that is not what I'm doing here. It would be interesting as well. 
It would be another patch.

> In that case, it's simply a limit on how long each query's 
> execution is allowed to last until it's considered as failed. IOW, each 
> query's scheduled start time is when the previous query ends.

Not under --rate... that is the point of throttling!  Under throttling, 
the latency should really be computed wrt to the schedule start time and 
not the actual start time which may be 10 seconds afterwards when things 
are going bad... Also, there is the question of whether the "failed query" 
is executed or not. Here I'm not executing them, in effect they were 
aborted by the application. With your suggestion they would be executed 
anyway but considered failed.

So what you are suggesting is another (interesting) functionnality, that 
could indeed be named "latency limit" (count slow above a threshold 
queries), what I'm doing here is "lag limit" (scheduled query could not 
start on time and are skipped, this is really specific to --rate).

In the updated patch attached, I changed the explanations, documentation 
and name to "lag limit" instead of "latency limit" to clarify this point. 
It was really a misnommer.

-- 
Fabien.