Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement) - Mailing list pgsql-hackers

From Fabien COELHO
Subject Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement)
Date
Msg-id alpine.DEB.2.02.1306112104420.6293@localhost6.localdomain6
Whole thread Raw
In response to Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement)  (Greg Smith <greg@2ndQuadrant.com>)
List pgsql-hackers
>>   - the tps is global, with a mutex to share the global stochastic process
>>   - there is an adaptation for the "fork" emulation
>>   - I do not know wheter this works with Win32 pthread stuff.
>
> Instead of this complexity,

Well, the mutex impact is very localized in the code. The complexity is 
more linked to the three "thread" implementations intermixed there.

> can we just split the TPS input per client?

Obviously it is possible. Note that it is more logical to do that per 
thread. I did one shared stochastic process because it makes more sense to 
have just one.

> That's all I was thinking of here, not adding a new set of threading issues. 
> If 10000 TPS is requested and there's 10 clients, just set the delay so that 
> each of them targets 1000 TPS.

Ok, so I understand that a mutex is too much!

> I'm guessing it's more accurate to have them all communicate as you've done 
> here, but it seems like a whole class of new bugs and potential bottlenecks 
> could come out of that.

I do not think that there is a performance or locking contension issue: it 
is about getting a mutex for a section which performs one integer add and 
two integer assignements, that is about 3 instructions, to be compared 
with the task of performing database operations over the network. There 
are several orders of magnitudes between those tasks. It would need a more 
than terrible mutex implementation to have any significant impact.

> Whenever someone touches the threading model for pgbench it usually 
> gives a stack of build farm headaches.  Better to avoid those unless 
> there's really a compelling reason to go through that.

I agree that the threading model in pgbench is a mess, mostly because of 
the 3 concurrent implementations intermixed in the code. Getting rid of 
the fork emulation and win32 special handling and only keeping the pthread 
implementation, which seems to be available even on windows, would be a 
relief. I'm not sure if there is still a rationale to have these 3 
implementations, but it ensures a maintenance mess:-(

I'll submit a version without mutex, but ISTM that this one is 
conceptually cleaner, although I'm not sure about what happens on windows.

-- 
Fabien.



pgsql-hackers by date:

Previous
From: Jim Nasby
Date:
Subject: Re: Parallell Optimizer
Next
From: Gavin Flower
Date:
Subject: Re: Parallell Optimizer