Thread: Units in postgresql.conf
One frequent source of confusion are the different units that the parameters in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB; bgwriter_delay is in milliseconds, checkpoint_warning is in seconds. Obviously, we can't change that without inconveniencing a lot of users. I think it would be useful to allow units to be added to these settings, for example shared_buffers = 1000kB checkpoint_warning = 30s This would also allow shared_buffers = 512MB which is a bit cumbersome to calculate right now (you'd need = 65536). I haven't thought yet how to parse or implement this, but would people find this useful? -- Peter Eisentraut http://developer.postgresql.org/~petere/
On 7/20/06, Peter Eisentraut <peter_e@gmx.net> wrote: > I think it would be useful to allow units to be added to these settings, for > example > > shared_buffers = 1000kB > checkpoint_warning = 30s > > This would also allow > > shared_buffers = 512MB > > which is a bit cumbersome to calculate right now (you'd need = 65536). > > I haven't thought yet how to parse or implement this, but would people find > this useful? I agree, a lot of newbies have issues with the configuration file. I have a tiny bit of code (about 20 lines I think) that will handle K, M, and G suffixes for memory. It would be equally easy to add S for seconds, .... In my code, if no suffix existed, I'd just revert to the default behavior. This is probably what we'd want to do in PostgreSQL as well. The only issue in PostgreSQL is knowing what the unit conversion and scaling factor is for each parameter (8K, 1K, milliseconds, etc); though, this wouldn't be hard to add at all. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation | fax: 732.331.1301 33 Wood Ave S, 2nd Floor | jharris@enterprisedb.com Iselin, New Jersey 08830 | http://www.enterprisedb.com/
On Thu, 20 Jul 2006, Peter Eisentraut wrote: > One frequent source of confusion are the different units that the parameters > in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB; > bgwriter_delay is in milliseconds, checkpoint_warning is in seconds. > Obviously, we can't change that without inconveniencing a lot of users. > > I think it would be useful to allow units to be added to these settings, for > example > > shared_buffers = 1000kB > checkpoint_warning = 30s > > This would also allow > > shared_buffers = 512MB > > which is a bit cumbersome to calculate right now (you'd need = 65536). > > I haven't thought yet how to parse or implement this, but would people find > this useful? Please! Yes! Gavin
On 7/20/06, Peter Eisentraut <peter_e@gmx.net> wrote: > One frequent source of confusion are the different units that the parameters > in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB; > bgwriter_delay is in milliseconds, checkpoint_warning is in seconds. > Obviously, we can't change that without inconveniencing a lot of users. > > I think it would be useful to allow units to be added to these settings, for > example > > shared_buffers = 1000kB > checkpoint_warning = 30s > > This would also allow > > shared_buffers = 512MB > > which is a bit cumbersome to calculate right now (you'd need = 65536). > > I haven't thought yet how to parse or implement this, but would people find > this useful? +1. In addition, that would make conffile self-documenting. -- marko
On Thursday 20 July 2006 05:04, Jonah H. Harris wrote: > On 7/20/06, Peter Eisentraut <peter_e@gmx.net> wrote: > > I think it would be useful to allow units to be added to these settings, > > for example > > > > shared_buffers = 1000kB > > checkpoint_warning = 30s > > > > This would also allow > > > > shared_buffers = 512MB > > > > which is a bit cumbersome to calculate right now (you'd need = 65536). > > > > I haven't thought yet how to parse or implement this, but would people > > find this useful? > > I agree, a lot of newbies have issues with the configuration file. I > have a tiny bit of code (about 20 lines I think) that will handle K, > M, and G suffixes for memory. It would be equally easy to add S for > seconds, .... > > In my code, if no suffix existed, I'd just revert to the default > behavior. This is probably what we'd want to do in PostgreSQL as > well. > > The only issue in PostgreSQL is knowing what the unit conversion and > scaling factor is for each parameter (8K, 1K, milliseconds, etc); > though, this wouldn't be hard to add at all. Yummy, Yummy, I'd say this would be a big boost in ability to tune for a lot of people. -- Darcy Buskermolen Wavefire Technologies Corp. http://www.wavefire.com ph: 250.717.0200 fx: 250.763.1759
Peter Eisentraut wrote: > One frequent source of confusion are the different units that the parameters > in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB; > bgwriter_delay is in milliseconds, checkpoint_warning is in seconds. > Obviously, we can't change that without inconveniencing a lot of users. > > I think it would be useful to allow units to be added to these settings, for > example <snip> > > I haven't thought yet how to parse or implement this, but would people find > this useful? +1 I'd find this useful myself, and I think it would eliminate many mistakes by newer admins. Joe
Josh Berkus wrote: > Well, it's on my TODO list for 8.2 to write a simple postgresql.conf > conversion utility in Perl. If you wanted to make a change like > that, it would make finishing that mandatory. I don't understand how that is related. Or what a conversion utility would be for that matter. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter, > One frequent source of confusion are the different units that the parameters > in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB; > bgwriter_delay is in milliseconds, checkpoint_warning is in seconds. > Obviously, we can't change that without inconveniencing a lot of users. > > I think it would be useful to allow units to be added to these settings, for > example > > shared_buffers = 1000kB > checkpoint_warning = 30s > > This would also allow > > shared_buffers = 512MB > > which is a bit cumbersome to calculate right now (you'd need = 65536). > > I haven't thought yet how to parse or implement this, but would people find > this useful? > Well, it's on my TODO list for 8.2 to write a simple postgresql.conf conversion utility in Perl. If you wanted to make a change like that, it would make finishing that mandatory. Just as well, right now half the vacuum settings are in a different section than another half. --Josh
Peter, > I don't understand how that is related. Or what a conversion utility > would be for that matter. Well, the main issue with changing the units of the PostgreSQL.conf file from a user perspective is that the numbers from you 8.0/8.1 conf file would no longer work. A little conversion utilitily to turn your 8.0 file into an 8.2 file would help solve that. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
On Thu, Jul 20, 2006 at 01:49:36PM +0200, Peter Eisentraut wrote: > One frequent source of confusion are the different units that the parameters > in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB; > bgwriter_delay is in milliseconds, checkpoint_warning is in seconds. > Obviously, we can't change that without inconveniencing a lot of users. > > I think it would be useful to allow units to be added to these settings, for > example > > shared_buffers = 1000kB > checkpoint_warning = 30s > > This would also allow > > shared_buffers = 512MB > > which is a bit cumbersome to calculate right now (you'd need = 65536). > > I haven't thought yet how to parse or implement this, but would > people find this useful? Absolutely! :) Cheers, D -- David Fetter <david@fetter.org> http://fetter.org/ phone: +1 415 235 3778 AIM: dfetter666 Skype: davidfetter Remember to vote!
Peter Eisentraut wrote: > One frequent source of confusion are the different units that the parameters > in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB; > bgwriter_delay is in milliseconds, checkpoint_warning is in seconds. > Obviously, we can't change that without inconveniencing a lot of users. > > I think it would be useful to allow units to be added to these settings, for > example > > shared_buffers = 1000kB > checkpoint_warning = 30s > > This would also allow > > shared_buffers = 512MB > > which is a bit cumbersome to calculate right now (you'd need = 65536). > > I haven't thought yet how to parse or implement this, but would people find > this useful? > It is good idea. I going to implement this. There is some proposal: Time units is easy: 1h = 60min = 3600s = 3600000ms Memory units: What kind of unit prefix will we use for memory? 1kB=1000B and 1kiBi=1024B or 1kB=1024kB. See http://en.wikipedia.org/wiki/Binary_prefix for detail. I suggest use IEC standard convention. By my opinion it is much better. And there are some other questions: 1) will be unit required? What will be default unit for value without unit? I suggest mandatory unit avoid the problem with unclear default value. 2) Each internal representation of setting use different unit. Shell I convert this representation to milliseconds and bytes? I think it is not good idea. It should generate some overflow. I suggest to recompute value and round it to integer. Zdenek
Josh Berkus wrote: > Well, the main issue with changing the units of the PostgreSQL.conf > file from a user perspective is that the numbers from you 8.0/8.1 > conf file would no longer work. No one is intending to do any such change. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Zdenek Kotala wrote: > Time units is easy: > 1h = 60min = 3600s = 3600000ms We don't need anything larger than seconds at the moment. > What kind of unit prefix will we use for memory? PostgreSQL has always used 1 kB = 1024 B. > 1) will be unit required? No. > What will be default unit for value without > unit? What we have now. > I suggest mandatory unit avoid the problem with unclear default > value. Not going to happen. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote:
> Zdenek Kotala wrote:
>> Time units is easy:
>> 1h = 60min = 3600s = 3600000ms
>
> We don't need anything larger than seconds at the moment.
>
>> What kind of unit prefix will we use for memory?
>
> PostgreSQL has always used 1 kB = 1024 B.
>
>> 1) will be unit required?
>
> No.
>
>> What will be default unit for value without
>> unit?
>
> What we have now.
>
>> I suggest mandatory unit avoid the problem with unclear default
>> value.
>
> Not going to happen.
>
Ok. Conclusion is for time s=second, ms=millisecond and for memory B,
kB, MB, GB. Unit is not mandatory and if it will missing the behavior
stays same - backward compatibility (no extra conversion utility).
Last question is if "page" unit should be useful too. For example:
#shared_buffers = 1000 # min 16 or max_connections*2,
8KB each
It means 8000kB. But if somebody compiles postgres with different page
size, than the size will be different. However, somebody should use for
example 8MB and number of buffers will be 8MB/page_size.
Zdenek
PS: I have some GUC patches in the patches queue. Could anybody
test/commit them? I would like continue on the latest version of guc
subsystem. Thanks
Peter Eisentraut wrote: > I think it would be useful to allow units to be added to these settings, for > example... > shared_buffers = 512MB > which is a bit cumbersome to calculate right now (you'd need = 65536). > > I haven't thought yet how to parse or implement this, but would people find > this useful? Would this extend to things like "random_page_cost" and similar? If the random_page_cost were specifiable in seconds or ms it might be easier to someday write a program to measure such values on particular hardware platforms. (though I guess for that to work, the config file would also need to add the reference cost (is it a non-random page access) as well...)
On Thu, 20 Jul 2006, Josh Berkus wrote: > Peter, > > > I don't understand how that is related. Or what a conversion utility > > would be for that matter. > > Well, the main issue with changing the units of the PostgreSQL.conf file > from a user perspective is that the numbers from you 8.0/8.1 conf file > would no longer work. A little conversion utilitily to turn your 8.0 > file into an 8.2 file would help solve that. Josh, I would imagine that Peter intends to handle backward compatibility by processing values without explicit units in the units assumed pre <8.2. Thanks, Gavin
<br /><blockquote cite="mid200607201851.19521.peter_e@gmx.net" type="cite"><blockquote type="cite"><pre wrap="">Time unitsis easy: 1h = 60min = 3600s = 3600000ms </pre></blockquote><pre wrap=""> We don't need anything larger than seconds at the moment. </pre></blockquote> Except for log_rotation_age perhaps?<br /><br/> -- Korry<br /><br />
Gavin, Peter, > I would imagine that Peter intends to handle backward compatibility by > processing values without explicit units in the units assumed pre <8.2. Aha, I misunderstood. -- Josh Berkus PostgreSQL @ Sun San Francisco
On Thursday 20 July 2006 18:16, Ron Mayer wrote:
> Peter Eisentraut wrote:
> > I think it would be useful to allow units to be added to these settings,
> > for example...
> > shared_buffers = 512MB
> > which is a bit cumbersome to calculate right now (you'd need = 65536).
> >
> > I haven't thought yet how to parse or implement this, but would people
> > find this useful?
>
> Would this extend to things like "random_page_cost" and similar?
>
> If the random_page_cost were specifiable in seconds or ms it might be
> easier to someday write a program to measure such values on particular
> hardware platforms. (though I guess for that to work, the config file
> would also need to add the reference cost (is it a non-random page access)
> as well...)
>
I'd think no, since random page cost doesn't actually map to any real world
value. Unless of course we wanted to add MV for "magic value", but then
people would want to use that for everything ;-D
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL