Thread: Reading recommendations
Hi I am looking for some references to literature. While we have used PostgreSQL in the past for a fair number of smaller projects, we are now starting to use it on a larger scale and hence need to get into issues of performance optimisation and tuning. While I am OK with using the EXPLAIN features, I am getting quite insecure when facing things like the options in the postgresql.conf file. For example reading the man page on fsync option, it tells me to "read the detailed documentation before using this!" I then read the Admin guide where I get told that the benefits of this feature are issue of debate, leaving me with little help as to how to make up my mind on this issue. So I turn to this mailing list, but starting reading in the archive realise that compared to the knowledge standard here, I am as wise as a baby. I have read most of Bruce Momjian's book on PostgreSQL (Can I update my 2001 edition somehow? ;-) I have Sams' PostgreSQL Developer's Handbook (which is OK too), but offers little to nothing on operational issues. I have read most of the admin (and user) guide, but it does not help me really understand the issues: > CPU_INDEX_TUPLE_COST (floating point) Sets the query optimizer’s > estimate of the cost of processing each index tuple during an index > scan. This is measured as a fraction of the cost of a sequential page > fetch. No idea what this means! (And should I feel bad for it?) I am an application programmer with a little UNIX know-how. What books or sources are out there that I can buy/download and that I should read to get to grips with the more advanced issues of running PostgreSQL? More on what we do (for those interested): We use PostgreSQL mainly with its PostGIS extension as the database backend for Zope-based applications. Adding PostGIS features is what has started to cause noticeable increase in the server load. We're using the GIS enabled system on this platform: PostgreSQL 7.3.4 PostGIS 0.8 Zope 2.7.5 Python 2.3.5 (Database-based functions are written in PL/PGSQL, not python!!) on a 2-CPU (450MHz Intel P3) Compaq box (some Proliant flavour) With a SCSI 4-disk RAID system (mirrored and striped) SunOS 5.8 (Which I think is Solaris 8) The server is administrated by my host (co-located). We cannot easily upgrade to a newer version of Solaris, because we could not find a driver for the disk controller used in this server. (And our host did not manage to write/patch one up.) As a business, we are creating and operating on-line communities, (for an example go to http://www.theguidlife.net) not only from a technical point of view, but also supporting the communities in producing content. BTW. If you are a SQL/python programmer in (or near) Lanarkshire, Scotland, we have a vacancy. ;-) Cheers Marc -- Marc Burgauer Sharedbase Ltd http://www.sharedbase.com Creating and supporting on-line communities
On Wed, Mar 30, 2005 at 12:07:29PM +0100, Marc Burgauer wrote: > > What books or sources are out there that I can buy/download and that I > should read to get to grips with the more advanced issues of running > PostgreSQL? See the Power PostgreSQL Performance & Configuration documents: http://www.powerpostgresql.com/Docs/ > BTW. If you are a SQL/python programmer in (or near) Lanarkshire, > Scotland, we have a vacancy. ;-) Allow telecommute from across the pond and I might be interested :-) -- Michael Fuhr http://www.fuhr.org/~mfuhr/
pgsql-performance-owner@postgresql.org wrote on 03/30/2005 10:58:21 AM: > > Allow telecommute from across the pond and I might be interested :-) Please post phone bills to this list. > > -- > Michael Fuhr > http://www.fuhr.org/~mfuhr/ > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly
VOIP over BitTorrent? ;-) -----Original Message----- From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Richard_D_Levine@Raytheon.com Sent: Wednesday, March 30, 2005 11:27 AM To: Michael Fuhr Cc: Marc Burgauer; pgsql-performance@postgresql.org; pgsql-performance-owner@postgresql.org Subject: Re: [PERFORM] Reading recommendations pgsql-performance-owner@postgresql.org wrote on 03/30/2005 10:58:21 AM: > > Allow telecommute from across the pond and I might be interested :-) Please post phone bills to this list. > > -- > Michael Fuhr
Mohan, Ross wrote: > VOIP over BitTorrent? Now *that* I want to see. Aught to be at least as interesting as the "TCP/IP over carrier pigeon" experiment - and more challenging to boot! -- Steve Wampler -- swampler@noao.edu The gods that smiled on your birth are now laughing out loud.
Yea, the upside is that you get better than the 1 byte/hour rate for pigeon-net. Downside is that simply because you use BiTorrent, the RIAA accuses you of everything from CD piracy to shipping pr*n to cyberterrorism, and you spend the next four years in Gitmo, comparing notes with your cellmates in Camp X-Ray, and watching pigeons fly overhead. -----Original Message----- From: Steve Wampler [mailto:swampler@noao.edu] Sent: Wednesday, March 30, 2005 11:52 AM To: Mohan, Ross Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Reading recommendations Mohan, Ross wrote: > VOIP over BitTorrent? Now *that* I want to see. Aught to be at least as interesting as the "TCP/IP over carrier pigeon" experiment - and morechallenging to boot! -- Steve Wampler -- swampler@noao.edu The gods that smiled on your birth are now laughing out loud.
On Wed, Mar 30, 2005 at 16:39:47 -0000, "Mohan, Ross" <RMohan@arbinet.com> wrote: > VOIP over BitTorrent? Plain VOIP shouldn't be a problem. And if you want to do tricky things you can use Asterisk on both ends. Asterisk is open source (GPL, duel licensed from Digium) and runs on low powered linux boxes. A card that talks to your existing analog phones and your existing phone line costs $200. You don't need special cards if you have IP phones or a headset connected to your computer and don't use your local phone company for the calls.
It was very challenging. I worked on the credit window sizing and retransmission timer estimation algorithms. We took into account weather patterns, size and age of the bird, feeding times, and the average number of times a bird circles before determining magnetic north. Interestingly, packet size had little effect in the final algorithms. pgsql-performance-owner@postgresql.org wrote on 03/30/2005 11:52:13 AM: > Mohan, Ross wrote: > > VOIP over BitTorrent? > > Now *that* I want to see. Aught to be at least as interesting > as the "TCP/IP over carrier pigeon" experiment - and more > challenging to boot! > It was very challenging. I worked on the credit window sizing and retransmission timer estimation algorithms. We took into account weather patterns, size and age of the bird, feeding times, and the average number of times a bird circles before determining magnetic north. Interestingly, packet size had little effect in the final algorithms. I would love to share them with all of you, but they're classified. > > -- > Steve Wampler -- swampler@noao.edu > The gods that smiled on your birth are now laughing out loud. > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster
Richard_D_Levine@raytheon.com wrote: >>Mohan, Ross wrote: >> >>>VOIP over BitTorrent? >> >>Now *that* I want to see. Aught to be at least as interesting >>as the "TCP/IP over carrier pigeon" experiment - and more >>challenging to boot! >> > > > It was very challenging. I worked on the credit window sizing and > retransmission timer estimation algorithms. We took into account weather > patterns, size and age of the bird, feeding times, and the average number > of times a bird circles before determining magnetic north. Interestingly, > packet size had little effect in the final algorithms. > > I would love to share them with all of you, but they're classified. Ah, but VOIPOBT requires many people all saying the same thing at the same time. The synchronization alone (since you need to distribute these people adequately to avoid overloading a trunk line...) is probably sufficiently hard to make it interesting. Then there are the problems of different accents, dilects, and languages ;) -- Steve Wampler -- swampler@noao.edu The gods that smiled on your birth are now laughing out loud.
I can see that PG'ers have a wicked sense of humor. -----Original Message----- From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Steve Wampler Sent: Wednesday, March 30, 2005 3:58 PM To: Richard_D_Levine@raytheon.com Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Reading recommendations Richard_D_Levine@raytheon.com wrote: >>Mohan, Ross wrote: >> >>>VOIP over BitTorrent? >> >>Now *that* I want to see. Aught to be at least as interesting as the >>"TCP/IP over carrier pigeon" experiment - and more challenging to >>boot! >> > > > It was very challenging. I worked on the credit window sizing and > retransmission timer estimation algorithms. We took into account > weather patterns, size and age of the bird, feeding times, and the > average number of times a bird circles before determining magnetic > north. Interestingly, packet size had little effect in the final > algorithms. > > I would love to share them with all of you, but they're classified. Ah, but VOIPOBT requires many people all saying the same thing at the same time. The synchronization alone (since you needto distribute these people adequately to avoid overloading a trunk line...) is probably sufficiently hard to make itinteresting. Then there are the problems of different accents, dilects, and languages ;) -- Steve Wampler -- swampler@noao.edu The gods that smiled on your birth are now laughing out loud. ---------------------------(end of broadcast)--------------------------- TIP 7: don't forget to increase your free space map settings
Steve Wampler <swampler@noao.edu> wrote on 03/30/2005 03:58:12 PM: > Richard_D_Levine@raytheon.com wrote: > > >>Mohan, Ross wrote: > >> > >>>VOIP over BitTorrent? > >> > >>Now *that* I want to see. Aught to be at least as interesting > >>as the "TCP/IP over carrier pigeon" experiment - and more > >>challenging to boot! > >> > > > > > > It was very challenging. I worked on the credit window sizing and > > retransmission timer estimation algorithms. We took into account weather > > patterns, size and age of the bird, feeding times, and the average number > > of times a bird circles before determining magnetic north. Interestingly, > > packet size had little effect in the final algorithms. > > > > I would love to share them with all of you, but they're classified. > > Ah, but VOIPOBT requires many people all saying the same thing at the > same time. The synchronization alone (since you need to distribute > these people adequately to avoid overloading a trunk line...) is probably > sufficiently hard to make it interesting. Then there are the problems of > different accents, dilects, and languages ;) Interestingly, we had a follow on contract to investigate routing optimization using flooding techniques. Oddly, it was commissioned by a consortium of local car washes. Work stopped when the park service sued us for the cost of cleaning all the statuary, and the company went out of business. We were serving "cornish game hens" at our frequent dinner parties for months. > > -- > Steve Wampler -- swampler@noao.edu > The gods that smiled on your birth are now laughing out loud.
On 2005-03-31 15:19, Richard_D_Levine@Raytheon.com wrote: >> >>Now *that* I want to see. Aught to be at least as interesting >> >>as the "TCP/IP over carrier pigeon" experiment - and more >> >>challenging to boot! .. > Interestingly, we had a follow on contract to investigate routing > optimization using flooding techniques. Oddly, it was commissioned by a > consortium of local car washes. Work stopped when the park service sued us > for the cost of cleaning all the statuary, and the company went out of > business. We were serving "cornish game hens" at our frequent dinner > parties for months. This method might have been safer (and it works great with Apaches): http://eagle.auc.ca/~dreid/ cheers stefan
Stefan Weiss wrote: > On 2005-03-31 15:19, Richard_D_Levine@Raytheon.com wrote: > >>>>>Now *that* I want to see. Aught to be at least as interesting >>>>>as the "TCP/IP over carrier pigeon" experiment - and more >>>>>challenging to boot! > > .. > >>Interestingly, we had a follow on contract to investigate routing >>optimization using flooding techniques. Oddly, it was commissioned by a >>consortium of local car washes. Work stopped when the park service sued us >>for the cost of cleaning all the statuary, and the company went out of >>business. We were serving "cornish game hens" at our frequent dinner >>parties for months. > > > This method might have been safer (and it works great with Apaches): > http://eagle.auc.ca/~dreid/ Aha - VOIPOBD as well as VOIPOBT! What more can one want? VOIPOCP, I suppose... -- Steve Wampler -- swampler@noao.edu The gods that smiled on your birth are now laughing out loud.
pgsql-performance-owner@postgresql.org wrote on 03/31/2005 10:48:09 AM: > Stefan Weiss wrote: > > On 2005-03-31 15:19, Richard_D_Levine@Raytheon.com wrote: > > > >>>>>Now *that* I want to see. Aught to be at least as interesting > >>>>>as the "TCP/IP over carrier pigeon" experiment - and more > >>>>>challenging to boot! > > > > .. > > > >>Interestingly, we had a follow on contract to investigate routing > >>optimization using flooding techniques. Oddly, it was commissioned by a > >>consortium of local car washes. Work stopped when the park service sued us > >>for the cost of cleaning all the statuary, and the company went out of > >>business. We were serving "cornish game hens" at our frequent dinner > >>parties for months. > > > > > > This method might have been safer (and it works great with Apaches): > > http://eagle.auc.ca/~dreid/ > > Aha - VOIPOBD as well as VOIPOBT! What more can one want? > > VOIPOCP, I suppose... Start collecting recipes for small game birds now. We ran out pretty quickly. Finally came up with "Pigeon Helper" and sold it to homeless shelters in New York. Sales were slow until we added a wine sauce. > > > -- > Steve Wampler -- swampler@noao.edu > The gods that smiled on your birth are now laughing out loud. > > ---------------------------(end of broadcast)--------------------------- > TIP 7: don't forget to increase your free space map settings