Re: System load consideration before spawning parallel workers - Mailing list pgsql-hackers

From Gavin Flower
Subject Re: System load consideration before spawning parallel workers
Date
Msg-id d02c9b7d-bedd-3a11-9aa3-cfbd45902743@archidevsys.co.nz
Whole thread Raw
In response to Re: System load consideration before spawning parallel workers  (Peter Eisentraut <peter.eisentraut@2ndquadrant.com>)
Responses Re: System load consideration before spawning parallel workers  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On 02/09/16 04:44, Peter Eisentraut wrote:
> On 8/1/16 2:17 AM, Gavin Flower wrote:
>> Possibly look how make does it with the '-l' flag?
>>
>> '-l 8' don't start more process when load is 8 or greater, works on
>> Linux at least...
> The problem with that approach is that it takes about a minute for the
> load averages figures to be updated, by which time you have already
> thrashed your system.
>
> You can try this out by building PostgreSQL this way.  Please save your
> work first, because you might have to hard-reboot your system.
>
Hmm...  I've built several versions of pg this way, without any obvious 
problems!

Looking at top, suggests that the load averages never go much above 8, 
and are usually less.

This is the bash script I use:


#!/bin/bash
# postgresql-build.sh


VERSION='9.5.0'

TAR_FILE="postgresql-$VERSION.tar.bz2"
echo 'TAR_FILE['$TAR_FILE']'
tar xvf $TAR_FILE

PORT='--with-pgport=5433'  ############################ std is 5432

BASE_DIR="postgresql-$VERSION"
echo 'BASE_DIR['$BASE_DIR']'
cd $BASE_DIR

PREFIX="--prefix=/usr/local/lib/postgres-$VERSION"
echo 'PREFIX['$PREFIX']'

LANGUAGES='--with-python'
echo 'LANGUAGES['$LANGUAGES']'

SECURITY='--with-openssl --with-pam --with-ldap'
echo 'PREFIX['$PREFIX']'

XML='--with-libxml --with-libxslt'
echo 'SECURITY['$SECURITY']'

TZDATA='--with-system-tzdata=/usr/share/zoneinfo'
echo 'TZDATA['$TZDATA']'

##DEBUG='--enable-debug'
##echo 'DEBUG['$DEBUG']'


./configure $PREFIX $LANGUAGES $SECURITY $XML $TZDATA $DEBUG

time make -j7 -l8 && time make -j7 -l8 check


Cheers,
Gavin




pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: incomplete removal of not referenced CTEs
Next
From: Andres Freund
Date:
Subject: Re: incomplete removal of not referenced CTEs