Thread: Linux vs FreeBSD

Hi all!

Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 

Our workload is lots of data import, followed by many queries to summarize (daily and weekly reports). Our main table
isa wide table that represents Twitter and Facebook interactions. Most of our reports work on a week's worth of data
(tableis partitioned by week), and the tables are approximately 25 GB plus 5 GB of indices, per week. Of course, while
reportsare ongoing, we're also importing next week's data. 

The host is a dedicated hardware machine at online.fr: 128 GB RAM, 2 x 3TB disk in RAID 1 configuration.

I started thinking of this after reading "PostgreSQL pain points" at https://lwn.net/Articles/591723/. In the comments,
bronsonsays FreeBSD does not exhibit the same problems (slow fsync, double buffering). On the list here, I've read
aboutproblems with certain kernel versions on Ubuntu. 

I'm not expecting anything magical, just some general guidelines and hints. Did anybody do the migration and was
happierafter? 

Thanks for any hints!
François Beausoleil

$ uname -a
Linux munn.ca.seevibes.com 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

$ psql -U postgres -c "select version()"
                                                   version
-------------------------------------------------------------------------------------------------------------
 PostgreSQL 9.1.11 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3, 64-bit

/proc/cpuinfo says: 8 CPUs, identified as "Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz"

On 4/3/2014 9:03 PM, François Beausoleil wrote:
> The host is a dedicated hardware machine at online.fr: 128 GB RAM, 2 x 3TB disk in RAID 1 configuration.

just a passing comment...

3TB disks are 7200rpm and suitable for nearline bulk storage (or desktop
use), not high performance database random access. you'd get way more IO
throughput with a raid10 array of 15k rpm SAS disks, maybe 10 x 600GB or
whatever, or with a suitable SSD configuration, although 3TB of SSD
might get expensive.



--
john r pierce                                      37N 122W
somewhere on the middle of the left coast

FreeBSD is OK if you are experienced. As a system it requires much more maturity by the admin
than lets say Ubuntu which is targeted at a larger user base.
I'd say, explore your other Linux options first, since you already have experience with Linux.
FreeBSD requires a much bigger learning curve.

On 04/04/2014 07:03, François Beausoleil wrote:
> Hi all!
>
> Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 
>
> Our workload is lots of data import, followed by many queries to summarize (daily and weekly reports). Our main table
isa wide table that represents Twitter and Facebook interactions. Most of our reports work on a week's worth of data
(tableis partitioned by week), and the tables are approximately 25 GB plus 5 GB of indices, per week. Of course, while
reportsare ongoing, we're also importing next week's data. 
>
> The host is a dedicated hardware machine at online.fr: 128 GB RAM, 2 x 3TB disk in RAID 1 configuration.
>
> I started thinking of this after reading "PostgreSQL pain points" at https://lwn.net/Articles/591723/. In the
comments,bronson says FreeBSD does not exhibit the same problems (slow fsync, double buffering). On the list here, I've
readabout problems with certain kernel versions on Ubuntu. 
>
> I'm not expecting anything magical, just some general guidelines and hints. Did anybody do the migration and was
happierafter? 
>
> Thanks for any hints!
> François Beausoleil
>
> $ uname -a
> Linux munn.ca.seevibes.com 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>
> $ psql -U postgres -c "select version()"
>                                                     version
> -------------------------------------------------------------------------------------------------------------
>   PostgreSQL 9.1.11 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3, 64-bit
>
> /proc/cpuinfo says: 8 CPUs, identified as "Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz"
>


--
Achilleas Mantzios
Head of IT DEV
IT DEPT
Dynacom Tankers Mgmt

On Apr 4, 2014, at 12:03 AM, François Beausoleil <francois@teksol.info> wrote:

> I have some performance issues on a Ubuntu 12.04 which I'd like to resolve. iowait varies a lot, between 5 and 50%.

Is the SAN dedicated to this app?  I wonder if the i/o, if not related to your app, is being pressed by some other
system/sthat are also on the SAN?  You'll need to relate the wait events to your activities. I've been squeezed like
thatin the past.  

Le 2014-04-04 à 08:11, Ray Stell a écrit :

>
> On Apr 4, 2014, at 12:03 AM, François Beausoleil <francois@teksol.info> wrote:
>
>> I have some performance issues on a Ubuntu 12.04 which I'd like to resolve. iowait varies a lot, between 5 and 50%.
>
> Is the SAN dedicated to this app?  I wonder if the i/o, if not related to your app, is being pressed by some other
system/sthat are also on the SAN?  You'll need to relate the wait events to your activities. I've been squeezed like
thatin the past.  

There are no SAN: it's a dedicated machine, dedicated hardware. As John Pierce guessed, having more disks would help. I
shouldprobably look more towards that than switching OS. 

Thanks!
François

As a side note, when we migrated the exact same pgsql 8.3 system from linux kernel 2.6 to 3.6,
we experienced an almost dramatic slowdown by 6 times.
Linux Kernel's were known to have issues around those dates, i recall.
We had to set synchronous_commit to off, this gave a huge boost ,
but this was no longer "apples" vs "apples".

On 04/04/2014 07:03, François Beausoleil wrote:
> Hi all!
>
> Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 
>
> Our workload is lots of data import, followed by many queries to summarize (daily and weekly reports). Our main table
isa wide table that represents Twitter and Facebook interactions. Most of our reports work on a week's worth of data
(tableis partitioned by week), and the tables are approximately 25 GB plus 5 GB of indices, per week. Of course, while
reportsare ongoing, we're also importing next week's data. 
>
> The host is a dedicated hardware machine at online.fr: 128 GB RAM, 2 x 3TB disk in RAID 1 configuration.
>
> I started thinking of this after reading "PostgreSQL pain points" at https://lwn.net/Articles/591723/. In the
comments,bronson says FreeBSD does not exhibit the same problems (slow fsync, double buffering). On the list here, I've
readabout problems with certain kernel versions on Ubuntu. 
>
> I'm not expecting anything magical, just some general guidelines and hints. Did anybody do the migration and was
happierafter? 
>
> Thanks for any hints!
> François Beausoleil
>
> $ uname -a
> Linux munn.ca.seevibes.com 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>
> $ psql -U postgres -c "select version()"
>                                                     version
> -------------------------------------------------------------------------------------------------------------
>   PostgreSQL 9.1.11 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3, 64-bit
>
> /proc/cpuinfo says: 8 CPUs, identified as "Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz"
>


--
Achilleas Mantzios
Head of IT DEV
IT DEPT
Dynacom Tankers Mgmt

On Fri, Apr 4, 2014 at 12:03 AM, François Beausoleil
<francois@teksol.info> wrote:
> Our workload is lots of data import, followed by many queries to summarize (daily and weekly reports). Our main table
isa wide table that represents Twitter and Facebook interactions. Most of our reports work on a week's worth of data
(tableis partitioned by week), and the tables are approximately 25 GB plus 5 GB of indices, per week. Of course, while
reportsare ongoing, we're also importing next week's data. 
>
> The host is a dedicated hardware machine at online.fr
>
> : 128 GB RAM, 2 x 3TB disk in RAID 1 configuration.

I use FreeBSD pretty much exclusively. I would recommend using it to
anyone, but as others have said, there is a learning curve. If you are
comfortable on the command line, the curve is not that great. From a
postgres perspective, there is not that much difference once you dig
thru google to find a good set of OS settings (I'm happy to share
mine.)

That all said, I think you should do two things: First, improve your
disk system. Is your RAID soft or hardware? What file system do you
run on it? You want the file system to be the fastest you can have
that has the features you need. Pg is pretty good about crash
recovery, so having a log-based file system is not absolutely
necessary to save you there, but it depends on how much downtime you
can take for fsck to run. Since you're on spinning platters, you want
to peel off your pg_xlog directory to another set of drives in mirror
configuration. I personally like SSDs for this.  You probably also do
not need the lvm layer here either. You imply that you are analyzing
one week while loading the next week. Perhaps set up a second disk
mirror (instead of RAID10 all of the disks) and use a separate
postgres table space to load the data on the "other" mirror from the
one you are currently analyzing, like odd/even week numbers.

The second item is that you said you are using "wide tables". You will
be amazed at how much better you could do by properly normalizing your
data as you import it, rather than what seems like just storing log
events and picking out what you need for your summaries. Postgres is
exceptionally good at optimizing queries with joins and summaries. I
would put forth some effort simplifying how you store the data to
match more closely with the answers you want from it.

On Thu, Apr 3, 2014 at 11:03 PM, François Beausoleil
<francois@teksol.info> wrote:
> Hi all!
>
> Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 
>
> $ uname -a
> Linux munn.ca.seevibes.com 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

From the research I've done online, this is likely your issue.  Kernel
3.2.0 has some issues that directly and severely impact I/O wait times
for PostgreSQL.  The suggested fixes (that seem to have worked for
most people reporting in) are to revert the OS to Ubuntu Server 10.04
or to install one of the HWE (HardWare Enablement) kernels into the
12.04 system (this would be one of the kernels from a later release of
Ubuntu provided in the 12.04 repositories).

--
Chris

"If you wish to make an apple pie from scratch, you must first invent
the Universe." -- Carl Sagan

On Wed, Apr  9, 2014 at 10:02:07AM -0500, Christofer C. Bell wrote:
> On Thu, Apr 3, 2014 at 11:03 PM, François Beausoleil
> <francois@teksol.info> wrote:
> > Hi all!
> >
> > Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 
> >
> > $ uname -a
> > Linux munn.ca.seevibes.com 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64
GNU/Linux
>
> From the research I've done online, this is likely your issue.  Kernel
> 3.2.0 has some issues that directly and severely impact I/O wait times
> for PostgreSQL.  The suggested fixes (that seem to have worked for
> most people reporting in) are to revert the OS to Ubuntu Server 10.04
> or to install one of the HWE (HardWare Enablement) kernels into the
> 12.04 system (this would be one of the kernels from a later release of
> Ubuntu provided in the 12.04 repositories).

This highlights a more fundamental problem of the difference between a
workstation-based on OS like Ubuntu and a server-based one like Debian
or FreeBSD.  I know Ubuntu has a "server" version, but fundamentally
Ubuntu's selection of kernels and feature churn make it less than ideal
for server deployments.

I am sure someone can post that they use Ubuntu just fine for server
deployments, but I continue to feel that Ubuntu is chosen by
administrators because it an OS they are familiar with on workstations,
rather than it being the best choice for servers.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

On Wed, Apr 9, 2014 at 9:02 AM, Christofer C. Bell
<christofer.c.bell@gmail.com> wrote:
> On Thu, Apr 3, 2014 at 11:03 PM, François Beausoleil
> <francois@teksol.info> wrote:
>> Hi all!
>>
>> Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 
>>
>> $ uname -a
>> Linux munn.ca.seevibes.com 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64
GNU/Linux
>
> From the research I've done online, this is likely your issue.  Kernel
> 3.2.0 has some issues that directly and severely impact I/O wait times
> for PostgreSQL.  The suggested fixes (that seem to have worked for
> most people reporting in) are to revert the OS to Ubuntu Server 10.04
> or to install one of the HWE (HardWare Enablement) kernels into the
> 12.04 system (this would be one of the kernels from a later release of
> Ubuntu provided in the 12.04 repositories).

12.04 supports 3.8.0 directly. There's a site on putting 3.10.17 or so
on it as well I found by googling for it. You need 3.10+ if you wanna
play with bcache which is how I found it. But you don't need to jump
through any hoops to get 3.8.0 on 12.04.4 LTS

--
To understand recursion, one must first understand recursion.

On Apr 9, 2014, at 1:33 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:

> On Wed, Apr 9, 2014 at 9:02 AM, Christofer C. Bell
> <christofer.c.bell@gmail.com> wrote:
>> On Thu, Apr 3, 2014 at 11:03 PM, François Beausoleil
>> <francois@teksol.info> wrote:
>>> Hi all!
>>>
>>> Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 
>>>
>>> $ uname -a
>>> Linux munn.ca.seevibes.com 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64
GNU/Linux
>>
>> From the research I've done online, this is likely your issue.  Kernel
>> 3.2.0 has some issues that directly and severely impact I/O wait times
>> for PostgreSQL.  The suggested fixes (that seem to have worked for
>> most people reporting in) are to revert the OS to Ubuntu Server 10.04
>> or to install one of the HWE (HardWare Enablement) kernels into the
>> 12.04 system (this would be one of the kernels from a later release of
>> Ubuntu provided in the 12.04 repositories).
>
> 12.04 supports 3.8.0 directly. There's a site on putting 3.10.17 or so
> on it as well I found by googling for it. You need 3.10+ if you wanna
> play with bcache which is how I found it. But you don't need to jump
> through any hoops to get 3.8.0 on 12.04.4 LTS

Or wait <checks watch> 8 days for14.04 LTS with kernel 3.14.

Cheers,
  Steve

On Wed, Apr 9, 2014 at 2:56 PM, Steve Atkins <steve@blighty.com> wrote:
>
> On Apr 9, 2014, at 1:33 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:
>
>> On Wed, Apr 9, 2014 at 9:02 AM, Christofer C. Bell
>> <christofer.c.bell@gmail.com> wrote:
>>> On Thu, Apr 3, 2014 at 11:03 PM, François Beausoleil
>>> <francois@teksol.info> wrote:
>>>> Hi all!
>>>>
>>>> Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 
>>>>
>>>> $ uname -a
>>>> Linux munn.ca.seevibes.com 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64
GNU/Linux
>>>
>>> From the research I've done online, this is likely your issue.  Kernel
>>> 3.2.0 has some issues that directly and severely impact I/O wait times
>>> for PostgreSQL.  The suggested fixes (that seem to have worked for
>>> most people reporting in) are to revert the OS to Ubuntu Server 10.04
>>> or to install one of the HWE (HardWare Enablement) kernels into the
>>> 12.04 system (this would be one of the kernels from a later release of
>>> Ubuntu provided in the 12.04 repositories).
>>
>> 12.04 supports 3.8.0 directly. There's a site on putting 3.10.17 or so
>> on it as well I found by googling for it. You need 3.10+ if you wanna
>> play with bcache which is how I found it. But you don't need to jump
>> through any hoops to get 3.8.0 on 12.04.4 LTS
>
> Or wait <checks watch> 8 days for14.04 LTS with kernel 3.14.

I'm not deploying any new distro version that soon. :) I know folks
just putting 12.04 into prod to replace etch and lenny. :)
--
To understand recursion, one must first understand recursion.

On 04/09/14 14:46, Scott Marlowe wrote:
>
> I'm not deploying any new distro version that soon. :) I know folks
> just putting 12.04 into prod to replace etch and lenny. :)

You can easily get the 3.11.0 kernel on 12.04.4 LTS by installing
the linux-generic-lts-saucy package.  IIRC, the fix for the iowait
issue was committed into 3.9.x but the first Ubuntu kernel that
appeared with the fix was the one for 'saucy'.

On Wednesday, April 09, 2014 09:02:02 PM Brent Wood wrote:
> Given the likely respective numbers of each OS actually out there, I'd
> suggests BSD is very over-represented in the high uptime list which is
> suggestive.

Suggestive of ... sysadmins who don't do kernel updates?

On 4 April 2014 11:03, François Beausoleil <francois@teksol.info> wrote:
> Hi all!
>
> Does PG perform that much better on FreeBSD? I have some performance issues on a Ubuntu 12.04 which I'd like to
resolve.iowait varies a lot, between 5 and 50%. Does FreeBSD better schedule I/O, which could alleviate some of the
issues,or not at all? I have no experience administering FreeBSD, but I'm willing to learn if I'll get some performance
enhancementsout of the switch. 

Ubuntu 14.04 LTS is being released in a few days and you might have
more success with its newer kernel. And try different schedulers if
you haven't already - IIRC switching to deadline resolved one of our
load problems.

--
Stuart Bishop <stuart@stuartbishop.net>
http://www.stuartbishop.net/

On Wed, Apr 9, 2014 at 3:36 PM, François Beausoleil
<francois@teksol.info> wrote:
>
> Le 2014-04-09 à 16:20, Bruce Momjian a écrit :
>
> On Wed, Apr  9, 2014 at 10:02:07AM -0500, Christofer C. Bell wrote:
>
> This highlights a more fundamental problem of the difference between a
> workstation-based on OS like Ubuntu and a server-based one like Debian
> or FreeBSD.  I know Ubuntu has a "server" version, but fundamentally
> Ubuntu's selection of kernels and feature churn make it less than ideal
> for server deployments.
>
> I am sure someone can post that they use Ubuntu just fine for server
> deployments, but I continue to feel that Ubuntu is chosen by
> administrators because it an OS they are familiar with on workstations,
> rather than it being the best choice for servers.
>
>
> I'm not a full-time sysadmin. I chose Ubuntu because I have familiarity with
> it, and because installing Puppet on it installed the certificates and
> everything I needed to get going. I tried Debian, but I had to fight and
> find the correct procedures to install the Puppet certificates and all.
> Ubuntu saved me some time back then.
>
> Cheers!
> François

I'm not wanting to get after anyone here, but I want it on the record
that I am not the source of the above quote discouraging the use of
Ubuntu in a server role.  That would be Bruce Momjian. While Bruce is
entitled to his opinion, it's not one I agree with and I don't want a
Google search years from now to tie my name to that viewpoint.

Thanks!

--
Chris

"If you wish to make an apple pie from scratch, you must first invent
the Universe." -- Carl Sagan

On 04/10/14 17:25, Christofer C. Bell wrote:
> I'm not wanting to get after anyone here, but I want it on the record
> that I am not the source of the above quote discouraging the use of
> Ubuntu in a server role.  That would be Bruce Momjian. While Bruce is
> entitled to his opinion, it's not one I agree with and I don't want a
> Google search years from now to tie my name to that viewpoint.

Who (in their right mind) would ever think of anything but BSD in a
server role?

<shaking head>


Jan

--
Jan Wieck
Senior Software Engineer
http://slony.info

Basically it goes beyond what ppl would describe as OS holly wars.
If one chooses to go by FreeBSD, then he better be prepared to handle the burden, both the part that is
imposed by the OS administration itself, as well as the part that is a side effect of the different base system.

Example of admin part :
Generally, compiling postgresql from source gives more freedom than be stuck on the OS's ports or PKGng
system. (the later being a very handy and welcome addition to FreeBSD).
Now what if e.g. the user wants pgsql software X (e.g. pgadmin3, p5-Pg, etc...) only to find out that most of those
ports need postgresql client as a dependency. He/she must be prepared to work his way through :
- manual installations (gmake config && gmake && gmake install)
- /usr/ports
- PKG binary installations
in decreasing order of freedom but increasing order of easiness, and in many cases work through a combination
of the above.

Example of base system part :
Recently I had to install pl-java on my FreeBSD workstation. There was a problem with libtrh, postgresql should be
recompiled
with explicitly setting : -lpthread in /usr/local/src/postgresql-9.3.4/src/backend/Makefile, without this the backend
wouldsimply hang 
at the very first invocation of a java function. This came after detailed following or email exchange of various
hackersgroups 
in both pgsql and FreeBSD lists, to describe the issue as accurately as possible, to help debug as most as possible, to
talk
to the right people, to give them incentive to answer back, etc.

I don't mean to scare the OP, but FreeBSD is not for everyone.

On 11/04/2014 00:50, Jan Wieck wrote:
> On 04/10/14 17:25, Christofer C. Bell wrote:
>> I'm not wanting to get after anyone here, but I want it on the record
>> that I am not the source of the above quote discouraging the use of
>> Ubuntu in a server role.  That would be Bruce Momjian. While Bruce is
>> entitled to his opinion, it's not one I agree with and I don't want a
>> Google search years from now to tie my name to that viewpoint.
>
> Who (in their right mind) would ever think of anything but BSD in a server role?
>
> <shaking head>
>
>
> Jan
>


--
Achilleas Mantzios
Head of IT DEV
IT DEPT
Dynacom Tankers Mgmt

On 11 Apr 2014, at 8:04, Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:

> Basically it goes beyond what ppl would describe as OS holly wars.
> If one chooses to go by FreeBSD, then he better be prepared to handle the burden, both the part that is
> imposed by the OS administration itself, as well as the part that is a side effect of the different base system.
>
> Example of admin part :
> Generally, compiling postgresql from source gives more freedom than be stuck on the OS's ports or PKGng
> system. (the later being a very handy and welcome addition to FreeBSD).
> Now what if e.g. the user wants pgsql software X (e.g. pgadmin3, p5-Pg, etc...) only to find out that most of those
> ports need postgresql client as a dependency. He/she must be prepared to work his way through :
> - manual installations (gmake config && gmake && gmake install)
> - /usr/ports
> - PKG binary installations
> in decreasing order of freedom but increasing order of easiness, and in many cases work through a combination
> of the above.

That argument holds for any package system on any OS I know of. Once you start custom compiling things outside the
controlof the package management system, you’re on your own. 

Custom compiling may give more freedom, but it’s hardly ever necessary on FreeBSD. For example, the only ports that I
everhad to custom compile were ports for software I was developing, which of course no package management system can
keeptrack of. 

In general, the various options the port Makefile provides for customisation are quite sufficient. It’s a plus to the
portssystem that you get any options at all. 

> Example of base system part :
> Recently I had to install pl-java on my FreeBSD workstation. There was a problem with libtrh, postgresql should be
recompiled
> with explicitly setting : -lpthread in /usr/local/src/postgresql-9.3.4/src/backend/Makefile, without this the backend
wouldsimply hang 
> at the very first invocation of a java function. This came after detailed following or email exchange of various
hackersgroups 
> in both pgsql and FreeBSD lists, to describe the issue as accurately as possible, to help debug as most as possible,
totalk 
> to the right people, to give them incentive to answer back, etc.

It seems to me that the reason you were custom compiling Postgres in the first place was a problem with the port. I’m
suretracking down the problem wasn’t easy, but that is not really relevant to the topic. Ports break sometimes (on any
OS)and it would have been sufficient to contact the port maintainer about the issue. 

For a quick (temporary) fix, you could probably have fixed the port by editing the port Makefile. With that, there’s no
reasonanymore to “custom compile” postgres and it leaves the dependency tracking of the port in place. Editing
Makefilesis indeed not for everyone, but at least you _can_ do that on FreeBSD. Not every package management system
willlet you do that. 

And yes, I have edited Makefiles, although the need hasn’t risen recently.

> I don't mean to scare the OP, but FreeBSD is not for everyone.

And that (again) could be said about any OS. Even Windows or OS X.
It depends on what you intend to use it for and what prior experience, preconceptions and expectations you might have.

Oh, and please try not to top-post when replying on this list.

> On 11/04/2014 00:50, Jan Wieck wrote:
>> On 04/10/14 17:25, Christofer C. Bell wrote:
>>> I'm not wanting to get after anyone here, but I want it on the record
>>> that I am not the source of the above quote discouraging the use of
>>> Ubuntu in a server role.  That would be Bruce Momjian. While Bruce is
>>> entitled to his opinion, it's not one I agree with and I don't want a
>>> Google search years from now to tie my name to that viewpoint.
>>
>> Who (in their right mind) would ever think of anything but BSD in a server role?
>>
>> <shaking head>
>>
>>
>> Jan

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.

On 11/04/2014 13:05, Alban Hertroys wrote:
> On 11 Apr 2014, at 8:04, Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:
>
>> Basically it goes beyond what ppl would describe as OS holly wars.
>> If one chooses to go by FreeBSD, then he better be prepared to handle the burden, both the part that is
>> imposed by the OS administration itself, as well as the part that is a side effect of the different base system.
>>
>> Example of admin part :
>> Generally, compiling postgresql from source gives more freedom than be stuck on the OS's ports or PKGng
>> system. (the later being a very handy and welcome addition to FreeBSD).
>> Now what if e.g. the user wants pgsql software X (e.g. pgadmin3, p5-Pg, etc...) only to find out that most of those
>> ports need postgresql client as a dependency. He/she must be prepared to work his way through :
>> - manual installations (gmake config && gmake && gmake install)
>> - /usr/ports
>> - PKG binary installations
>> in decreasing order of freedom but increasing order of easiness, and in many cases work through a combination
>> of the above.
> That argument holds for any package system on any OS I know of. Once you start custom compiling things outside the
controlof the package management system, you’re on your own. 
I am not against FreeBSD in any way, as a matter of fact i am struggling for about 20 years
to keep it alive at least in my working environment, being my primary development workstation.
> Custom compiling may give more freedom, but it’s hardly ever necessary on FreeBSD. For example, the only ports that I
everhad to custom compile were ports for software I was developing, which of course no package management system can
keeptrack of. 
Try to install/setup PgSQL-backed openldap with unixODBC when your KDE has iodbc as a prerequisite.
Or try to install pljava, for which of course no OS port/OS package/PgSQL extension exists, yet.
Good luck with any of those.
> In general, the various options the port Makefile provides for customisation are quite sufficient. It’s a plus to the
portssystem that you get any options at all. 
>
>> Example of base system part :
>> Recently I had to install pl-java on my FreeBSD workstation. There was a problem with libtrh, postgresql should be
recompiled
>> with explicitly setting : -lpthread in /usr/local/src/postgresql-9.3.4/src/backend/Makefile, without this the
backendwould simply hang 
>> at the very first invocation of a java function. This came after detailed following or email exchange of various
hackersgroups 
>> in both pgsql and FreeBSD lists, to describe the issue as accurately as possible, to help debug as most as possible,
totalk 
>> to the right people, to give them incentive to answer back, etc.
> It seems to me that the reason you were custom compiling Postgres in the first place was a problem with the port. I’m
suretracking down the problem wasn’t easy, but that is not really relevant to the topic. Ports break sometimes (on any
OS)and it would have been sufficient to contact the port maintainer about the issue. 
No, i wasn't compiling postgresql from standard distribution because of a problem with the port.
(although the port had the same exact behavior)
I always run postgresql compiled by hand, since I see no reason to sacrifice my peace of mind
for a short-lived joy going with the ports or PKGng system.
As a matter of fact, PostgreSQL is among the few software packages that i would advice strongly
against using ports or pkgs of any kind. Might work in Debian. Would not risk this in FreeBSD.
> For a quick (temporary) fix, you could probably have fixed the port by editing the port Makefile. With that, there’s
noreason anymore to “custom compile” postgres and it leaves the dependency tracking of the port in place. Editing
Makefilesis indeed not for everyone, but at least you _can_ do that on FreeBSD. Not every package management system
willlet you do that. 
Sure, but the way to do this is not by simply editing a Makefile, but with writing an extra patch
inside /usr/ports/databases/postgresql93-server/files/ . Which is more burden than easiness.
> And yes, I have edited Makefiles, although the need hasn’t risen recently.
With plain vanilla ports it is rarely needed.
>> I don't mean to scare the OP, but FreeBSD is not for everyone.
> And that (again) could be said about any OS. Even Windows or OS X.
> It depends on what you intend to use it for and what prior experience, preconceptions and expectations you might
have.
Playing with words aside, going with FreeBSD is not for the average Ubuntu user.
>
> Oh, and please try not to top-post when replying on this list.
I did just for this message, because i did not feel appropriate to quote anything that the previous
poster wrote.
>> On 11/04/2014 00:50, Jan Wieck wrote:
>>> On 04/10/14 17:25, Christofer C. Bell wrote:
>>>> I'm not wanting to get after anyone here, but I want it on the record
>>>> that I am not the source of the above quote discouraging the use of
>>>> Ubuntu in a server role.  That would be Bruce Momjian. While Bruce is
>>>> entitled to his opinion, it's not one I agree with and I don't want a
>>>> Google search years from now to tie my name to that viewpoint.
>>> Who (in their right mind) would ever think of anything but BSD in a server role?
>>>
>>> <shaking head>
>>>
>>>
>>> Jan
> Alban Hertroys
> --
> If you can't see the forest for the trees,
> cut the trees and you'll find there is no forest.
>
>
>


--
Achilleas Mantzios
Head of IT DEV
IT DEPT
Dynacom Tankers Mgmt

On 11 Apr 2014, at 12:39, Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:

I moved this bit of the conversation up as it’s relevant to the OP:

> On 11/04/2014 13:05, Alban Hertroys wrote:
>> On 11 Apr 2014, at 8:04, Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:

>>> I don't mean to scare the OP, but FreeBSD is not for everyone.
>> And that (again) could be said about any OS. Even Windows or OS X.
>> It depends on what you intend to use it for and what prior experience, preconceptions and expectations you might
have.
> Playing with words aside, going with FreeBSD is not for the average Ubuntu user.

(Words are fun!)

That was actually what I was getting at. A user coming from Ubuntu will have some preconceived ideas about how things
work(because that’s how Ubuntu does things) that get in their way when moving to a different OS where they sometimes
workjust slightly differently. 

My advice to the OP:

Install FreeBSD on a system to play around with, get a feel for how it works and whether you like it or not.
See how it performs with Postgres on different file-systems; UFS2 or ZFS - UFS is the faster of the two, but ZFS makes
upfor that big time in maintainability if you provide it with enough memory. 
If you require locale-specific collations (native language specific sort ordering), check that it does what you expect.

If you instead put your database on FreeBSD straight away, you’ve suddenly made yourself responsible for a system that
youdo not know how to maintain or tune properly. 

There are solutions to that of course, such as hiring someone to maintain it for you or to educate you in how to
maintainit yourself (which might not be a bad idea for your current situation with Ubuntu either). 

>>> Example of base system part :
>>> Recently I had to install pl-java on my FreeBSD workstation. There was a problem with libtrh, postgresql should be
recompiled
>>> with explicitly setting : -lpthread in /usr/local/src/postgresql-9.3.4/src/backend/Makefile, without this the
backendwould simply hang 
>>> at the very first invocation of a java function. This came after detailed following or email exchange of various
hackersgroups 
>>> in both pgsql and FreeBSD lists, to describe the issue as accurately as possible, to help debug as most as
possible,to talk 
>>> to the right people, to give them incentive to answer back, etc.
>> It seems to me that the reason you were custom compiling Postgres in the first place was a problem with the port.
I’msure tracking down the problem wasn’t easy, but that is not really relevant to the topic. Ports break sometimes (on
anyOS) and it would have been sufficient to contact the port maintainer about the issue. 
> No, i wasn't compiling postgresql from standard distribution because of a problem with the port.
> (although the port had the same exact behavior)
> I always run postgresql compiled by hand, since I see no reason to sacrifice my peace of mind
> for a short-lived joy going with the ports or PKGng system.
> As a matter of fact, PostgreSQL is among the few software packages that i would advice strongly
> against using ports or pkgs of any kind. Might work in Debian. Would not risk this in FreeBSD.

Although it is getting a bit specific, would you care to elaborate why you would advice strongly against using ports or
packagesfor Postgres on FreeBSD? Because that’s a rather strong statement you’re making and so far the only argument
I’veseen is that there is no port for pl/java. I’m curious as to why you are so strongly set on custom-compiling
Postgres.

BTW, isn’t the usual solution to a missing port to create your own (local) port? I can’t claim I have ever done that
(neverneeded to), but apparently that’s the way to go about it. The obvious benefit is that it will fit in with the
packagemanagement system, while you could even provide the port to others if you’d be willing to take responsibility
formaintaining that port. 

>> And yes, I have edited Makefiles, although the need hasn’t risen recently.
> With plain vanilla ports it is rarely needed.

True, that’s usually only necessary in the rare case that a port isn’t compiling or when you’re stubborn about not
wantingcertain dependencies (I know someone who used to be dead-set against gettext, for example). 

>> Oh, and please try not to top-post when replying on this list.
> I did just for this message, because i did not feel appropriate to quote anything that the previous
> poster wrote.

Fair enough.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.

On Fri, 11 Apr 2014 14:05:43 +0200
Alban Hertroys <haramrae@gmail.com> wrote:

> My advice to the OP:
>
> Install FreeBSD on a system to play around with, get a feel for how
> it works and whether you like it or not. See how it performs with
> Postgres on different file-systems; UFS2 or ZFS - UFS is the faster
> of the two, but ZFS makes up for that big time in maintainability if
> you provide it with enough memory. If you require locale-specific
> collations (native language specific sort ordering), check that it
> does what you expect.

Curious: Why not consider OpenBSD also?

SteveT

Steve Litt                *  http://www.troubleshooters.com/
Troubleshooting Training  *  Human Performance

On Fri, 11 Apr 2014 09:16:04 -0400
Steve Litt <slitt@troubleshooters.com> wrote:
> Curious: Why not consider OpenBSD also?

Or NetBSD.

--
D'Arcy J.M. Cain <darcy@druid.net>         |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 788 2246     (DoD#0082)    (eNTP)   |  what's for dinner.
IM: darcy@Vex.Net, VoIP: sip:darcy@druid.net

On 11/04/2014 15:05, Alban Hertroys wrote:
> Although it is getting a bit specific, would you care to elaborate why you would advice strongly against using ports
orpackages for Postgres on FreeBSD? Because that’s a rather strong statement  
> you’re making and so far the only argument I’ve seen is that there is no port for pl/java. I’m curious as to why you
areso strongly set on custom-compiling Postgres. BTW, isn’t the usual solution  
> to a missing port to create your own (local) port? I can’t claim I have ever done that (never needed to), but
apparentlythat’s the way to go about it. The obvious benefit is that it will fit in  
> with the package management system, while you could even provide the port to others if you’d be willing to take
responsibilityfor maintaining that port.  

pl/java has nothing to do with this. The argument against using packages/ports for postgresql
upgrades, is that upgrades in general involve :
- reading HISTORY thoroughly and understanding every bit of it, especially the migration part,
and the changes part
- backing up the current database
- installing the new binaries
- running pg_upgrade
- solving problems that pg_upgrade detects and trying again
- testing your in house C/Java/etc... functions
- testing your whole app + utilities against the new version

Now, tell me, how much of this can the /usr/ports/databases/postgresqlXX-server port can do?
Would you trust the system to do this for you in an automated maybe weekly  pkg upgrade task
that would handle e.g. cdrecord and postgresql-xxx in the same manner ?

Now about writing ports, i can say to you this is a PITA. Its a great concept, but you must truly
commit to having a part of your life slot maintaining the port you submitted. This could be fun at first,
but in the long run, this is not easy.

> Fair enough.

You are welcome :)

> Alban Hertroys
> --
> If you can't see the forest for the trees,
> cut the trees and you'll find there is no forest.
>
>
>


--
Achilleas Mantzios
Head of IT DEV
IT DEPT
Dynacom Tankers Mgmt

On Fri, 11 Apr 2014 17:15:28 +0300
Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:
> pl/java has nothing to do with this. The argument against using
> packages/ports for postgresql upgrades, is that upgrades in general
> involve :
> - reading HISTORY thoroughly and understanding every bit of it,
> especially the migration part, and the changes part

Do this before installing anything.

> - backing up the current database

Also this.

> - installing the new binaries

This is the only thing that the ports or pkgsrc (NetBSD) facility does
for you.

> - running pg_upgrade
> - solving problems that pg_upgrade detects and trying again
> - testing your in house C/Java/etc... functions
> - testing your whole app + utilities against the new version

After installing the upgrade all of this has to be done.  None of it is
going to be done by ports/pkgsrc/apt-get or any other install
facility including your own brain sweat.  All ports/pkgsrc does for you
is the install part.

> Now, tell me, how much of this can
> the /usr/ports/databases/postgresqlXX-server port can do? Would you

Step 3.  Period.

> trust the system to do this for you in an automated maybe weekly  pkg
> upgrade task that would handle e.g. cdrecord and postgresql-xxx in
> the same manner ?

I wouldn't trust any system to blindly install any program I certainly
wouldn't automate any upgrades on a production server.  The most I
would do automatically is build the latest package for manual
installation.

> Now about writing ports, i can say to you this is a PITA. Its a great
> concept, but you must truly commit to having a part of your life slot
> maintaining the port you submitted. This could be fun at first, but
> in the long run, this is not easy.

Why?  Once you submit your port to the community you could update when
you could or poke the community to do it for you.  All hail open source.

--
D'Arcy J.M. Cain <darcy@druid.net>         |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 788 2246     (DoD#0082)    (eNTP)   |  what's for dinner.
IM: darcy@Vex.Net, VoIP: sip:darcy@druid.net