Re: Postgresql in a Virtual Machine - Mailing list pgsql-performance

On Tue, 26 Nov 2013, Xenofon Papadopoulos wrote:

We have been running several Postgres databases on VMs for the last 9
months. The largest one currently has a few hundreds of millions of rows
(~1.5T of data, ~100G of frequently queried data ) and performs at ~1000
tps. Most of our transactions are part of a 2PC, which effectively results
to high I/O as asynchronous commit is disabled.

Main benefits so far:

- ESXi HA makes high availability completely transparent and reduces the
number of failover servers (we're running N+1 clusters)

- Our projects' load can often miss our expectations, and it changes over
the time. Scaling up/down has helped us cope.

how do you add another server without having to do a massive data copy in the process?

David Lang

- Live relocation of databases helps with hardware upgrades and spreading
of load.

Main issues:

- We are not overprovisioning at all (using virtualization exclusively for
the management benefits), so we don't know its impact to performance.

- I/O has often been a bottleneck. We are not certain whether this is due
to the impact of virtualization or due to mistakes in our sizing and
configuration. So far we have been coping by spreading the load across
more spindles and by increasing the memory.





On Tue, Nov 26, 2013 at 1:26 AM, Merlin Moncure <mmoncure@gmail.com> wrote:

On Mon, Nov 25, 2013 at 4:57 PM, David Lang <david@lang.hm> wrote:

On Mon, 25 Nov 2013, Merlin Moncure wrote:

On Mon, Nov 25, 2013 at 2:01 PM, Lee Nguyen <leemobile@gmail.com>

wrote:

Hi,

Having attended a few PGCons, I've always heard the remark from a few
presenters and attendees that Postgres shouldn't be run inside a VM.

That

bare metal is the only way to go.

Here at work we were entertaining the idea of running our Postgres
database
on our VM farm alongside our application vm's.  We are planning to run

a

few
Postgres synchronous replication nodes.

Why shouldn't we run Postgres in a VM?  What are the downsides? Does
anyone
have any metrics or benchmarks with the latest Postgres?

Unfortunately (and it really pains me to say this) we live in an
increasingly virtualized world and we just have to go ahead and deal
with it.  I work at a mid cap company and we have a zero tolerance
policy in terms of applications targeting hardware: in short, you
can't.  VMs have downsides: you get less performance per buck and have
another thing to fail but the administration advantages are compelling
especially for large environments.  Furthermore, for any size company
it makes less sense to run your own data center with each passing day;
the cloud providers are really bringing up their game. This is
economic specialization at work.

being pedantic, you can get almost all the management benefits on bare
metal, and you can rent bare metal from hosting providors, cloud VMs are

not

the only option. 'Cloud' makes sense if you have a very predictably spiky
load and you can add/remove machines to meet that load, but if you end up
needing to have the machines running a significant percentage of the

time,

dedicated boxes are cheaper (as well as faster)

Well, that depends on how you define 'most'.  The thing is for me is
that for machines around the office (just like with people) about 10%
of them do 90% of the work.  Being able to slide them around based on
that (sometime changing) need is a tremendous time and cost saver.
For application and infrastructure development dealing with hardware
is just a distraction.   I'd rather click on some interface and say,
'this application needs 25k iops guaranteed' and then make a cost
driven decision on software optimization.  It's hard to let go after
decades of hardware innovation (the SSD revolution was the final shoe
to drop) but for me the time has finally come.  As recently as a year
ago I was arguing databases needed to be run against metal.

merlin


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance