Thread: Slow parliament election processing in Estonia blamed on Postgres

I received the following email via our press email address.  It seems
slow election processing in Estonia was blamed on the Postgres database.
Any idea how to respond to this?

---------------------------------------------------------------------------

Hello,

There was parliament elections in Estonia last sunday. Everything else
went fine except the system lagged for most important hours during
entering the votes from the departments.

Helmes (www.helmes.ee), the developer of many other national systems,
released a note to media on Monday accusing Postgresql database that it
did not manage with the dataflow. Although to calculate for a second, then
about 600 inserting people from the election departments who pass totally
50'000 rows to tables during two hours is a very slow day for most
sql-systems. They mainly need only the next information for voting -
department, candidate, total votes. So the real fault was in the brain of
the database developer - it is silly to say subaru is bad if you cant
drive rally-style and crash the car in the first corner.

I hope Postgresql management will take action on this and give a reply in
media, as accuses were against "freeware databases" - issues like this
give them bad reputation and these databases wont be taken seriously the
next time. I have used postgre and mysql for ten years to build up
photobanks with millions of pictures inside and developed other large
systems and met no mistakes, it all begins with good database design.

News headlines in estonian concerning this topic:
http://www.delfi.ee/news/rk/uudised/helmes-viga-oli-vabavaralises-andmebaasimootoris.d?id=41628665

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +

On 3/9/11 6:59 AM, Bruce Momjian wrote:
> I received the following email via our press email address.  It seems
> slow election processing in Estonia was blamed on the Postgres database.
> Any idea how to respond to this?

I responded to the sender with this:

"Yes, I'd heard about this.  Currently, we have no contacts in the
Estonian press, nor even a community volunteer who could translate a
response into Estonian.  So I'm not sure how we can "respond" other than
in people's blogs."

Other than that, I can just say "A bad workman blames his tools."

--
                                  -- Josh Berkus
                                     PostgreSQL Experts Inc.
                                     http://www.pgexperts.com

On Wed, Mar 9, 2011 at 1:45 PM, Gary Carter
<gary.carter@enterprisedb.com> wrote:
> How about stating something along the lines of:
> There are hundreds if not thousands of applications using Postgres that
> experience much heavier transactions loads and run in an exemplary fashion.
> Some examples include:
> Organization name, size of database or simultaneous users, or number of
> transactions per day (granted we may need to leave off the org name, but you
> get the idea)

I don't know exactly the type/size of the election in Estonia, but it
may be similar to an Argentina's Salta Province Open Primary
Elections, where we use PostgreSQL without problems since several
years,  the last facts are:

~700 "election departments"
~500 candidates
~500000 registered voters
~32000 rows of accumulate vote totals (votes for a candidate in a
city/region/department)

Each "election department" has an "Election Supervisor" that send the
accumulated results through fax or internet, the image stored in a
PostgreSQL bytea field, then loaded and shown several times to data
entry operators and administrators.

There was syncrhonic replication to a local slave server and
asyncrhonic replication to a remote server (where publication of
results were done) using pyreplica (python triggers and connector):
https://docs.google.com/present/view?id=dd9bm82g_402fjtsdmdd

In importants hours there are around 300 active connections to each database.

The whole process time was around 6 hours (until the last result was
received), the postgresql load was minimal, everything ran smoothly.

Replication logs also works as audit trails (~80000 sql logs), and
they can be used to simulate the load reprocessing them (rebuilding
the database exactly as the data came in) only took minutes.

Database size is 168MB, 1.5GB uncompressed

Server Hardware is normal (quad core xeon X3220, 4GB RAM, SATA RAID1)

Hope this helps,

Best regards,

Mariano Reingart
http://www.sistemasagiles.com.ar
http://reingart.blogspot.com

Excerpts from Mariano Reingart's message of jue mar 10 02:49:16 -0300 2011:

> I don't know exactly the type/size of the election in Estonia, but it
> may be similar to an Argentina's Salta Province Open Primary
> Elections, where we use PostgreSQL without problems since several
> years,

Mariano, this should be published as a case study somewhere.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support