Thread: An article about Etsy... and a migration from Postgres to MySQL

An article about Etsy... and a migration from Postgres to MySQL

From
"Milen A. Radev"
Date:
"When "clever" goes wrong: how Etsy overcame poor architectural choices"

http://arstechnica.com/business/news/2011/10/when-clever-goes-wrong-how-etsy-overcame-poor-architectural-choices.ars

--
Milen A. Radev

Re: An article about Etsy... and a migration from Postgres to MySQL

From
Simon Riggs
Date:
On Tue, Oct 4, 2011 at 10:18 AM, Milen A. Radev <milen@radev.net> wrote:

> "When "clever" goes wrong: how Etsy overcame poor architectural choices"
>
> http://arstechnica.com/business/news/2011/10/when-clever-goes-wrong-how-etsy-overcame-poor-architectural-choices.ars
>

This is very well written. It almost sounds like the problem was
caused by Postgres.

I take it as first evidence of a marketing department at the new MySQL
team. Hats off, guys and happy list monitoring.

I think we should keep the quote on our website to remind people...

"Etsy is the sum total of its data, so when it came to choosing where
to store our critical information there was only one sound choice.
PostgreSQL's strong emphasis on quality, stability, and data integrity
contribute to making it the premier open source database."

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: An article about Etsy... and a migration from Postgres to MySQL

From
PostgreSQL - Hans-Jürgen Schönig
Date:
On Oct 4, 2011, at 3:10 PM, Simon Riggs wrote:

> On Tue, Oct 4, 2011 at 10:18 AM, Milen A. Radev <milen@radev.net> wrote:
>
>> "When "clever" goes wrong: how Etsy overcame poor architectural choices"
>>
>> http://arstechnica.com/business/news/2011/10/when-clever-goes-wrong-how-etsy-overcame-poor-architectural-choices.ars
>>
>
> This is very well written. It almost sounds like the problem was
> caused by Postgres.
>


i got similar feeling when i have read this one ...
why should a switch to a system which cannot even properly do stored procs magically solve the problem?
it is like blaming the owner of a wine-yard for being drunk all day long ...


> I take it as first evidence of a marketing department at the new MySQL
> team. Hats off, guys and happy list monitoring.


yes, same feelings here as well ...

    regards,

        hans

--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de


Re: An article about Etsy... and a migration from Postgres to MySQL

From
Selena Deckelmann
Date:
Hi!

On Tue, Oct 4, 2011 at 6:10 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
>
> On Tue, Oct 4, 2011 at 10:18 AM, Milen A. Radev <milen@radev.net> wrote:
>
> > "When "clever" goes wrong: how Etsy overcame poor architectural choices"
> >
> >
http://arstechnica.com/business/news/2011/10/when-clever-goes-wrong-how-etsy-overcame-poor-architectural-choices.ars
> >
>
> This is very well written. It almost sounds like the problem was
> caused by Postgres.

I didn't read it that way. It seemed like a cautionary tale about
overuse of stored procedures to express business logic in a company
that had serious scaling and operational and communication problems
between DBAs, sysadmins and developers. The new-hires turned to a
technology stack that was well understood. And in the end, the article
mentions that they still have Postgres at the core.

I see this as a wake up call that our advocacy needs to focus on the
case studies, like that of Urban Airship, to demonstrate how to scale
infrastructure with Postgres. Keeping this information either secret
or difficult to find results in throwing out or scaling back use of
Postgres.

-selena
--
http://chesnok.com

Re: An article about Etsy... and a migration from Postgres to MySQL

From
"Jehan-Guillaume (ioguix) de Rorthais"
Date:
On 04/10/2011 17:12, Selena Deckelmann wrote:
> Hi!
>
> On Tue, Oct 4, 2011 at 6:10 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
>>
>> On Tue, Oct 4, 2011 at 10:18 AM, Milen A. Radev <milen@radev.net> wrote:
>>
>>> "When "clever" goes wrong: how Etsy overcame poor architectural choices"
>>>
>>>
http://arstechnica.com/business/news/2011/10/when-clever-goes-wrong-how-etsy-overcame-poor-architectural-choices.ars
>>>
>>
>> This is very well written. It almost sounds like the problem was
>> caused by Postgres.
>
> I didn't read it that way. It seemed like a cautionary tale about
> overuse of stored procedures to express business logic in a company
> that had serious scaling and operational and communication problems
> between DBAs, sysadmins and developers. The new-hires turned to a
> technology stack that was well understood. And in the end, the article
> mentions that they still have Postgres at the core.

That was my understanding as well. The new tech chief came from Yahoo
and built with his team an architecture they knew from their Flickr
experience...that is based on mysql.

In my opinion, it really means that they fail both in their choice
technically and humanly.

> I see this as a wake up call that our advocacy needs to focus on the
> case studies, like that of Urban Airship, to demonstrate how to scale
> infrastructure with Postgres. Keeping this information either secret
> or difficult to find results in throwing out or scaling back use of
> Postgres.

+1

> -selena
> --
> http://chesnok.com

Re: An article about Etsy... and a migration from Postgres to MySQL

From
Chris Travers
Date:
On Tue, Oct 4, 2011 at 8:12 AM, Selena Deckelmann <selena@chesnok.com> wrote:

> I didn't read it that way. It seemed like a cautionary tale about
> overuse of stored procedures to express business logic in a company
> that had serious scaling and operational and communication problems
> between DBAs, sysadmins and developers. The new-hires turned to a
> technology stack that was well understood. And in the end, the article
> mentions that they still have Postgres at the core.


That's close to how I read it. However there have to be some missing
pieces because in general, my experience is that ORM's often write bad
queries and that complex operations are often easier to tune in stored
procs than in ORMs.  Could the issue have to do with caching in the
middleware?


>
> I see this as a wake up call that our advocacy needs to focus on the
> case studies, like that of Urban Airship, to demonstrate how to scale
> infrastructure with Postgres. Keeping this information either secret
> or difficult to find results in throwing out or scaling back use of
> Postgres.

Agreed.

Best Wishes,
Chris Travers

Re: An article about Etsy... and a migration from Postgres to MySQL

From
Joshua Kramer
Date:
> I see this as a wake up call that our advocacy needs to focus on the
> case studies, like that of Urban Airship, to demonstrate how to scale
> infrastructure with Postgres. Keeping this information either secret
> or difficult to find results in throwing out or scaling back use of
> Postgres.

Hey, Ned Lilly - are you on this list?  Do you have any examples of highly
scaled xTuple installations?  (For those who are unaware, xTuple is an
open source ERP solution based on a Qt frontend.  All of the business
logic resides in Postgres stored procedures.)

http://www.xtuple.org

Cheers,
-JK

Re: An article about Etsy... and a migration from Postgres to MySQL

From
Chris Travers
Date:
On Tue, Oct 4, 2011 at 9:42 AM, Joshua Kramer <josh@globalherald.net> wrote:
>
>> I see this as a wake up call that our advocacy needs to focus on the
>> case studies, like that of Urban Airship, to demonstrate how to scale
>> infrastructure with Postgres. Keeping this information either secret
>> or difficult to find results in throwing out or scaling back use of
>> Postgres.
>
> Hey, Ned Lilly - are you on this list?  Do you have any examples of highly
> scaled xTuple installations?  (For those who are unaware, xTuple is an open
> source ERP solution based on a Qt frontend.  All of the business logic
> resides in Postgres stored procedures.)
>
I can't speak for xTuple, but for LedgerSMB, we have at least one user
who processes payment runs of 5k invoices at a time via Perl/CGI and
PostgreSQL (total of 20k invoices per week, and expecting this number
to rise) with the main logic handled in stored procedures.  If it was
just a matter of selection, we could run that selection in a few
seconds but a lot of info has to be written to the db so that states
don't change between web requests.  Consequently it takes only a bit
longer than that.  The real bottleneck is actually the CGI scripts
which generate HTML forms representing 5000 invoices.......

Keep in mind this is being run against a database with a million
invoices in it, and probably 10 million invoice lines, and a portion
of the lines must be summed up to ensure the invoice is fully paid
off.  We have found PostgreSQL performs very well.

In general, I have found that tuning stored procedures is easier than
tuning ORM-generated queries, which is why I am convinced there must
be more to the story than we are being told.

Best Wishes,
Chris Travers

Re: An article about Etsy... and a migration from Postgres to MySQL

From
Chris Travers
Date:
(Forgot to forward to the list)

> I can't speak for xTuple, but for LedgerSMB, we have at least one user
> who processes payment runs of 5k invoices at a time via Perl/CGI and
> PostgreSQL (total of 20k invoices per week, and expecting this number
> to rise) with the main logic handled in stored procedures.  If it was
> just a matter of selection, we could run that selection in a few
> seconds but a lot of info has to be written to the db so that states
> don't change between web requests.  Consequently it takes only a bit
> longer than that.  The real bottleneck is actually the CGI scripts
> which generate HTML forms representing 5000 invoices.......
>
Sorry, with the persistent, discretionary locking info written it
takes only a bit over 10x the pure read-only performance.  This could
probably be optimized further for disk I/O, but the web interface is a
much bigger bottleneck.

Best Wishes,
Chris Travers

Re: An article about Etsy... and a migration from Postgres to MySQL

From
Darren Duncan
Date:
Simon Riggs wrote:
> On Tue, Oct 4, 2011 at 10:18 AM, Milen A. Radev <milen@radev.net> wrote:
>
>> "When "clever" goes wrong: how Etsy overcame poor architectural choices"
>>
>> http://arstechnica.com/business/news/2011/10/when-clever-goes-wrong-how-etsy-overcame-poor-architectural-choices.ars
>>
>
> This is very well written. It almost sounds like the problem was
> caused by Postgres.
>
> I take it as first evidence of a marketing department at the new MySQL
> team. Hats off, guys and happy list monitoring.
>
> I think we should keep the quote on our website to remind people...
>
> "Etsy is the sum total of its data, so when it came to choosing where
> to store our critical information there was only one sound choice.
> PostgreSQL's strong emphasis on quality, stability, and data integrity
> contribute to making it the premier open source database."
>

I don't see that quote in the article.  I also saw comments talking about the
article being rewritten. -- Darren Duncan


Re: An article about Etsy... and a migration from Postgres to MySQL

From
Ned Lilly
Date:
Hi Josh,

We've got installations with hundreds of concurrent heads-down users
doing very transaction-intensive things, with very high volumes of
human-entered Sales Orders (thousands a day), machine-managed EDI
(thousands of invoices, shipping notices, purchase orders, etc.), and
tens of thousands of complex general ledger transactions a day,
including serial or lot-controlled inventory movement across multiple
locations.  Some users have millions of SKUs, hundreds of thousands of
contacts and accounts (customers, vendors, etc) in the CRM subsystem, etc.

To the best of my knowledge, we have never had performance problems
caused by PostgreSQL or our use of stored procedures to handle the
business logic.  When we do have issues, they are easily addressed by
adding indexes or fixing inefficiences in what client-side code we do
still have.  Happily, living in the open source world, those issues are
typically brought to our attention (and solved) rather quickly :)

Cheers,
Ned


On 10/4/2011 12:42 PM, Joshua Kramer wrote:
>
>> I see this as a wake up call that our advocacy needs to focus on the
>> case studies, like that of Urban Airship, to demonstrate how to scale
>> infrastructure with Postgres. Keeping this information either secret
>> or difficult to find results in throwing out or scaling back use of
>> Postgres.
>
> Hey, Ned Lilly - are you on this list? Do you have any examples of
> highly scaled xTuple installations? (For those who are unaware, xTuple
> is an open source ERP solution based on a Qt frontend. All of the
> business logic resides in Postgres stored procedures.)
>
> http://www.xtuple.org
>
> Cheers,
> -JK
>

--
Ned Lilly
President and CEO
xTuple
119 West York Street
Norfolk, VA 23510
tel. 757.461.3022 x101
email: ned@xtuple.com
www.xtuple.com