Re: [GENERAL] Caching and Blobs in PG? Was: Can PG replace redis,amqp, s3 in the future? - Mailing list pgsql-general

From Peter J. Holzer
Subject Re: [GENERAL] Caching and Blobs in PG? Was: Can PG replace redis,amqp, s3 in the future?
Date
Msg-id 20170505201925.GC19234@hjp.at
Whole thread Raw
In response to Re: [GENERAL] Caching and Blobs in PG? Was: Can PG replace redis,amqp, s3 in the future?  (John R Pierce <pierce@hogranch.com>)
List pgsql-general
On 2017-05-05 11:46:55 -0700, John R Pierce wrote:
> On 5/5/2017 11:28 AM, Peter J. Holzer wrote:
>
>     On 2017-05-04 23:08:25 +0200, Sven R. Kunze wrote:
>
>         On 03.05.2017 12:57, Thomas Güttler wrote:
>
>             Am 02.05.2017 um 05:43 schrieb Jeff Janes:
>
>                 No.  You can certainly use PostgreSQL to store blobs.  But then, you
>                 need to store the PostgreSQL data **someplace**.
>                 If you don't store it in S3, you have to store it somewhere else.
>
>             I don't understand what you mean here. AFAIK storing blobs in PG is not
>             recommended since it is not very efficient.
>
>         Seems like several people here disagree with this conventional wisdom.
>
>     I think it depends very much on what level of "efficiency" you need. On
>     my home server (i5 processor, 32GB RAM, Samsung 850 SSD - not a piece of
>     junk, but not super powerful either) I can retrieve a small blob from a
>     100GB table in about 0.1 ms, and for large blobs the speed approaches
>     200MB/s. For just about everything I'd do on that server (or even at
>     work) this is easily fast enough.
>
>
> S3 is often used for terabyte to petabyte file collections.   I would not want
> to burden my relational database with this.

I repeat the the first sentence I wrote: "I think it depends very much
on what level of 'efficiency' you need." Just because some people need
to store petabytes of blob data doesn't mean everybody does. If you need
to store petabytes of blobs, PostgreSQL may not be the right tool. But
it may be the right tool if you just need to store a few thousand PDFs.
To tell people to never store blobs in PostgreSQL because PostgreSQL is
"not efficient" is just bullshit. There are many factors which determine
how you should store your data, and "efficiency" (however that is
defined, if it's defined at all and not just used as a buzzword) is only
one of them - and rarely, in my experience, the most important one.

        hp

--
   _  | Peter J. Holzer    | A coding theorist is someone who doesn't
|_|_) |                    | think Alice is crazy.
| |   | hjp@hjp.at         | -- John Gordon
__/   | http://www.hjp.at/ |    http://downlode.org/Etext/alicebob.html

Attachment

pgsql-general by date:

Previous
From: Vik Fearing
Date:
Subject: Re: [GENERAL] Vaccum Query
Next
From: Bill Moran
Date:
Subject: Re: [GENERAL] Can PG replace redis, amqp, s3 in the future?