Re: [GENERAL] storing large files in database - performance - Mailing list pgsql-general

From Peter J. Holzer
Subject Re: [GENERAL] storing large files in database - performance
Date
Msg-id 20170516161440.GE20574@hjp.at
Whole thread Raw
In response to [GENERAL] storing large files in database - performance  (Eric Hill <Eric.Hill@jmp.com>)
List pgsql-general
On 2017-05-16 12:25:03 +0000, Eric Hill wrote:
> I searched and found a few discussions of storing large files in the database
> in the archives, but none that specifically address performance and how large
> of files can realistically be stored in the database.
>
>
>
> I have a node.js application using PostgreSQL to store uploaded files.  The
> column in which I am storing the file contents is of type “bytea” with
> “Storage” type set to “EXTENDED”.

I have mentioned this little experiment before, but I finally put the
results on my web site: https://www.hjp.at/databases/blob-bench/

(Please note that so far I have run this only on one system.
Generalizing to other systems might be premature).


> Storing a 12.5 MB file is taking 10 seconds, and storing a 25MB file
> is taking 37 seconds.  Two notable things about those numbers:  It
> seems like a long time, and the time seems to grow exponentially with
> file size rather than linearly.
>
>
>
> Do these numbers surprise you?

Yes. on my system, storing a 25 MB bytea value takes well under 1 second.

        hp

--
   _  | Peter J. Holzer    | we build much bigger, better disasters now
|_|_) |                    | because we have much more sophisticated
| |   | hjp@hjp.at         | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>

Attachment

pgsql-general by date:

Previous
From: Tom Lane
Date:
Subject: Re: [GENERAL] Different query plan used for the same query depending on how parameters are passed
Next
From: Martin Goodson
Date:
Subject: Re: [GENERAL] EnterpriseDB installed PostgreSQL 9.6 vs. REPMGR. Round4 - compilation issues on RHEL 7.2