Re: Best Strategy for Large Number of Images - Mailing list pgsql-general

From Marc Millas
Subject Re: Best Strategy for Large Number of Images
Date
Msg-id CADX_1aZhXLG9H20d9WMh5nAaWHPbWWjQngDk=V_cB2aVE+Oqzg@mail.gmail.com
Whole thread Raw
In response to Re: Best Strategy for Large Number of Images  (Estevan Rech <softrech@gmail.com>)
List pgsql-general
Hi,
when I am doing this I store the data in one postgres cluster, with some kind of id for each image , and the images in another with the id as link.
The customer app is written so that it issues a dedicated http request for each image. (I use nginx to create a dedicated "path".
pro: the images dont over charge the shared buffer cache of the data db., its easier to cache the images if its relevant perf wise. backups are much easier on the data db as its much smaller, .very easy to scale horizontaly by adding dedicated images db, .etc...
con: as with all 2 storage solution, even the disk option, you "may" encounter phase pb ie. id existing in one db and not in the other. but this depends on how you maintain the images.

my 2 cents


Marc MILLAS
Senior Architect
+33607850334



On Wed, Dec 15, 2021 at 9:12 PM Estevan Rech <softrech@gmail.com> wrote:
Adrian, 

I have an application that takes pictures of items and uses them as evidence in a report.

These images are saved and then used to generate a report.

Each item averages 10 photos and I have about 2 million photos currently, with an average growth of 1 million photos over the next year.

I think about using it within the database because of the ease of selecting records with the content of the images (when necessary).

I think my biggest concern is to continue to use this strategy for the next 2 years and encounter a PostgreSQL limitation or some failure and have to migrate database again or have significant loss of images. And on the contrary too, if I use it on disk and in 2 years I find some failure or file limit in the operating system (currently it's windows server 2016)...

Related to this is the backup service that seems to me that in a disaster, I can restore the database relatively quickly if it's in the database. On the disk, I believe that the restoration is much slower and I don't think it's reliable that all the images are copied/restored.

pgsql-general by date:

Previous
From: Gavan Schneider
Date:
Subject: Re: Best Strategy for Large Number of Images
Next
From: Adrian Klaver
Date:
Subject: Re: Why can't I have a "language sql" anonymous block?