Re: BLOB's bypassing the OS Filesystem for better Image - Mailing list pgsql-performance

    My laptop reads an entire compiled linux kernel (23000 files totalling
250 MBytes) in about 1.5 seconds if they're in cache. It's about 15.000
files/second. You think it's slow ? If you want to read them in random
order, you'll probably use something else than a laptop drive, but you get
the idea.

    Filesystem is reiser4.

    If you use ext2, you'll have a problem with many files in the same
directory because I believe it uses a linear search, hence time
proportional to the number of files (ouch). I once tried to put a million
1-kbyte files in a directory ; it was with reiserfs 3, and it didn't seem
to feel anything close to molested. I believe it took some 10 minutes, but
it was two years ago so I don't remember very well. NTFS took a day, that
I do remember ! By curiosity I tried to stuff 1 million 1KB files in a
directory on my laptop right now, It took a bit less than two minutes.

On Tue, 26 Apr 2005 11:34:45 +0200, apoc9009@yahoo.de <apoc9009@yahoo.de>
wrote:

>
>> Which filesystems? I know ext2 used to have issues with many-thousands
>> of files in one directory, but that was a directory scanning issue
>> rather than file reading.
>
>  From my Point of view i think it is better to let one Process do the
> operation to an Postgres Cluster Filestructure as
> if i bypass it with a second process.
>
> For example:
> A User loads up some JPEG Images over HTTP.
>
> a) (Filesystem)
> On Filesystem it would be written in a File with a random generated
> Filename (timestamp or what ever)
> (the Directory Expands and over a Million Fileobjects with will be
> archived, written, replaced, e.t.c)
>
> b) (Database)
> The JPEG Image Information will be stored into a BLOB as Part of a
> special Table, where is linked
> wit the custid of the primary Usertable.
>
>  From my Point of view is any outside Process (must be created, forked,
> Memory allocated, e.t.c)
> a bad choice. I think it is generall better to Support the Postmaster in
> all Ways and do some
> Hardware RAID Configurations.
>
>>> My Question:
>>> Can i speedup my Webapplication if i store my JPEG Images with small
>>> sizes inside my PostgreSQL Database (on verry large Databasis over 1
>>> GByte
>>> and above without Images at this time!)
>>
>>
>> No. Otherwise the filesystem people would build their filesystems on
>> top of PostgreSQL not the other way around. Of course, if you want
>> image updates to be part of a database transaction, then it might be
>> worth storing them in the database.
>
> Hmm, ORACLE is going the other Way. All File Objects can be stored into
> the Database if the DB
> has the IFS Option (Database Filesystem and Fileserver insinde the
> Database).
>
>
>>
>>> I hope some Peoples can give me a Tip or Hint where in can
>>> some usefull Information about it!
>>
>> Look into having a separate server (process or actual hardware) to
>> handle requests for static text and images. Keep the Java server for
>> actually processing
>
>
> Thanks
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
>                http://archives.postgresql.org
>