Thread: Linux / PostgreSQL question
I was just doing some huge operations with PostgreSQL and it all crashed out with a "too many files open" message plastered all over the place.. Now in /proc/sys/fs/file-max there is only 4096, that limit could have easily been reached. Does changing the value in the file effectively change the limit system-wide? I changed it and rebooted but it was set right back to 4096.. I've been out of the Linux loop for a long time (FreeBSD junkie now) so I don't know how to set that up to permanently change the limit. Marc: You use Linux, don't you? Have you ever run into this? Any help is very appreciated, thanks! -Mitch
"Mitch Vincent" <mitch@venux.net> writes: > I was just doing some huge operations with PostgreSQL and it all crashed out > with a "too many files open" message plastered all over the place.. > > Now in /proc/sys/fs/file-max there is only 4096, that limit could have > easily been reached. Does changing the value in the file effectively change > the limit system-wide? I changed it and rebooted but it was set right back > to 4096.. A reboot would reset values in proc... For Red Hat Linux 6.2 and up, you would add a line to /etc/sysctl.conf: # Max open files: fs.file-max = 8192 -- Trond Eivind Glomsrød Red Hat, Inc.
On Wed, Sep 13, 2000 at 01:04:32PM -0700, Mitch Vincent wrote: > I was just doing some huge operations with PostgreSQL and it all crashed out > with a "too many files open" message plastered all over the place.. > This is really a Linux question, not postgresql, but you knew that... (I'm keeping hackers on this message, so if it comes up again, the answer's in the archives with the question) > Now in /proc/sys/fs/file-max there is only 4096, that limit could have > easily been reached. Does changing the value in the file effectively change > the limit system-wide? I changed it and rebooted but it was set right back > to 4096.. I've been out of the Linux loop for a long time (FreeBSD junkie > now) so I don't know how to set that up to permanently change the limit. Almost right. Why'd you reboot? It's a runtime configuration. Proc is not a file system, it's a pseudo-filesystem interface to kernel internals. Just do something like: echo 32768 > /proc/sys/fs/file-max And you may need to up the number of inodes, too: echo 65536 > /proc/sys/fs/inode-max You'll probably want to put these in rc.boot, or rc.local, or something, to set this at boot time, as well. Ross -- Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> NSBRI Research Scientist/Programmer Computer and Information Technology Institute Rice University, 6100 S. Main St., Houston, TX 77005
I didn't see any mention of it on the TODO so I thought I'd ask if anyone had thought about full test indexing for 7.1 (I'm guessing not).. If not, I'd like to suggest it be put on the TODO -- if nothing else so someone could pick it up in the far future if they wanted to.. It doesn't seem like too many are worried about it so the request is pretty selfish, though I'm sure it would help many people especially after 7.1 and TOAST make text fields unlimited in size. Thanks! -Mitch
See contrib/fulltextindex. [ Charset ISO-8859-1 unsupported, converting... ] > I didn't see any mention of it on the TODO so I thought I'd ask if anyone > had thought about full test indexing for 7.1 (I'm guessing not).. > > If not, I'd like to suggest it be put on the TODO -- if nothing else so > someone could pick it up in the far future if they wanted to.. It doesn't > seem like too many are worried about it so the request is pretty selfish, > though I'm sure it would help many people especially after 7.1 and TOAST > make text fields unlimited in size. > > Thanks! > > -Mitch > > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian wrote:
>
> See contrib/fulltextindex.
An easy answer, but not a very good solution in the real world.
contrib/fulltextindex requires you to jump through hoops in developing
queries to retrieve your data. It's also very space-inefficient in that
a table with a fulltextindex on a field needs another table with a
complete set of values for that field, as well as any substrings of that
field, and then it wants two indexes on that table. Add that up!
It would be nice to see a true index which was full text. It would be
nice to see a true index which allowed an individual field to index to
many entries through a function interface. This would straightforwardly
allow people to create their own simple functions to perform full-text,
keyword or other indexing schemes quite simply.
It naively appears to me that the function interface is moving closer to
achieving this with the enhancements in 7.1 to the use of setof()
returns combined with the earlier enhancement to indexing on function
results.
If a function fulltextindex(text) returned a setof() the substrings in
its text argument, how hard will it be to index on that return value and
allow WHERE field=fulltextindex('substring') to use that index?
Of course such a fulltextindex() function would have to know not to do
any processing on the string when called in the second situation. Is it
possible for functions to do this sort of trick? It seems a bit beyond
the pale!
I would _love_ to see full-text or keyword indexing natively in
PostgreSQL.
Regards, Andrew.
--
_____________________________________________________________________ Andrew McMillan, e-mail:
Andrew@cat-it.co.nz
Catalyst IT Ltd, PO Box 10-225, Level 22, 105 The Terrace, Wellington
Me: +64 (21) 635 694, Fax: +64 (4) 499 5596, Office: +64 (4) 499 2267
Andrew McMillan wrote:
>
> Bruce Momjian wrote:
> >
> > See contrib/fulltextindex.
>
> An easy answer, but not a very good solution in the real world.
>
> contrib/fulltextindex requires you to jump through hoops in developing
> queries to retrieve your data. It's also very space-inefficient in that
> a table with a fulltextindex on a field needs another table with a
> complete set of values for that field, as well as any substrings of that
> field, and then it wants two indexes on that table. Add that up!
>
> It would be nice to see a true index which was full text. It would be
> nice to see a true index which allowed an individual field to index to
> many entries through a function interface. This would straightforwardly
> allow people to create their own simple functions to perform full-text,
> keyword or other indexing schemes quite simply.
>
> It naively appears to me that the function interface is moving closer to
> achieving this with the enhancements in 7.1 to the use of setof()
> returns combined with the earlier enhancement to indexing on function
> results.
>
> If a function fulltextindex(text) returned a setof() the substrings in
> its text argument, how hard will it be to index on that return value and
> allow WHERE field=fulltextindex('substring') to use that index?
>
> Of course such a fulltextindex() function would have to know not to do
> any processing on the string when called in the second situation. Is it
> possible for functions to do this sort of trick? It seems a bit beyond
> the pale!
>
> I would _love_ to see full-text or keyword indexing natively in
> PostgreSQL.
I tottally agree with you. FTI is not a good solution. It seems natural
that PostgreSQL will have a built-in (and better) FTI, now that the
TOAST project will be implemented in 7.1.
Poul L. Christiansen
Andrew McMillan <Andrew@catalyst.net.nz> writes: > > I would _love_ to see full-text or keyword indexing natively in > PostgreSQL. > I would rather love to see a great fulltext engine integrated with PostgreSQL. It would be cool to be able to have ranked results and different ways of searching the index(regex, soundex, synonym). Having a solution that where pluggable for different languages would be nice. regards, Gunnar