Re: Too many open files from postgres. - Mailing list pgsql-general

From Mavinakuli, Prasanna (STSD)
Subject Re: Too many open files from postgres.
Date
Msg-id 93DACB38648AC54CA322B72DA1BDB5A13B1EA5@bgeexc04.asiapacific.cpqcorp.net
Whole thread Raw
List pgsql-general
Also (obvsly) we aren't able to get a new connection after hitting max
fd limit..

Thx,
Prasanna.
-----Original Message-----
From: Mavinakuli, Prasanna (STSD)
Sent: Wednesday, August 09, 2006 10:10 AM
To: pgsql-general@postgresql.org
Cc: Mavinakuli, Prasanna (STSD)
Subject: Too many open files from postgres.

Hello All,

We are using postgres with HP-UX.
Usually We will have many records,and will do vacuum once in 2 days.

We have observed that postgres opens many files in data directory And
won't close till next vacuum (I experience it) When it hits max limit,
we won't able to open any other files.

So what's the solution ..?

1)Is it really postgres issue..doesn't it close opened files immediately
after completion of writing..?
If so how do we handle the scenario of hitting max fd within next
immediate vaccuming..?

2)ideally what's the size of each file..?when it opens second
file..?what's criterion..?
I can observe files of different sizes in the same directories..?


Thx in advance,
Prasanna.

*lsof* gives following output.

postmaste 22481 sfmdb    9u   REG             64,0x8     16384   6562
/var/opt/sfmdb/pgsql/base/17142/16635
postmaste 22481 sfmdb   10u   REG             64,0x8      8192   4373
/var/opt/sfmdb/pgsql/base/17142/16595
postmaste 22481 sfmdb   11u   REG             64,0x8     16384   6563
/var/opt/sfmdb/pgsql/base/17142/16636
postmaste 22481 sfmdb   12u   REG             64,0x8    180224   6510
/var/opt/sfmdb/pgsql/base/17142/16614
postmaste 22481 sfmdb   13u   REG             64,0x8    204800   4341
/var/opt/sfmdb/pgsql/base/17142/1259
postmaste 22481 sfmdb   14u   REG             64,0x8     49152   6506
/var/opt/sfmdb/pgsql/base/17142/16613
postmaste 22481 sfmdb   15u   REG             64,0x8     98304   6475
/var/opt/sfmdb/pgsql/base/17142/16610
postmaste 22481 sfmdb   16u   REG             64,0x8    360448   4340
/var/opt/sfmdb/pgsql/base/17142/1249
postmaste 22481 sfmdb   17u   REG             64,0x8    106496   6569
/var/opt/sfmdb/pgsql/base/17142/16640
postmaste 22481 sfmdb   18u   REG             64,0x8    106496   4358
/var/opt/sfmdb/pgsql/base/17142/16392
postmaste 22481 sfmdb   19u   REG             64,0x8     16384   3673
/var/opt/sfmdb/pgsql/base/17142/16652
postmaste 22481 sfmdb   20u   REG             64,0x8     57344   4337
/var/opt/sfmdb/pgsql/base/17142/1247
postmaste 22481 sfmdb   21u   REG             64,0x8     16384   6500
/var/opt/sfmdb/pgsql/base/17142/16612
postmaste 22481 sfmdb   22u   REG             64,0x8     16384   4372
/var/opt/sfmdb/pgsql/base/17142/16418
postmaste 22481 sfmdb   23u   REG             64,0x8     32768   6568
/var/opt/sfmdb/pgsql/base/17142/16639
postmaste 22481 sfmdb   24u   REG             64,0x8    122880   6570
/var/opt/sfmdb/pgsql/base/17142/16641
postmaste 22481 sfmdb   25u   REG             64,0x8    540672   4224
/var/opt/sfmdb/pgsql/base/17142/1255
postmaste 22481 sfmdb   26u   REG             64,0x8     16384   6554
/var/opt/sfmdb/pgsql/base/17142/16630
postmaste 22481 sfmdb   27u   REG             64,0x8     40960   4356
/var/opt/sfmdb/pgsql/base/17142/16390
postmaste 22481 sfmdb   28u   REG             64,0x8      8192   4360
/var/opt/sfmdb/pgsql/base/17142/16396
postmaste 22481 sfmdb   29u   REG             64,0x8     16384   6109
/var/opt/sfmdb/pgsql/base/17142/16605
postmaste 22481 sfmdb   30u   REG             64,0x8     16384   4361
/var/opt/sfmdb/pgsql/base/17142/16398
postmaste 22481 sfmdb   31u   REG             64,0x8     16384   6460
/var/opt/sfmdb/pgsql/base/17142/16606
postmaste 22481 sfmdb   32u   REG             64,0x8      8192   4362
/var/opt/sfmdb/pgsql/base/17142/16400
postmaste 22481 sfmdb   33u   REG             64,0x8     16384   6553
/var/opt/sfmdb/pgsql/base/17142/16629
postmaste 22481 sfmdb   34u   REG             64,0x8     16384   6574
/var/opt/sfmdb/pgsql/base/17142/16647
postmaste 22481 sfmdb   35u   REG             64,0x8     16384   6634
/var/opt/sfmdb/pgsql/base/17142/17175
postmaste 22481 sfmdb   36u   REG             64,0x8     16384   6461
/var/opt/sfmdb/pgsql/base/17142/16607
postmaste 22481 sfmdb   37u   REG             64,0x8      8192   4342
/var/opt/sfmdb/pgsql/base/17142/16384
postmaste 22481 sfmdb   38u   REG             64,0x8     16384   6516
/var/opt/sfmdb/pgsql/base/17142/16616
postmaste 22481 sfmdb   39u   REG             64,0x8     24576   4344
/var/opt/sfmdb/pgsql/base/17142/16386
postmaste 22481 sfmdb   40u   REG             64,0x8     16384   6565
/var/opt/sfmdb/pgsql/base/17142/16637
postmaste 22481 sfmdb   41u   REG             64,0x8     16384   4359
/var/opt/sfmdb/pgsql/base/17142/16394
postmaste 22481 sfmdb   42u   REG             64,0x8     16384   5566
/var/opt/sfmdb/pgsql/base/17142/16604
postmaste 22481 sfmdb   43u   REG             64,0x8      8192   6752
/var/opt/sfmdb/pgsql/base/17142/17244
postmaste 22481 sfmdb   44u   REG             64,0x8      8192   6755
/var/opt/sfmdb/pgsql/base/17142/17249
postmaste 22481 sfmdb   45u   REG             64,0x8      8192   6746
/var/opt/sfmdb/pgsql/base/17142/17234
postmaste 22481 sfmdb   46u   REG             64,0x8      8192   6743
/var/opt/sfmdb/pgsql/base/17142/17229
postmaste 22481 sfmdb   47u   REG             64,0x8   1228800   6759
/var/opt/sfmdb/pgsql/base/17142/17256
postmaste 22481 sfmdb   48u   REG             64,0x8      8192   6623
/var/opt/sfmdb/pgsql/base/17142/17145
postmaste 22481 sfmdb   49u   REG             64,0x8      8192   6627
/var/opt/sfmdb/pgsql/base/17142/17165
postmaste 22481 sfmdb   50u   REG             64,0x8      8192   6638
/var/opt/sfmdb/pgsql/base/17142/17180
postmaste 22481 sfmdb   51u   REG             64,0x8      8192   3954
/var/opt/sfmdb/pgsql/base/17142/17150
postmaste 22481 sfmdb   52u   REG             64,0x8      8192   6630
/var/opt/sfmdb/pgsql/base/17142/17170
postmaste 22481 sfmdb   53u   REG             64,0x8    368640   6771
/var/opt/sfmdb/pgsql/base/17142/17281
postmaste 22481 sfmdb   54u   REG             64,0x8      8192   6664
/var/opt/sfmdb/pgsql/base/17142/17195
postmaste 22481 sfmdb   55u   REG             64,0x8     16384   6700
/var/opt/sfmdb/pgsql/base/17142/17210
postmaste 22481 sfmdb   56u   REG             64,0x8      8192   6673
/var/opt/sfmdb/pgsql/base/17142/17200
postmaste 22481 sfmdb   57u   REG             64,0x8      8192   6691
/var/opt/sfmdb/pgsql/base/17142/17205
postmaste 22481 sfmdb   58u   REG             64,0x8     16384   6566
/var/opt/sfmdb/pgsql/base/17142/16638
postmaste 22481 sfmdb   59u   REG             64,0x8    311296   6875
/var/opt/sfmdb/pgsql/base/17142/110001
postmaste 22481 sfmdb   60u   REG             64,0x8    786432   6776
/var/opt/sfmdb/pgsql/base/17142/17291
postmaste 22481 sfmdb   61u   REG             64,0x8    155648   6873
/var/opt/sfmdb/pgsql/base/17142/109997
postmaste 22481 sfmdb   62u   REG             64,0x8    303104   6775
/var/opt/sfmdb/pgsql/base/17142/17289
postmaste 22481 sfmdb   63u   REG             64,0x8    155648   6868
/var/opt/sfmdb/pgsql/base/17142/109987
postmaste 22481 sfmdb   64u   REG             64,0x8    409600   6768
/var/opt/sfmdb/pgsql/base/17142/17276
postmaste 22481 sfmdb   65u   REG             64,0x8     98304   6872
/var/opt/sfmdb/pgsql/base/17142/109995
postmaste 22481 sfmdb   66u   REG             64,0x8    204800   6774
/var/opt/sfmdb/pgsql/base/17142/17287
postmaste 22481 sfmdb   67u   REG             64,0x8     98304   6848
/var/opt/sfmdb/pgsql/base/17142/109947
postmaste 22481 sfmdb   68u   REG             64,0x8    221184   6732
/var/opt/sfmdb/pgsql/base/17142/17222
postmaste 22481 sfmdb   69u   REG             64,0x8      8192   6738
/var/opt/sfmdb/pgsql/base/17142/17224
postmaste 22481 sfmdb   70u   REG             64,0x8    524288   6863
/var/opt/sfmdb/pgsql/base/17142/109977
postmaste 22481 sfmdb   71u   REG             64,0x8   3489792   6762
/var/opt/sfmdb/pgsql/base/17142/17261
postmaste 22481 sfmdb   72u   REG             64,0x8    524288   6864
/var/opt/sfmdb/pgsql/base/17142/109979
postmaste 22481 sfmdb   73u   REG             64,0x8         0   4370
/var/opt/sfmdb/pgsql/base/17142/16414
postmaste 22486 sfmdb  cwd    DIR             64,0x8      8192   6618
/var/opt/sfmdb/pgsql/base/110181
postmaste 22486 sfmdb  txt    REG             64,0x6   5248968   6932
/opt (/dev/vg00/lvol6)
postmaste 22486 sfmdb  mem    REG             64,0x7     82448   5327
/usr/lib/hpux32/libnss_dns.so.1
postmaste 22486 sfmdb  mem    REG             64,0x7     13197   3619
/usr/lib/tztab
postmaste 22486 sfmdb  mem    REG             64,0x7    302812     60
/usr/lib/hpux32/libxti.so.1
postmaste 22486 sfmdb  mem    REG             64,0x7   2160832    443
/usr/lib/hpux32/libm.so.1
postmaste 22486 sfmdb  mem    REG             64,0x7   1546064     58
/usr/lib/hpux32/libnsl.so.1
postmaste 22486 sfmdb  mem    REG             64,0x7     28020     40
/usr/lib/hpux32/libdl.so.1
postmaste 22486 sfmdb  mem    REG             64,0x7     37916    435
/usr/lib/hpux32/libgen.so.1
postmaste 22486 sfmdb  mem    REG             64,0x7   4572100    425
/usr/lib/hpux32/libc.so.1
postmaste 22486 sfmdb  mem    REG             64,0x7    747560    422
/usr/lib/hpux32/dld.so
postmaste 22486 sfmdb  mem    REG             64,0x7    147596    481
/usr/lib/hpux32/uld.so
postmaste 22486 sfmdb  mem    REG             64,0x8       532   5555
/var/spool/pwgr/status
postmaste 22486 sfmdb    0u   CHR              3,0x2       0t0     72
/dev/null
postmaste 22486 sfmdb    1u   CHR              3,0x2    0t3395     72
/dev/null
postmaste 22486 sfmdb    2u   CHR              3,0x2    0t3395     72
/dev/null
postmaste 22486 sfmdb    3u  unix             64,0x8       0t0   7510
/var/spool/sockets/pwgr/client17806 (0x422fde40)
postmaste 22486 sfmdb    4u   REG             64,0x8     16384   6952
/var/opt/sfmdb/pgsql/base/110181/16650
postmaste 22486 sfmdb    5u  IPv4 0xe0000001423f3040       0t0    UDP
localhost:55716->localhost:55716 (Idle)
postmaste 22486 sfmdb    6u   REG             64,0x8     24576   6908
/var/opt/sfmdb/pgsql/base/110181/16412
postmaste 22486 sfmdb    7u   REG             64,0x8      8192   4122
/var/opt/sfmdb/pgsql/global/1262
postmaste 22486 sfmdb    8u  unix         0x450de580       0t0
/tmp/.s.PGSQL.10864 (->0x417b3740)
postmaste 22486 sfmdb    9u   REG             64,0x8     16384   6945
/var/opt/sfmdb/pgsql/base/110181/16635
postmaste 22486 sfmdb   10u   REG             64,0x8      8192   6912
/var/opt/sfmdb/pgsql/base/110181/16595
postmaste 22486 sfmdb   11u   REG             64,0x8     81920   6928
/var/opt/sfmdb/pgsql/base/110181/16614
postmaste 22486 sfmdb   12u   REG             64,0x8     73728   6882
/var/opt/sfmdb/pgsql/base/110181/1259
postmaste 22486 sfmdb   13u   REG             64,0x8     16384   6927
/var/opt/sfmdb/pgsql/base/110181/16613
postmaste 22486 sfmdb   14u   REG             64,0x8     73728   6924
/var/opt/sfmdb/pgsql/base/110181/16610
postmaste 22486 sfmdb   15u   REG             64,0x8    253952   6881
/var/opt/sfmdb/pgsql/base/110181/1249
postmaste 22486 sfmdb   16u   REG             64,0x8     16384   6940
/var/opt/sfmdb/pgsql/base/110181/16630
postmaste 22486 sfmdb   17u   REG             64,0x8     32768   6896
/var/opt/sfmdb/pgsql/base/110181/16390
postmaste 22486 sfmdb   18u   REG             64,0x8      8192   6899
/var/opt/sfmdb/pgsql/base/110181/16396
postmaste 22486 sfmdb   19u   REG             64,0x8     16384   6919
/var/opt/sfmdb/pgsql/base/110181/16605
postmaste 22486 sfmdb   20u   REG             64,0x8     16384   6900
/var/opt/sfmdb/pgsql/base/110181/16398
postmaste 22486 sfmdb   21u   REG             64,0x8     32768   6949
/var/opt/sfmdb/pgsql/base/110181/16639
postmaste 22486 sfmdb   22u   REG             64,0x8    106496   6897
/var/opt/sfmdb/pgsql/base/110181/16392
postmaste 22486 sfmdb   23u   REG             64,0x8     16384   6920
/var/opt/sfmdb/pgsql/base/110181/16606
postmaste 22486 sfmdb   24u   REG             64,0x8      8192   6901
/var/opt/sfmdb/pgsql/base/110181/16400
postmaste 22486 sfmdb   25u   REG             64,0x8     16384   6921
/var/opt/sfmdb/pgsql/base/110181/16607
postmaste 22486 sfmdb   26u   REG             64,0x8      8192   6885
/var/opt/sfmdb/pgsql/base/110181/16384
postmaste 22486 sfmdb   27u   REG             64,0x8    106496   6950
/var/opt/sfmdb/pgsql/base/110181/16640
postmaste 22486 sfmdb   28u   REG             64,0x8    122880   6951
/var/opt/sfmdb/pgsql/base/110181/16641
postmaste 22486 sfmdb   29u   REG             64,0x8    540672   6620
/var/opt/sfmdb/pgsql/base/110181/1255
postmaste 22486 sfmdb   30u   REG             64,0x8     16384   6954
/var/opt/sfmdb/pgsql/base/110181/16652
postmaste 22486 sfmdb   31u   REG             64,0x8     49152   6880
/var/opt/sfmdb/pgsql/base/110181/1247
postmaste 22486 sfmdb   32u   REG             64,0x8     16384   6926
/var/opt/sfmdb/pgsql/base/110181/16612
postmaste 22486 sfmdb   33u   REG             64,0x8     16384   6911
/var/opt/sfmdb/pgsql/base/110181/16418
postmaste 22486 sfmdb   34u   REG             64,0x8     16384   6939
/var/opt/sfmdb/pgsql/base/110181/16629
postmaste 22486 sfmdb   35u   REG             64,0x8     16384   6883
/var/opt/sfmdb/pgsql/base/110181/16647
postmaste 22486 sfmdb   36u   REG             64,0x8     16384   6918
/var/opt/sfmdb/pgsql/base/110181/16604
postmaste 22486 sfmdb   37u   REG             64,0x8      8192   7021
/var/opt/sfmdb/pgsql/base/110181/110240
postmaste 22486 sfmdb   38u   REG             64,0x8      8192   6999
/var/opt/sfmdb/pgsql/base/110181/110200
postmaste 22486 sfmdb   39u   REG             64,0x8     16384   6948
/var/opt/sfmdb/pgsql/base/110181/16638
postmaste 22486 sfmdb   40u   REG             64,0x8     16384   6898
/var/opt/sfmdb/pgsql/base/110181/16394
postmaste 22486 sfmdb   41u   REG             64,0x8   1130496   3682
/var/opt/sfmdb/pgsql/base/110181/16642
postmaste 22486 sfmdb   42u   REG             64,0x8     40960   6955
/var/opt/sfmdb/pgsql/base/110181/16653
postmaste 22486 sfmdb   43u   REG             64,0x8      8192   6989
/var/opt/sfmdb/pgsql/base/110181/110182
postmaste 22486 sfmdb   44u   REG             64,0x8     16384   7053
/var/opt/sfmdb/pgsql/base/110181/110377
postmaste 22486 sfmdb   45u   REG             64,0x8      8192   7005
/var/opt/sfmdb/pgsql/base/110181/110211
postmaste 22486 sfmdb   46u   REG             64,0x8     16384   7051
/var/opt/sfmdb/pgsql/base/110181/110373
postmaste 22486 sfmdb   47u   REG             64,0x8      8192   7002
/var/opt/sfmdb/pgsql/base/110181/110206
postmaste 22486 sfmdb   48u   REG             64,0x8     16384   6930
/var/opt/sfmdb/pgsql/base/110181/16616
postmaste 22486 sfmdb   49u   REG             64,0x8     16384   6886
/var/opt/sfmdb/pgsql/base/110181/16386
postmaste 22486 sfmdb   50u   REG             64,0x8      8192   7014
/var/opt/sfmdb/pgsql/base/110181/110226
postmaste 22486 sfmdb   51u   REG             64,0x8     40960   7034
/var/opt/sfmdb/pgsql/base/110181/110263
postmaste 22486 sfmdb   52u   REG             64,0x8     16384   7068
/var/opt/sfmdb/pgsql/base/110181/110407
postmaste 22486 sfmdb   53u   REG             64,0x8     16384   6946
/var/opt/sfmdb/pgsql/base/110181/16636
postmaste 22486 sfmdb   54u   REG             64,0x8     16384   7052
/var/opt/sfmdb/pgsql/base/110181/110375
postmaste 22486 sfmdb   55u   REG             64,0x8     16384   7050
/var/opt/sfmdb/pgsql/base/110181/110371
postmaste 22486 sfmdb   56u   REG             64,0x8      8192   7008
/var/opt/sfmdb/pgsql/base/110181/110216
postmaste 22486 sfmdb   57u   REG             64,0x8      8192   6990
/var/opt/sfmdb/pgsql/base/110181/110184
postmaste 22486 sfmdb   58u   REG             64,0x8    606208   7037
/var/opt/sfmdb/pgsql/base/110181/110268
postmaste 22486 sfmdb   59u   REG             64,0x8    229376   7046
/var/opt/sfmdb/pgsql/base/110181/110366
postmaste 22486 sfmdb   60u   REG             64,0x8    393216   7047
/var/opt/sfmdb/pgsql/base/110181/110367
postmaste 22486 sfmdb   61u   REG             64,0x8    229376   7069
/var/opt/sfmdb/pgsql/base/110181/110409

pgsql-general by date:

Previous
From: ptjm@interlog.com (Patrick TJ McPhee)
Date:
Subject: Re: Restoring database from old DATA folder
Next
From: "Mavinakuli, Prasanna (STSD)"
Date:
Subject: Too many open files from postgres.