Postgres Pro
Downloads
Home   >   mailing lists

Thread: 2GB file limit.

2GB file limit.

From
"Arsalan Zaidi"
Date:
Hi.

I'm trying to COPY a rather large file (> 3GB) into a db. Unfortunately, the
COPY command throws an error telling me that the file is too large. I can
operate on the file using 'wc', 'cat' and even through Java, but when I try
and use vi to open it, it is automatically deleted.

What's up with this? Why can some programs access the file and others like
Postgresql and vim, not? I'm using 2.4.17 with ext3 on an x86 machine.

Thanks in advance.

--Arsalan

-------------------------------------------------------------------
People often hate those things which they do not know, or cannot understand.
--Ali Ibn Abi Talib (AS)

Re: 2GB file limit.

From
Marko Kreen
Date:
On Mon, Dec 31, 2001 at 01:36:17PM +0530, Arsalan Zaidi wrote:
> I'm trying to COPY a rather large file (> 3GB) into a db. Unfortunately, the
> COPY command throws an error telling me that the file is too large. I can
> operate on the file using 'wc', 'cat' and even through Java, but when I try
> and use vi to open it, it is automatically deleted.
>
> What's up with this? Why can some programs access the file and others like
> Postgresql and vim, not? I'm using 2.4.17 with ext3 on an x86 machine.

Those programs dont use O_LARGEFILE when opening?  (man 2 open)

--
marko

Re: 2GB file limit.

From
Oliver Elphick
Date:
On Mon, 2001-12-31 at 08:06, Arsalan Zaidi wrote:
> Hi.
>
> I'm trying to COPY a rather large file (> 3GB) into a db. Unfortunately, the
> COPY command throws an error telling me that the file is too large. I can
> operate on the file using 'wc', 'cat' and even through Java, but when I try
> and use vi to open it, it is automatically deleted.
>
> What's up with this? Why can some programs access the file and others like
> Postgresql and vim, not? I'm using 2.4.17 with ext3 on an x86 machine.

I don't know why COPY and vi barf on it, but if you can use things like
wc on it, try piping it into psql:

  psql -d your_database -c "COPY your_table FROM STDIN" < very_big_file


--
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight                              http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C

     "Ye have heard that it hath been said, Thou shalt love
      thy neighbour, and hate thine enemy. But I say unto
      you, Love your enemies, bless them that curse you, do
      good to them that hate you, and pray for them which
      despitefully use you, and persecute you;"
                                     Matthew 5:43,44
Attachment