[ERROR] Copy from CSV fails due to memory error. - Mailing list pgsql-hackers

From Kostas Chasialis
Subject [ERROR] Copy from CSV fails due to memory error.
Date
Msg-id 21D3CCA3-328F-46AC-93CB-CDEBF81BD4F8@gmail.com
Whole thread Raw
Responses Re: [ERROR] Copy from CSV fails due to memory error.
List pgsql-hackers
Hey.

I am facing an issue when I try to run the following command

COPY <table_name> FROM <file> WITH DELIMITER E',’;

This file, is rather large, it's around 178GBs.

When I try to run this COPY command I get the following error:

ERROR:  out of memory
DETAIL:  Failed on request of size 2048 in memory context "AfterTriggerEvents".
CONTEXT:  COPY ssbm300_lineorder, line 50796791

Clearly a memory allocation function is failing but I have no clue how to fix it.

I have tried experimenting with shared_buffers value in postgresql.conf file but after searching a bit I quickly
realizedthat I do not know what I am doing there so I left it with default value. Same with work_mem value. 

Did you face this issue before? Can you help me resolve it?

Thanks in advance!




pgsql-hackers by date:

Previous
From: Juan José Santamaría Flecha
Date:
Subject: Re: WIN32 pg_import_system_collations
Next
From: Amit Kapila
Date:
Subject: Re: row filtering for logical replication