Thread: Broken pipe
Hi all, I sent the following email to the php mailing list also but maybe this is a more appropriate mailing list. I wrote a php script which is running very long queries (hours) on a database. I seem to have a problem to run the code when there are single queries which take long times (like 5 hours for an update query), from the log of the database I received the following code: 2005-09-30 17:12:13 IDT postgres : LOG: 00000: duration: 18730038.678 ms statement: UPDATE product_temp SET nleft=(SELECT 2005-09-30 17:12:13 IDT postgres : LOCATION: exec_simple_query, postgres.c:1035 2005-09-30 17:12:13 IDT postgres : LOG: 08006: could not send data to client: Broken pipe 2005-09-30 17:12:13 IDT postgres : LOCATION: internal_flush, pqcomm.c:1050 2005-09-30 17:12:13 IDT postgres : LOG: 08P01: unexpected EOF on client connection 2005-09-30 17:12:13 IDT postgres : LOCATION: SocketBackend, postgres.c:287 2005-09-30 17:12:13 IDT postgres : LOG: 00000: disconnection: session time: 6:04:58.52 2005-09-30 17:12:13 IDT postgres : LOCATION: log_disconnections, postgres.c:3403 Now after the 5 hours update it need to echo into a log file a line which say that it ended this command (just for me to know the times), my assumption is that PHP read the code into memory at start and opened the connection to the file, after a time which he waited to any given "life sign" he gave up and closed the connection to the file, and when the code came back to the file it encountered no connection to the file (broken pipe). Am I correct at my assumption? if so how can I set the PHP to wait how much I tell him? Ofcourse if im wrong I would like to know the reason also :) Thanks in advance, Ben-Nes Yonatan
On Sun, Oct 02, 2005 at 12:07:18PM +0200, Ben-Nes Yonatan wrote: > I wrote a php script which is running very long queries (hours) on a > database. > I seem to have a problem to run the code when there are single queries > which take long times (like 5 hours for an update query), from the log > of the database I received the following code: My experience with long running backend queries is that IE gives up after a while but Netscape hangs on a lot longer. I don't know if this is still the case. It was a while while I worked out why it worked for me but not my boss... You need to work who is giving up, the client which causes PHP to give up, or if PHP itself is giving up. PostgreSQL doesn't care overly much, as long as you complete the transaction. My suggestion would be to spawn the update into a seperate process and send a "Processing, please wait" to the client. When the stuff is done, set a flag and indicate to the client that processing is complete. Hope this helps, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them.
Attachment
Martijn van Oosterhout wrote: >On Sun, Oct 02, 2005 at 12:07:18PM +0200, Ben-Nes Yonatan wrote: > > >>I wrote a php script which is running very long queries (hours) on a >>database. >>I seem to have a problem to run the code when there are single queries >>which take long times (like 5 hours for an update query), from the log >>of the database I received the following code: >> >> > >My experience with long running backend queries is that IE gives up >after a while but Netscape hangs on a lot longer. I don't know if this >is still the case. It was a while while I worked out why it worked for >me but not my boss... > >You need to work who is giving up, the client which causes PHP to give >up, or if PHP itself is giving up. PostgreSQL doesn't care overly much, >as long as you complete the transaction. > >My suggestion would be to spawn the update into a seperate process and >send a "Processing, please wait" to the client. When the stuff is done, >set a flag and indicate to the client that processing is complete. > >Hope this helps, > > Well this is not the case (sadly.. :)) cause the process is running at the server without any browser interface.
Am Sonntag, den 02.10.2005, 16:39 +0200 schrieb Ben-Nes Yonatan: > Martijn van Oosterhout wrote: ... > Well this is not the case (sadly.. :)) cause the process is running at > the server without any browser interface. Use the web as trigger only and a persistent long running process to read the job table and do the action - including setting a flag when its done.
----- Original Message ----- From: "Tino Wildenhain" <tino@wildenhain.de> To: "Ben-Nes Yonatan" <da@canaan.co.il> Cc: "Martijn van Oosterhout" <kleptog@svana.org>; <pgsql-general@postgresql.org> Sent: Sunday, October 02, 2005 4:26 PM Subject: Re: [GENERAL] Broken pipe > Am Sonntag, den 02.10.2005, 16:39 +0200 schrieb Ben-Nes Yonatan: >> Martijn van Oosterhout wrote: > ... >> Well this is not the case (sadly.. :)) cause the process is running at >> the server without any browser interface. > > Use the web as trigger only and a persistent long running process > to read the job table and do the action - including setting a flag > when its done. > Why should I use the web as a trigger (I assume that you mean with a browser) when im running it from the command line? I dont understand what do you mean with "a persistent long running process to read the job table", Its one process which need to be run step by step, with small files (total of less then 1.7 million rows) its working gr8 but when I try to load it with more data it just stop with the error: 2005-09-30 17:12:13 IDT postgres : LOG: 00000: duration: 18730038.678 ms statement: UPDATE product_temp SET nleft=(SELECT 2005-09-30 17:12:13 IDT postgres : LOCATION: exec_simple_query, postgres.c:1035 2005-09-30 17:12:13 IDT postgres : LOG: 08006: could not send data to client: Broken pipe 2005-09-30 17:12:13 IDT postgres : LOCATION: internal_flush, pqcomm.c:1050 2005-09-30 17:12:13 IDT postgres : LOG: 08P01: unexpected EOF on client connection 2005-09-30 17:12:13 IDT postgres : LOCATION: SocketBackend, postgres.c:287 2005-09-30 17:12:13 IDT postgres : LOG: 00000: disconnection: session time: 6:04:58.52 user=postgres database=poweraise.com 2005-09-30 17:12:13 IDT postgres : LOCATION: log_disconnections, postgres.c:3403 I also tried persistent connection (thought that maybe you meant that even if it unlikely) and it didnt work too. About the flag I guess that its important only if the process need a browser as a trigger. Shana Tova! (hebrew happy new year :)) Ben-Nes Yonatan