Thread: Low level socket locking...
Hi I am writing a C program to work as an interface between postgres and a web page. This page will get a LOT of activity. The system is to be set up like this: Web page, CGI generated | Connection to handler process via unix socket | Handler process, that has a postgres connection open. This is set up in this way because I need to do many similar jobs with the database, but obviously, every time the web page is accessed, a new copy of the cgi script is created, and if I were to create a new connection to the database every time this would a) mean spawning LOTS of copies of the postgres backend, very slow, and b) I wouldnt get the advantages of any caching that is done. Thus I send a simple message from the CGI to the handler process, which then interracts with the database, and returns the result. Now, my problem is this. The page being accessed is likely, in time, to get enough hits that as the jobs are handled one at a time by the handler process (by the nature of the postgres C librray functions), it is very feasable that a backlog could form. I was planning on simply fork()ing the handler process. The database would be open on each child, and I could happily process them all easilly. However it occurred to me that I do not know how the socket code between the postgres libraries and the postgres backend is handled? Is there a socket locking process in place, to prevent two children passing data down the same socket at the same time, thus causing garbled data to be received at the other end? If I fork() and send data from one of say, three child processes, am I guarenteed to get the data back at the same child? (this all assumes that I open the database connection and then fork() after) Thanx in advance for any info
=>From: M Simms <grim@argh.demon.co.uk> =>... =>I am writing a C program to work as an interface between postgres and a =>web page. This page will get a LOT of activity. =>... =>I was planning on simply fork()ing the handler process. The database would be =>open on each child, and I could happily process them all easilly. However =>it occurred to me that I do not know how the socket code between the =>postgres libraries and the postgres backend is handled? Is there a socket =>locking process in place, to prevent two children passing data down the same =>socket at the same time, thus causing garbled data to be received at the other =>end? If I fork() and send data from one of say, three child processes, am =>I guarenteed to get the data back at the same child? =>(this all assumes that I open the database connection and then fork() after) I've written a similar program for Oracle and Sybase, and (while I can't speak directly to the question of whether you can do what you want in PostgreSQL), I'd strongly suggest opening the DB connection *after* you fork(). You won't have to worry about multiple children confusing the back end, and it'll work for other databases too, if you need that capability later. The sequence you want to do is: * Prepare to become a daemon (lose controlling TTY, chdir("/"), close all open descriptors, etc) * Open the socket you're going to receive requests on. * Fork() as many children as you need > In each child, connect to the database, then do accept/read/write/close on the socket to respond to the client's requests > In the parent, watch for children dying, and spawn new ones to replace them. You can make this even nicer by having the parent & children collude to do load balancing -- if all the children have clients, the parent should start some more. If many children are idle for a while, they should die off. You can even have the parent serve requests on a "control channel" to change the number of children or to retrieve statistics about the daemon's activity. d.
M Simms wrote: > > This is set up in this way because I need to do many similar jobs with the > database, but obviously, every time the web page is accessed, a new copy of > the cgi script is created, and if I were to create a new connection to the > database every time this would a) mean spawning LOTS of copies of the > postgres backend, very slow, and b) I wouldnt get the advantages of any > caching that is done. > Thus I send a simple message from the CGI to the handler process, which > then interracts with the database, and returns the result. AFAIK, you have three possibilities: - Write a daemon who waits for connections from the CGI and does all the database-related stuff. This is slightly hard to implement but the most flexible solution, probably not the best thought... - Use FastCGI (http://fastcgi.idle.com), it's very similar to the first solution, since the fastcgi module will spawn one or more persistent processes and will dispatch the requests to them thru sockets (eventually, you may move the processes on a different machine). You would open the database connection only once when the process is spawned. Advantages: Very fast as the processes may be written in C. - Use PHP3 with pg_pConnect (persistent Connect), the PHP module will mantain a cache of open connections and efficently reuse each of them. Hope this helps... Bye! -- Daniele ------------------------------------------------------------------------------- "Oh, I've seen copies [of Linux Journal] around the terminal room at The Labs." (By Dennis Ritchie) ------------------------------------------------------------------------------- Se telecom italia aggiungesse uno scatto al giorno ad ogni abbonato, dal primo Gennaio avrebbe rubato 682.040.000.000 Lire. ------------------------------------------------------------------------------- Daniele Orlandi - Utility Line Italia - http://www.orlandi.com Via Mezzera 29/A - 20030 - Seveso (MI) - Italy -------------------------------------------------------------------------------