Thread: Re: fork/exec
Hi Bruce, I have a couple of questions regarding your intended fork/exec changes related to BackendFork: * I see you are passing the values of things like UsedShmemSegID, UsedShmemSegAddr etc on the command line. Was your intention to pass other shared mem addresses like FreeSpaceMap, LWLockArray in this way too, or did you intend to register these addresses within another section of shared memory, or some other idea (FWIW, I've got a cygwin version fork/exec'ing the backends using the former method). * With regards to BackendFork itself, was your plan to reduce this to simply formatting the argument list, and calling fork/exec, with things like random number seeding and the like moved into PostgresMain? (ie. the BackendFork call would actually be performed by the Postmaster, with BackendFork rewritten such that no "global" context is required between the fork and exec calls; which'll be what we need to make Win32/CreateProcess changes possible). If not, could you give me an idea of what you had intended? Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see <a href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em ailpolicy.html</a>
Claudio Natoli <claudio.natoli@memetrics.com> writes:
> * I see you are passing the values of things like UsedShmemSegID,
> UsedShmemSegAddr etc on the command line. Was your intention to pass other
> shared mem addresses like FreeSpaceMap, LWLockArray in this way too, or did
> you intend to register these addresses within another section of shared
> memory, or some other idea (FWIW, I've got a cygwin version fork/exec'ing
> the backends using the former method).
Cluttering the backend command line with intra-shmem addresses is messy
and unnecessary. The shmem index map (see ShmemInitStruct) was designed
to allow backends to discover this stuff for themselves, and we should
resurrect that function.
regards, tom lane
Tom Lane wrote: > Claudio Natoli <claudio.natoli@memetrics.com> writes: > > * I see you are passing the values of things like UsedShmemSegID, > > UsedShmemSegAddr etc on the command line. Was your intention to pass other > > shared mem addresses like FreeSpaceMap, LWLockArray in this way too, or did > > you intend to register these addresses within another section of shared > > memory, or some other idea (FWIW, I've got a cygwin version fork/exec'ing > > the backends using the former method). > > Cluttering the backend command line with intra-shmem addresses is messy > and unnecessary. The shmem index map (see ShmemInitStruct) was designed > to allow backends to discover this stuff for themselves, and we should > resurrect that function. Agreed. We have to pass the shared memory address, but the rest should be registered in shared memory somewhere and we can initialize those values. The old code used to point _using_ those memory pointers, but I don't think that is necessary --- in fork/exec mode, we can just use share memory to initialize the pointers properly. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Claudio Natoli wrote: > * With regards to BackendFork itself, was your plan to reduce this to simply > formatting the argument list, and calling fork/exec, with things like random > number seeding and the like moved into PostgresMain? (ie. the BackendFork > call would actually be performed by the Postmaster, with BackendFork > rewritten such that no "global" context is required between the fork and > exec calls; which'll be what we need to make Win32/CreateProcess changes > possible). If not, could you give me an idea of what you had intended? You will see code in postmaster.c that calls write_nondefault_variables(). That basically dumps out GUC variables into a binary file to be read in by fork/exec backends. My idea is that we will need another similar file for postmaster constants. The random seed value is unique per-backend, and we want to keep it secure so we can't pass it on the command line. I am not sure how to deal with that. I hesitate to add a per-backend file to pass such things. Does anyone have ideas on how to pass a data value to an exec()'ed child? Maybe we have to use a pipe between parent/child and pass the values that way. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Agreed. We have to pass the shared memory address, but the rest should
> be registered in shared memory somewhere and we can initialize those
> values. The old code used to point _using_ those memory pointers, but I
> don't think that is necessary --- in fork/exec mode, we can just use
> share memory to initialize the pointers properly.
Yeah. It might be useful to extend the shmem segment header (which at
the moment is mostly just for identification) to hold one or two
critical addresses, such as the address of the LWLock array. But the
index map used to work for this back when we had fork/exec in the Unix
implementation, so it should be possible to make it work again without
undue amounts of pain.
regards, tom lane
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> The random seed value is unique per-backend, and we want to keep it
> secure so we can't pass it on the command line. I am not sure how to
> deal with that.
Uh, you mean the cancel key? There's no need for the random() seed per
se to be shared between postmaster and backend, AFAICS.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > The random seed value is unique per-backend, and we want to keep it > > secure so we can't pass it on the command line. I am not sure how to > > deal with that. > > Uh, you mean the cancel key? There's no need for the random() seed per > se to be shared between postmaster and backend, AFAICS. Oh, good. I couldn't remember if it was the postmaster or child that validates that key. I now remember only the postmaster needs the secret because it sends the signal. Not sure what random seed Claudio was asking about. Probably GUC's "seed" parameter --- Claudio, that is already covered in that GUC binary file I create that I mentioned in an earlier email. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Oh, good. I couldn't remember if it was the postmaster or child that
> validates that key. I now remember only the postmaster needs the secret
> because it sends the signal.
However, the child process needs the secret for long enough to send it
off to the client at the completion of the authentication handshake.
So there is a problem to resolve here.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Oh, good. I couldn't remember if it was the postmaster or child that > > validates that key. I now remember only the postmaster needs the secret > > because it sends the signal. > > However, the child process needs the secret for long enough to send it > off to the client at the completion of the authentication handshake. > So there is a problem to resolve here. Oh, yea, so they do both need it. Good point. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073