On Wed, Apr 6, 2022 at 5:22 PM Imseih (AWS), Sami <simseih@amazon.com> wrote:
> > At the beginning of a parallel operation, we allocate a chunk of>
> > dynamic shared memory which persists even after some or all workers
> > have exited. It's only torn down at the end of the parallel operation.
> > That seems like the appropriate place to be storing any kind of data
> > that needs to be propagated between parallel workers. The current
> > patch uses the main shared memory segment, which seems unacceptable to
> > me.
>
> Correct, DSM does track shared data. However only participating
> processes in the parallel vacuum can attach and lookup this data.
>
> The purpose of the main shared memory is to allow a process that
> Is querying the progress views to retrieve the information.
Sure, but I think that you should likely be doing what Andres
recommended before:
# Why isn't the obvious thing to do here to provide a way to associate workers
# with their leaders in shared memory, but to use the existing progress fields
# to report progress? Then, when querying progress, the leader and workers
# progress fields can be combined to show the overall progress?
That is, I am imagining that you would want to use DSM to propagate
data from workers back to the leader and then have the leader report
the data using the existing progress-reporting facilities. Now, if we
really need a whole row from each worker that doesn't work, but why do
we need that?
--
Robert Haas
EDB: http://www.enterprisedb.com
