On 2017/04/18 18:12, Kyotaro HORIGUCHI wrote: > At Mon, 17 Apr 2017 16:19:13 -0400, Keith Fiske wrote: >> So after reading a recent thread on the steep learning curve for PG >> internals [1], I figured I'd share where I've gotten stuck with this in a >> new thread vs hijacking that one. >> >> One of the goals I had with pg_partman was to see if I could get the >> partitioning python scripts redone as C functions using a dynamic >> background worker to be able to commit in batches with a single call. My >> thinking was to have a user-function that can accept arguments for things >> like the interval value, batch size, and other arguments to the python >> script, then start/stop a dynamic bgw up for each batch so it can commit >> after each one. The dymanic bgw would essentially just have to call the >> already existing partition_data() plpgsql function, but I have to be able >> to pass the argument values that the user gave down into the dynamic bgw. >> >> I've reached a roadblock in that bgw_main_arg can only accept a single >> argument that must be passed by value for a dynamic bgw. I already worked >> around this for passing the database name to the my existing use of a bgw >> with doing partition maintenance (pass a simple integer to use as an index >> array value). But I'm not sure how to do this for passing multiple values >> in. I'm assuming this would be the place where I'd see about storing values >> in shared memory to be able to re-use later? I'm not even sure if that's >> the right approach, and if it is, where to even start to understand how to >> do that. > > On the other hand, AFAICS, DSM doesn't seem well documented. I > mangaged to find a related document in Postgres Wiki but it seems > a bit old. > > https://wiki.postgresql.org/wiki/Parallel_Internal_Sort > > This is a little complex than static shared memory, and it is > *not* guaranteed to mapped at the same address among workers. You > will see an instance in LaunchParallelWorkers() and the related > functions in parallel.c. The basic of its usage would be as the > follows. > > - Create a segment : > dsm_segment *seg = dsm_create(size); > - Send its handle via the bgw_main_arg. > worker.bgw_main_arg = dsm_segment_handle(seg); > - Attach the memory on the other side. > dsm_segment *seg = dsm_attach(main_arg); > > On both side, the address of the attached shared memory is > obtained using dsm_segment_address(seg). > > dsm_detach(seg) detaches the segment. All users of this segment > detach the segment, it will be destroyed.
Perhaps, the more modern DSA mechanism could be applicable here, too.
Some recent commits demonstrate examples of DSA usage, such as BRIN autosummarization commit (7526e10224f) and tidbitmap.c's shared iteration support commit (98e6e89040a05).
