Duplicate Workers entries in some EXPLAIN plans - Mailing list pgsql-hackers

From Maciek Sakrejda
Subject Duplicate Workers entries in some EXPLAIN plans
Date
Msg-id CAOtHd0AvAA8CLB9Xz0wnxu1U=zJCKrr1r4QwwXi_kcQsHDVU=Q@mail.gmail.com
Whole thread Raw
Responses Re: Duplicate Workers entries in some EXPLAIN plans
List pgsql-hackers
Hello,

I originally reported this in pgsql-bugs [0], but there wasn't much feedback there, so moving the discussion here. When using JSON, YAML, or XML-format EXPLAIN on a plan that uses a parallelized sort, the Sort nodes list two different entries for "Workers", one for the sort-related info, and one for general worker info. This is what this looks like in JSON (some details elided):

{
  "Node Type": "Sort",
  ...
  "Workers": [
    {
      "Worker Number": 0,
      "Sort Method": "external merge",
      "Sort Space Used": 20128,
      "Sort Space Type": "Disk"
    },
    {
      "Worker Number": 1,
      "Sort Method": "external merge",
      "Sort Space Used": 20128,
      "Sort Space Type": "Disk"
    }
  ],
  ...
  "Workers": [
    {
      "Worker Number": 0,
      "Actual Startup Time": 309.726,
      "Actual Total Time": 310.179,
      "Actual Rows": 4128,
      "Actual Loops": 1,
      "Shared Hit Blocks": 2872,
      "Shared Read Blocks": 7584,
      "Shared Dirtied Blocks": 0,
      "Shared Written Blocks": 0,
      "Local Hit Blocks": 0,
      "Local Read Blocks": 0,
      "Local Dirtied Blocks": 0,
      "Local Written Blocks": 0,
      "Temp Read Blocks": 490,
      "Temp Written Blocks": 2529
    },
    {
      "Worker Number": 1,
      "Actual Startup Time": 306.523,
      "Actual Total Time": 307.001,
      "Actual Rows": 4128,
      "Actual Loops": 1,
      "Shared Hit Blocks": 3356,
      "Shared Read Blocks": 7100,
      "Shared Dirtied Blocks": 0,
      "Shared Written Blocks": 0,
      "Local Hit Blocks": 0,
      "Local Read Blocks": 0,
      "Local Dirtied Blocks": 0,
      "Local Written Blocks": 0,
      "Temp Read Blocks": 490,
      "Temp Written Blocks": 2529
    }
  ],
  "Plans:" ...
}

This is technically valid JSON, but it's extremely difficult to work with, since default JSON parsing in Ruby, node, Python, Go, and even Postgres' own jsonb only keep the latest key--the sort information is discarded (other languages probably don't fare much better; this is what I had on hand). As Tom Lane pointed out in my pgsql-bugs thread, this has been reported before [1] and in that earlier thread, Andrew Dunstan suggested that perhaps the simplest solution is to just rename the sort-related Workers node. Tom expressed some concerns about a breaking change here, though I think the current behavior means vanishingly few users are parsing this data correctly. Thoughts?

Thanks,
Maciek

[0]: https://www.postgresql.org/message-id/CADXhmgSr807j2Pc9aUjW2JOzOBe3FeYnQBe_f9U%2B-Mm4b1HRUw%40mail.gmail.com

pgsql-hackers by date:

Previous
From: Tomas Vondra
Date:
Subject: Re: 回复:Bug about drop index concurrently
Next
From: Alexey Kondratov
Date:
Subject: Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions