Re: Row pattern recognition - Mailing list pgsql-hackers
| From | Henson Choi |
|---|---|
| Subject | Re: Row pattern recognition |
| Date | |
| Msg-id | CAAAe_zDR8XqOfYL1ay5o6Y2OogJNed3oR+aG=Hm1Ja=t=v6=bw@mail.gmail.com Whole thread |
| In response to | Re: Row pattern recognition (Henson Choi <assam258@gmail.com>) |
| Responses |
Re: Row pattern recognition
|
| List | pgsql-hackers |
Hi hackers,
As a final memory-safety check before wrapping up the RPR patch, I ran the
branch under Valgrind. The short version: no memory errors were found.
Setup, on Linux/AMD64 (Valgrind is not usable on macOS):
configure: --enable-debug --enable-cassert -DUSE_VALGRIND
also defined: -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES
-DRAW_EXPRESSION_COVERAGE_TEST
valgrind: --suppressions=src/tools/valgrind.supp --trace-children=yes
--track-origins=yes --leak-check=no --error-exitcode=128
The round-trip defines keep the parse/plan-tree copy and write/read paths on by
default, so _outRPRPattern, _readRPRPattern and _copyRPRPattern are exercised on
every plan tree, not just at parse time.
I ran installcheck against this instance in three phases.
Phase A, round-trips on: rpr, rpr_nfa, rpr_explain and rpr_integration. These
are normal-size patterns, so the node copy/write-read paths are verified along
the way. All passed.
Phase B, round-trips off: rpr_base, which includes the large 32767-element
pattern. Serializing a tree that big under Valgrind is impractically slow, and
the serialization path is already covered by the normal-size patterns in
Phase A, so I disabled debug_copy_parse_plan_trees and
debug_write_read_parse_plan_trees here to exercise the NFA build and
scanRPRPattern's large arrays without the serialization bottleneck. Passed.
Phase C: window, preceded by test_setup, create_index and sanity_check so the
table, indexes and stats are in place. This one reported a regression diff: a
few EXPLAIN plans came out with Seq Scan / Hash Join where the expected output
has Index Only Scan / Nested Loop or Merge Join. That is a planner artifact of
running window outside the full parallel schedule (a different stats/cost
environment), not anything RPR changed; the same queries match expectations
under a normal check-world run. I included window mainly to exercise the
WindowAgg/RPR interaction under instrumentation, and for that the diff does not
matter, because the memory verdict comes from the Valgrind logs rather than the
regression output.
Results: no VALGRINDERROR markers in any per-pid log, and every log is empty
(with --quiet, a clean run leaves 0-byte logs), window included. The postmaster
exited 0, so --error-exitcode=128 never fired, and the tests ran tens of times
slower than usual, which confirms the code really did run under instrumentation.
So across parse, NFA build, pattern scan, EXPLAIN and node serialization, no
invalid read/write, uninitialised-value use, out-of-bounds access or invalid
free was observed.
The usual caveat applies: this is dynamic analysis, so it only attests to the
paths the tests actually exercised. As I add the coverage tests discussed in
the other thread, I will re-run the same setup over the expanded suite.
Best regards,
Henson
As a final memory-safety check before wrapping up the RPR patch, I ran the
branch under Valgrind. The short version: no memory errors were found.
Setup, on Linux/AMD64 (Valgrind is not usable on macOS):
configure: --enable-debug --enable-cassert -DUSE_VALGRIND
also defined: -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES
-DRAW_EXPRESSION_COVERAGE_TEST
valgrind: --suppressions=src/tools/valgrind.supp --trace-children=yes
--track-origins=yes --leak-check=no --error-exitcode=128
The round-trip defines keep the parse/plan-tree copy and write/read paths on by
default, so _outRPRPattern, _readRPRPattern and _copyRPRPattern are exercised on
every plan tree, not just at parse time.
I ran installcheck against this instance in three phases.
Phase A, round-trips on: rpr, rpr_nfa, rpr_explain and rpr_integration. These
are normal-size patterns, so the node copy/write-read paths are verified along
the way. All passed.
Phase B, round-trips off: rpr_base, which includes the large 32767-element
pattern. Serializing a tree that big under Valgrind is impractically slow, and
the serialization path is already covered by the normal-size patterns in
Phase A, so I disabled debug_copy_parse_plan_trees and
debug_write_read_parse_plan_trees here to exercise the NFA build and
scanRPRPattern's large arrays without the serialization bottleneck. Passed.
Phase C: window, preceded by test_setup, create_index and sanity_check so the
table, indexes and stats are in place. This one reported a regression diff: a
few EXPLAIN plans came out with Seq Scan / Hash Join where the expected output
has Index Only Scan / Nested Loop or Merge Join. That is a planner artifact of
running window outside the full parallel schedule (a different stats/cost
environment), not anything RPR changed; the same queries match expectations
under a normal check-world run. I included window mainly to exercise the
WindowAgg/RPR interaction under instrumentation, and for that the diff does not
matter, because the memory verdict comes from the Valgrind logs rather than the
regression output.
Results: no VALGRINDERROR markers in any per-pid log, and every log is empty
(with --quiet, a clean run leaves 0-byte logs), window included. The postmaster
exited 0, so --error-exitcode=128 never fired, and the tests ran tens of times
slower than usual, which confirms the code really did run under instrumentation.
So across parse, NFA build, pattern scan, EXPLAIN and node serialization, no
invalid read/write, uninitialised-value use, out-of-bounds access or invalid
free was observed.
The usual caveat applies: this is dynamic analysis, so it only attests to the
paths the tests actually exercised. As I add the coverage tests discussed in
the other thread, I will re-run the same setup over the expanded suite.
Best regards,
Henson
pgsql-hackers by date: