Thread: differential code coverage

differential code coverage

From
Andres Freund
Date:
Hi,

For quite a while I'd been wishing to see *differential* code coverage, to see
what changed in code coverage between two major releases. Unfortunately lcov
didn't provide that. A few months ago a PR for it has been merged into lcov
([1]). There hasn't yet been a release though. And the feature definitely has
some rough edges, still.

I'm planning to generate the 15->16 differential code coverage, once the
feature freeze has been reached.

Another nice thing provided by the in-development lcov is hierarchical
output. I find that to be much easier to navigate than the current flat
output, as e.g. used on coverage.postgresql.org.


I've attached a screenshot showing the coverage differences of the backend
between 15 and HEAD, as of a few days ago. "UNC" is uncovered new code, "LBC"
is lost baseline coverage.


I think for now it'd likely be a small script that'd generate the code
coverage across versions. Do we want to have that in the source tree?


Is there any interest in a) using the hierarchical output b) differential
output on coverage.pg.o?

Greetings,

Andres Freund

[1] https://github.com/linux-test-project/lcov/pull/169

Attachment

Re: differential code coverage

From
Daniel Gustafsson
Date:
> On 4 Apr 2023, at 18:03, Andres Freund <andres@anarazel.de> wrote:

> I'm planning to generate the 15->16 differential code coverage, once the
> feature freeze has been reached.

Cool!

> I think for now it'd likely be a small script that'd generate the code
> coverage across versions. Do we want to have that in the source tree?

If it's published on pg.o like discussed below then I think it makes sense to
include it.

> Is there any interest in a) using the hierarchical output b) differential
> output on coverage.pg.o?

I would like to see that.  If there are concerns about replacing the current
coverage report with one from an unreleased (and potentially buggy) lcov, we
could perhaps use diff.coverage.pg.o until it's released?

--
Daniel Gustafsson




Re: differential code coverage

From
Andres Freund
Date:
Hi,

On 2023-04-04 09:03:45 -0700, Andres Freund wrote:
> For quite a while I'd been wishing to see *differential* code coverage, to see
> what changed in code coverage between two major releases. Unfortunately lcov
> didn't provide that. A few months ago a PR for it has been merged into lcov
> ([1]). There hasn't yet been a release though. And the feature definitely has
> some rough edges, still.
> 
> I'm planning to generate the 15->16 differential code coverage, once the
> feature freeze has been reached.
> 
> Another nice thing provided by the in-development lcov is hierarchical
> output. I find that to be much easier to navigate than the current flat
> output, as e.g. used on coverage.postgresql.org.

I've generated the output for 15 vs HEAD, now that we're past feature freeze.

Simpler version:
https://anarazel.de/postgres/cov/15-vs-HEAD-01/

Version that tries to date the source lines using information from git:
https://anarazel.de/postgres/cov/15-vs-HEAD-02/

The various abbreviations are explained (somewhat) if you hover over them.


There's a few interesting bit of new code not covered by teests, that should
easily be coverable:

https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/bin/pg_dump/pg_dumpall.c.gcov.html#1014

https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/backend/utils/adt/array_userfuncs.c.gcov.html#675
https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/backend/utils/adt/array_userfuncs.c.gcov.html#921

https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/backend/utils/adt/pg_locale.c.gcov.html#1964
https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/backend/utils/adt/pg_locale.c.gcov.html#2219

https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/backend/parser/parse_expr.c.gcov.html#3108
https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/backend/nodes/nodeFuncs.c.gcov.html#1480
https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/backend/nodes/nodeFuncs.c.gcov.html#1652

https://anarazel.de/postgres/cov/15-vs-HEAD-01/src/backend/optimizer/util/plancat.c.gcov.html#348

https://anarazel.de/postgres/cov/15-vs-HEAD-01/contrib/postgres_fdw/connection.c.gcov.html#1316
https://anarazel.de/postgres/cov/15-vs-HEAD-01/contrib/postgres_fdw/deparse.c.gcov.html#399

Greetings,

Andres Freund