Thread: Generating code coverage reports

Generating code coverage reports

From
Michelle Caisse
Date:
<p>I have a patch that I will be submitting to add to the build system the capability of reporting on test code
coveragemetrics for the test suite. Actually it can show coverage for any application run against PostgreSQL. Download
<ahref="http://wiki.postgresql.org/wiki/Image:Coverage.tar.gz" title="Image:Coverage.tar.gz">Image:Coverage.tar.gz</a>
tosee an example report. Gunzip and un-tar the file and click on coverage/index.html. I had to delete most of the files
todecrease the file size for upload, so only the links for access work. <p>gcov reports line, branch, and function
coverage,but lcov only reports on line coverage. I've added a link to the html to the gcov output that shows summary
statisticsfor each file for line, branch, and function calls. <p>The report gives a very clear and browseable view of
whatparts of the system might benefit from more extensive testing. It's obviously useful for planning future testing,
butalso can be used in conjunction with debugging to see what lines and functions are being exercised or missed by
existingtests of the functionality under investigation. It could even be helpful to give a static view of lines hit by
abug test case in lieue of using a debugger. Also, when you're writing a unit test for new functionality, it would be
goodto check what you're actually hitting with the test. <p>It uses gcov together with gcc to generate the statistics,
andthe lcov suite to create the html report. Both of these would obviously have to be installed to get a coverage
report,but this would be an optional feature of the build. It only works with gcc. <p>To generate coverage statistics,
yourun configure with --enable-coverage and after building and running tests, you do make coverage. The process
generatesdata files in the same directories as source & object files and produces a coverage directory at the top
levelwith the html files. I've also set it up so a tar file with the html is generated. <p>More information on gcov at
<aclass="external free" href="http://gcc.gnu.org/onlinedocs/gcc/Gcov.html" rel="nofollow"
title="http://gcc.gnu.org/onlinedocs/gcc/Gcov.html">http://gcc.gnu.org/onlinedocs/gcc/Gcov.html</a>,lcov at <a
class="externalfree" href="http://ltp.sourceforge.net/documentation/how-to/ltp.php" rel="nofollow"
title="http://ltp.sourceforge.net/documentation/how-to/ltp.php">http://ltp.sourceforge.net/documentation/how-to/ltp.php</a>
(coverage/lcovtabs).<br /><p>-- Michelle<br /><pre class="moz-signature" cols="72">-- 
 
Michelle Caisse               Sun Microsystems
California, U.S.     <a class="moz-txt-link-freetext" href="http://sun.com/postgresql">http://sun.com/postgresql</a>

</pre>

Re: Generating code coverage reports

From
Tom Lane
Date:
Michelle Caisse <Michelle.Caisse@Sun.COM> writes:
> I have a patch that I will be submitting to add to the build system the 
> capability of reporting on test code coverage metrics for the test 
> suite.

Cool.

> To generate coverage statistics, you run configure with 
> --enable-coverage and after building and running tests, you do make 
> coverage. The process generates data files in the same directories as 
> source & object files and produces a coverage directory at the top level 
> with the html files.

How does that work with a VPATH build?  Are you trying to say that you
still have to have the original build tree around in order to collect
coverage data?
        regards, tom lane


Re: Generating code coverage reports

From
Michelle Caisse
Date:
It should be possible to make it work  for a VPATH build with appropriate arguments to gcov and lcov, but currently it
expectsthe object files and generated data files to be in the build directory.<br /><br /> You need access to the build
treeto generate coverage statistics and to generate the report with "make coverage" after running the tests or
application.<br /><br /> -- Michelle<br /><br /> Tom Lane wrote: <blockquote cite="mid13595.1215721192@sss.pgh.pa.us"
type="cite"><prewrap="">Michelle Caisse <a class="moz-txt-link-rfc2396E"
href="mailto:Michelle.Caisse@Sun.COM"><Michelle.Caisse@Sun.COM></a>writes: </pre><blockquote type="cite"><pre
wrap="">Ihave a patch that I will be submitting to add to the build system the 
 
capability of reporting on test code coverage metrics for the test 
suite.   </pre></blockquote><pre wrap="">
Cool.
 </pre><blockquote type="cite"><pre wrap="">To generate coverage statistics, you run configure with 
--enable-coverage and after building and running tests, you do make 
coverage. The process generates data files in the same directories as 
source & object files and produces a coverage directory at the top level 
with the html files.   </pre></blockquote><pre wrap="">
How does that work with a VPATH build?  Are you trying to say that you
still have to have the original build tree around in order to collect
coverage data?
        regards, tom lane
 </pre></blockquote><br /><pre class="moz-signature" cols="72">-- 
Michelle Caisse               Sun Microsystems
California, U.S.     <a class="moz-txt-link-freetext" href="http://sun.com/postgresql">http://sun.com/postgresql</a>

</pre>