Oct 23, 2009

Instrumenting NetBeans with Cobertura for Fun and Profit

Using Cobertura within a NetBeans project is not that difficult, but there are a few subtleties that I often miss. This setup will use Cobertura if the library is part of the test classpath; otherwise, it will silently skip the instrumentation step. For shared development environments, developers that do not wish to instrument will have to create an empty Cobertura library to avoid the dreaded “Missing Reference” problem.

The majority of the heavy lifting is done within build.xml. After NetBeans has set up Ant with its own macros, check to see if the Cobertura library is installed, then load the Cobertura Ant tasks:

<!-- If the Cobertura library has been added to the classpath and it is the right library, set the flag -->
<target name="-check-cobertura-availability" if="libs.Cobertura.classpath">
<available property="cobertura.available"
    classname="net.sourceforge.cobertura.coveragedata.HasBeenInstrumented"
    classpath="${javac.test.classpath}"/>
</target>

<!-- Load in the Cobertura tasks and set runtime properties to reasonable values -->
<target name="-load-cobertura-tasks" if="cobertura.available">
<taskdef classpath="${libs.Cobertura.classpath}" resource="tasks.properties"/>
<property name="test-sys-prop.net.sourceforge.cobertura.datafile" value="${build.dir}/test/cobertura.ser"/>
<property name="build.test.coverage.dir" value="${build.dir}/test/coverage"/>
</target>

<target name="-post-init" depends="-check-cobertura-availability,-load-cobertura-tasks"/>
      

The following step is annoying, but it helps insure that stale instrumented classes do not shadow fresh compiles:

<!-- Flush all previously instrumented code -->
<target name="-pre-compile">
<delete dir="${build.test.classes.dir}"/>
</target>
<target name="-pre-compile-single">
<delete dir="${build.test.classes.dir}"/>
</target>
      

This is the most important step. Since we are interested in instrumenting the application classes but do not want to ship instrumented code, we need to dump the instrumented class files somewhere else instead of using Cobertura’s default behavior to modify in situ. Since NetBeans complains if classpath directories are missing at start time (“Missing Reference”), it is best to dump them in a known location. The test build directory is a perfect candidate:

<!-- Instrument the application code -->
<target name="-pre-compile-test" if="cobertura.available">
<delete file="${test-sys-prop.net.sourceforge.cobertura.datafile}"/>
<cobertura-instrument todir="${build.test.classes.dir}"
    datafile="${test-sys-prop.net.sourceforge.cobertura.datafile}">
    <fileset dir="${build.classes.dir}">
	<include name="**/*.class"/>
	<!-- any classes that will not be exercised ("too simple to test") should be listed here -->
    </fileset>
</cobertura-instrument>
</target>
      

There is still an issue here: by default, NetBeans searches the classpath used to build tests before it searches the test build directory. This means that the default behavior is that the test runner will load the non-instrumented application classes instead of the instrumented ones. Changing the search order for running tests in the project properties solves this problem, but will not affect developers running tests without instrumentation.

Finally, generate a human-readable report (and a test report as well):

<!-- Produce the coverage report -->
<target name="-cobertura-report" if="cobertura.available">
<cobertura-report srcdir="${src.dir}" destdir="${build.test.coverage.dir}"
    datafile="${test-sys-prop.net.sourceforge.cobertura.datafile}"/>
</target>

<!-- Produce the coverage and JUnit test reports -->
<target name="-post-test-run" depends="-cobertura-report">
<junitreport todir="${build.test.results.dir}">
    <fileset dir="${build.test.results.dir}">
	<include name="**/TEST-*.xml"/>
    </fileset>
    <report format="frames" todir="${build.test.results.dir}"/>
</junitreport>
</target>
      

Happy happy joy joy: now test coverage reports will be generated for those that care, and silently skipped for those who don’t.