Wednesday, December 2, 2020

Add Checkstyle support to Eclipse, Maven, and Jenkins

After PMD and SpotBugs we will have a look at Checkstyle integration into the IDE and our maven builds. Parts of this tutorial are already covered by Lars' tutorial on Using the Checkstyle Eclipse plug-in.

Step 1: Add Eclipse IDE Support

First install the Checkstyle Plugin via the Eclipse Marketplace. Before we enable the checker, we need to define a ruleset to run against. As in the previous tutorials, we will setup project specific rules backed by one ruleset that can also be used by maven later on.

Create a new file for your rules in <yourProject>.releng/checkstyle/checkstyle_rules.xml. If you are familiar with writing rules just add them. In case you are new, you might want to start with one of the default rulesets of checkstyle.

Once we have some rules, we need to add them to our projects. Therefore right click on a project and select Checkstyle/Activate Checkstyle. This will add the project nature and a builder. To make use of our common ruleset, create a file <project>/.checkstyle with following content.

<?xml version="1.0" encoding="UTF-8"?>

<fileset-config file-format-version="1.2.0" simple-config="false" sync-formatter="false">
  <local-check-config name="Skills Checkstyle" location="/yourProject.releng/checkstyle/checkstyle_rules.xml" type="project" description="">
    <additional-data name="protect-config-file" value="false"/>
  </local-check-config>
  <fileset name="All files" enabled="true" check-config-name="Skills Checkstyle" local="true">
    <file-match-pattern match-pattern=".java$" include-pattern="true"/>
  </fileset>
</fileset-config>

Make sure to adapt the name and location attributes of local-check-config according to your project structure.

Checkstyle will now run automatically on builds or can be triggered manually via the context menu: Checkstyle/Check Code with Checkstyle.

Step 2: Modifying Rules

While we had to do our setup manually, we can now use the UI integration to adapt our rules. Select the Properties context entry from a project and navigate to Checkstyle, page Local Check Configurations. There select your ruleset and click Configure... The following dialog allows to add/remove rules and to change rule properties. All your changes are backed by our checkstyle_rules.xml file we created earlier.

Step 3: Maven Integration

We need to add the Maven Checkstyle Plugin to our build. Therefore add following section to your master pom:

	<properties>
		<maven.checkstyle.version>3.1.1</maven.checkstyle.version>
	</properties>

	<build>
		<plugins>
			<!-- enable checkstyle code analysis -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-checkstyle-plugin</artifactId>
				<version>${maven.checkstyle.version}</version>
				<configuration>
					<configLocation>../../releng/yourProject.releng/checkstyle/checkstyle_rules.xml</configLocation>
					<linkXRef>false</linkXRef>
				</configuration>

				<executions>
					<execution>
						<id>checkstyle-integration</id>
						<phase>verify</phase>
						<goals>
							<goal>check</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>

In the configuration we address the ruleset we also use for the IDE plugin. Make sure that the relative path fits to your project setup. In the provided setup execution is bound to the verify phase.

Step 4: File Exclusions

Excluding files has to be handled differently for IDE and Maven. The Eclipse plugin allows to define inclusions and exclusions via file-match-pattern entries in the .checkstyle configuration file. To exclude a certain package use:

  <fileset name="All files" enabled="true" check-config-name="Skills Checkstyle" local="true">
    ...
    <file-match-pattern match-pattern="org.yourproject.generated.package.*$" include-pattern="false"/>
  </fileset>

In maven we need to add exclusions via the plugin configuration section. Typically such exclusions would go to the pom of a specific project and not the master pom:

	<build>
		<plugins>
			<!-- remove generated resources from checkstyle code analysis -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-checkstyle-plugin</artifactId>
				<version>${maven.checkstyle.version}</version>

				<configuration>
					<excludes>**/org/yourproject/generated/package/**/*</excludes>
				</configuration>
			</plugin>
		</plugins>
	</build>

Step 5: Jenkins Integration

If you followed my previous tutorials on code checkers, then this is business as usual: use the warnings-ng plugin on Jenkins to track our findings:

	recordIssues tools: [checkStyle()]

Try out the live chart on the skills project.

Tuesday, November 24, 2020

Add SpotBugs support to Eclipse, Maven, and Jenkins

SpotBugs (successor of FindBugs) is a tool for static code analysis, similar like PMD. Both tools help to detect bad code constructs which might need improvement. As they partly detect different issues, they may be well combined and used simultaneously.

Step 1: Add Eclipse IDE Support

The SpotBugs Eclipse Plugin can be installed directly via the Eclipse Marketplace.

After installation projects can be configured to use it from the projects Properties context menu. Navigate to the SpotBugs category and enable all checkboxes on the main site. Further set Minimum rank to report to 20 and Minimum confidence to report to Low.

Once done SpotBugs immediately scans the project for problems. Found issues are displayed as custom markers in editors. Further they are visible in the Bug Explorer view as well as in the Problems view.

SpotBugs also comes with a label decoration on elements in the Package Explorer. If you do not like these then disable all Bug count decorator entries in Preferences/General/Appearance/Label Decorations.

Step 2: Maven Integration

Integration is done via the SpotBugs Maven Plugin. To enable, add following section to your master pom:

	<properties>
		<maven.spotbugs.version>4.1.4</maven.spotbugs.version>
	</properties>

	<build>
		<plugins>
			<!-- enable spotbugs code analysis -->
			<plugin>
				<groupId>com.github.spotbugs</groupId>
				<artifactId>spotbugs-maven-plugin</artifactId>
				<version>${maven.spotbugs.version}</version>

				<configuration>
					<effort>Max</effort>
					<threshold>Low</threshold>
					<fork>false</fork>
				</configuration>

				<executions>
					<execution>
						<id>spotbugs-integration</id>
						<phase>verify</phase>
						<goals>
							<goal>spotbugs</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>

The execution entry takes care that the spotbugs goal is automatically executed during the verify phase. If you remove the execution section you would have to call the spotbugs goal separately:

mvn spotbugs:spotbugs

Step 3: File Exclusions

You might have code that you do not want to get checked (eg generated files). Exclusions need to be defined in an xml file. A simple filter on package level looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<FindBugsFilter>
    <!-- skip EMF generated packages -->
    <Match>
        <Package name="~org\.eclipse\.skills\.model.*" />
    </Match>
</FindBugsFilter>

See the documentation for a full description of filter definitions.

Once defined this file can be used from the SpotBugs Eclipse plugin as well as from the maven setup.

To simplify the maven configuration we can add following profile to our master pom:

	<profiles>
		<profile>
			<!-- apply filter when filter file exists -->
			<id>auto-spotbugs-exclude</id>
			<activation>
				<file>
					<exists>.settings/spotbugs-exclude.xml</exists>
				</file>
			</activation>

			<build>
				<plugins>
					<!-- enable spotbugs exclude filter -->
					<plugin>
						<groupId>com.github.spotbugs</groupId>
						<artifactId>spotbugs-maven-plugin</artifactId>
						<version>${maven.spotbugs.version}</version>

						<configuration>
							<excludeFilterFile>.settings/spotbugs-exclude.xml</excludeFilterFile>
						</configuration>
					</plugin>
				</plugins>
			</build>
		</profile>
	</profiles>

It gets automatically enabled when a file .settings/spotbugs-exclude.xml exists in the current project.

Step 4: Jenkins Integration

Like with PMD, we again use the warnings-ng plugin on Jenkins to track our findings:

	recordIssues tools: [spotBugs(useRankAsPriority: true)]

Try out the live chart on the skills project.

Final Thoughts

PMD is smoother on integration as it stores its rulesets in a common file which can be shared by maven and the Eclipse plugin. SpotBugs currently requires to manage rulesets separately. Still both can be implemented in a way that users automatically get the same warnings in maven and the IDE.

Friday, November 20, 2020

Add Code Coverage Reports to Eclipse, Maven, and Jenkins

Code coverage may provide some insights in your tests. They show which classes, lines of codes, and conditional branches are called by your tests. A high percentage of coverage does not automatically mean that your tests are great - as you might not have a single assertion in your test code - but at least they can give you an impression of dark areas in your code base.

This article is heavily based on the article of Lorenzo Bettini on JaCoCo Code Coverage and Report of multiple Eclipse plug-in projects, so the credits for this setup are his!

Step 1: Eclipse IDE Setup

Coverage in Java projects is typically tracked with the JaCoCo library. The according plugin for Eclipse is called EclEmma and is available via the Eclipse Marketplace.

After installation you have a new run target 

that adds coverage information to your execution. Only thing to do is to rerun your unit tests and check out the Coverage view.


Multiple coverage sessions can be combined into one. That allows to accumulate the results of multiple unit tests into a single coverage report.

Step 2: Tycho integration

For the next steps I expect that you basically followed my tycho tutorials and have a similar setup.

First we need to enable JaCoCo in our builds:

	<build>
		<plugins>
			<!-- enable JaCoCo code coverage -->
			<plugin>
				<groupId>org.jacoco</groupId>
				<artifactId>jacoco-maven-plugin</artifactId>
				<version>0.8.6</version>

				<configuration>
					<output>file</output>
				</configuration>

				<executions>
					<execution>
						<id>jacoco-initialize</id>
						<phase>pre-integration-test</phase>
						<goals>
							<goal>prepare-agent</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>

Tycho surefire executes unit tests in the maven integration-test phase, therefore we start the agent right before. This plugin needs to be active for any plugin of type eclipse-plugin-test (see tycho tutorial), but it is safe to put it in the master pom of your *.releng project.

Now each test run creates coverage reports. For analysis purposes we need to merge them into a single one. Therefore create a new General/Project in your workspace, named *.releng.coverage. In the pom.xml file we need to add a step to aggregate all reports into one:

	<build>
		<plugins>
			<plugin>
				<groupId>org.jacoco</groupId>
				<artifactId>jacoco-maven-plugin</artifactId>
				<version>${jacoco.version}</version>
				<executions>
					<execution>
						<phase>verify</phase>
						<goals>
							<goal>report-aggregate</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>

Afterwards we need to define dependencies for the projects containing our source code:

	<dependencies>
		<!-- Code dependencies to show coverage on -->
		<dependency>
			<groupId>com.example</groupId>
			<artifactId>com.example.plugin1</artifactId>
			<version>0.1.0-SNAPSHOT</version>
			<scope>compile</scope>
		</dependency>

		<dependency>
			<groupId>com.example</groupId>
			<artifactId>com.example.plugin2</artifactId>
			<version>0.1.0-SNAPSHOT</version>
			<scope>compile</scope>
		</dependency>
		...
	</dependencies>

Further we need dependencies to our test fragments (mind the different scope) :

	<dependencies>
		...
		<!-- Test dependencies -->
		<dependency>
			<groupId>com.example</groupId>
			<artifactId>com.example.project1.test</artifactId>
			<version>0.1.0-SNAPSHOT</version>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>com.example</groupId>
			<artifactId>com.example.project2.test</artifactId>
			<version>0.1.0-SNAPSHOT</version>
			<scope>test</scope>
		</dependency>
		...
	</dependencies>

If unsure, have a look at a complete pom file.

Finally add the new project as a module to your master pom:

	<modules>
		...
		<module>../your.project.releng.coverage</module>
		...
	</modules>

The maven build now generates *.releng.coverage/target/site/jacoco-aggregate/jacoco.xml which can be picked up by various tools. Further you get a nice HTML report in the same folder for free.

Step 3: Jenkins reports

While you may directly publish the HTML report on your jenkins builds, I prefer to use the Code Coverage plugin. With a single instruction in your pipeline

	publishCoverage adapters: [jacocoAdapter(path: 'releng/com.example.releng.coverage/target/site/jacoco-aggregate/jacoco.xml')], sourceFileResolver: sourceFiles('STORE_LAST_BUILD')

it generates nice, interactive reports like these:

You may also have a look at this live report to play around with.

Wednesday, November 18, 2020

Add PMD support to Eclipse, Maven, and Jenkins

PMD is a static code analyzer that checks your source for problematic code constructs, design patterns, and code style.

The code smells reported on grown projects might be huge at first, but PMD allows to customize its rules and to adapt them to your needs.

Step 1: Add PMD support to Eclipse

I am using eclipse-pmd which can be installed from the Eclipse Marketplace.

Step 2: Define a ruleset

PMD needs a ruleset to run against. It is stored as an xml file and can be global, workspace specific or project specific. The choice is up to you. For eclipse projects I typically have a "releng" project to host all my configuration files.

A default ruleset looks like this:

<?xml version="1.0"?>
<ruleset name="Custom Rules"
	xmlns="http://pmd.sourceforge.net/ruleset/2.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://pmd.sourceforge.net/ruleset/2.0.0 https://pmd.sourceforge.io/ruleset_2_0_0.xsd">

	<description>custom ruleset</description>

	<rule ref="category/java/bestpractices.xml" />
	<rule ref="category/java/codestyle.xml" />
	<rule ref="category/java/design.xml" />
	<rule ref="category/java/documentation.xml" />
	<rule ref="category/java/errorprone.xml" />
	<rule ref="category/java/multithreading.xml" />
	<rule ref="category/java/performance.xml" />
	<rule ref="category/java/security.xml" />
</ruleset>

Store your ruleset somewhere in your workspace or on your file system.

Step 3: Enable PMD on project level

Right click on a project in your Eclipse workspace and select Properties. In PMD section check Enable PMD for this project and Add... the ruleset file stored before. The Name is not important and can be freely chosen.

Your rules are live now and PMD should immediately start to add warnings to your code and the Problems view.

Step 4: Refine your rules

The default ruleset might report some issues you do want to treat differently in your project. Therefore you may change rules by setting parameters or disable unwanted rules at all. To alter a rule, you first have to find it in the list of available rules. For disabling you just need to add an exclude node to your rule settings file, eg:

	<rule ref="category/java/bestpractices.xml">
		<!-- logger takes care of guarding -->
		<exclude name="GuardLogStatement" />
	</rule>

Configuring a rule can be done like this:

	<rule ref="category/java/codestyle.xml/ClassNamingConventions">
		<properties>
			<property name="utilityClassPattern"
				value="[A-Z][a-zA-Z0-9]+" />
		</properties>
	</rule>

A full working ruleset as used by one of my projects can be viewed online.

Whenever you change your ruleset you need to recompile your project to get these rules applied. You may do so by selecting Project/Clean... from the main menu.

Step 5: Maven integration

Integration is done by the maven-pmd-plugin. Just add following section to your pom:

	<build>
		<plugins>
			<!-- enable PMD code analysis -->
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-pmd-plugin</artifactId>
				<version>3.13.0</version>
				<configuration>
					<linkXRef>false</linkXRef>
					<rulesets>path/to/your/ruleset.xml</rulesets>
				</configuration>
			</plugin>
		</plugins>
	</build>

Make sure to adapt the path to your ruleset accordingly.

Afterwards run your build using

mvn pmd:pmd pmd:cpd

If you use the maven-site-plugin, you may additionally generate html reports of PMD findings.

Step 6: Jenkins integration

Static reports are nice, but charts over time/commits are even better. In case you use Jenkins you may have a look at the warnings-ng plugin. When you generate yout pmd.xml files via maven, this plugin can pick them up and draw nice reports. In a pipeline build this only needs one line:

recordIssues(tools: [cpd(), pmdParser()])

to get charts like these:


Try out the live chart on the skills project.

Finally the plugin even allows to compare the amount of issues against a baseline. This allows to add  quality gates, eg to fail the build in case your issue count increases. I strongly encourage to enforce such rules. Otherwise warnings are nice but do get ignored by everybody.