Monday, December 3, 2018

Jenkins 6: Advanced Configuration Area

Our build is advancing. Today we want to move optional fields into an advanced section and provide reasonable defaults for these entries.

Jenkins Tutorials

For a list of all jenkins related tutorials see Jenkins Tutorials Overview.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online.

Step 1: Advanced UI Section

To simplify the job setup we now move all parameters except the build message to an advanced section.
The only thing necessary in the config.jelly file is to create the section and move all affected input elements into it:
 <f:entry title="Custom build message" field="buildMessage">
  <f:textbox default="${descriptor.getDefaultBuildMessage()}" />
 </f:entry>

 <f:advanced>
  <f:entry title="Fail this build" field="failBuild">
   <f:checkbox />
  </f:entry>


  <f:entry title="Build Delay" field="buildDelay">
   <f:select />
  </f:entry>
 </f:advanced>

Afterwards the UI looks like this:


Step 2: Java Refactoring

Basically we do not need to change anything in the Java code to make this work. However we want to prepare a little for pipeline builds, so we remove non-required parameters from the constructor and create separate setters for them. To make Jenkins aware of these setters, use the @DataBoundSetter annotation:
public class HelloBuilder extends Builder implements SimpleBuildStep {

 private boolean fFailBuild = false;

 private String fBuildMessage;

 private String fBuildDelay = "none";

 @DataBoundConstructor
 public HelloBuilder(String buildMessage) {
  fBuildMessage = buildMessage;
 }

 @DataBoundSetter
 public void setFailBuild(boolean failBuild) {
  fFailBuild = failBuild;
 }
 
 @DataBoundSetter
 public void setBuildDelay(String buildDelay) {
  fBuildDelay = buildDelay;
 }
}

Whenever a parameter is not required, remove it from the constructor and use a setter for it.

Jenkins 5: Combo Boxes

Combo boxes are the next UI element we will add to our builder.

Jenkins Tutorials

For a list of all jenkins related tutorials see Jenkins Tutorials Overview.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online.

Step 1: UI Definition

In the config.jelly file we simply define that we want to use a combo box:
 <f:entry title="Build Delay" field="buildDelay">
  <f:select />
 </f:entry>
The definition does not contain entries to select. These will be populated by the Descriptor class.

Step 2: Item Definition

Jenkins will look for a method called doFill<field>Items in our Descriptor class to populate the combo. We are doing a first approach now to understand the scheme:
  public ListBoxModel doFillBuildDelayItems() {
   ListBoxModel model = new ListBoxModel();
   
   model.add(new Option("None", "none"));
   model.add(new Option("Short", "short"));
   model.add(new Option("Long", "long"));
   
   return model;
  }
ListBoxModel is basically an ArrayList of Option instances. The first string represents the text visible to the user, the second one the value that will actually be stored in our variable (see next step).

If we would populate the combo this way, the first item would always be selected by default, even if we re-open a job that was configured differently. The Option constructor allows for a third parameter defining the selected state. We then just need to know the value that got stored with the job definition. Therefore we can inject the desired query parameter into our method parameters:
  public ListBoxModel doFillBuildDelayItems(@QueryParameter String buildDelay) {
   ListBoxModel model = new ListBoxModel();

   model.add(new Option("None", "none", "none".equals(buildDelay)));
   model.add(new Option("Short", "short", "short".equals(buildDelay)));
   model.add(new Option("Long", "long" , "long".equals(buildDelay)));

   return model;
  }
Now buildDelay contains the value that got stored by the user when the build step was originally configured. By comparing its string representation we can set the right option in the combo. Typically combo options could be populated from an Enum. To reduce the risk of typos we could write a small helper to create our Options:
 public static Option createOption(Enum<?> enumOption, String jobOption) {
  return new Option(enumOption.toString(), enumOption.name(), enumOption.name().equals(jobOption));
 }

Step 3: Glueing it all together

Finally we need to extend our constructor with the new parameter. Then we can use it in our build step:
public class HelloBuilder extends Builder implements SimpleBuildStep {

 private String fBuildDelay;

 @DataBoundConstructor
 public HelloBuilder(boolean failBuild, String buildMessage, String buildDelay) {
  fBuildDelay = buildDelay;
 }

 @Override
 public void perform(Run<?, ?> run, FilePath workspace, Launcher launcher, TaskListener listener)
   throws InterruptedException, IOException {
  listener.getLogger().println("This is the Hello plugin!");
  listener.getLogger().println(getBuildMessage());

  switch (getBuildDelay()) {
  case "long":
   Thread.sleep(10 * 1000);
   break;

  case "short":
   Thread.sleep(3 * 1000);
   break;

  case "none":
   // fall through
  default:
   // nothing to do
  }

  if (isFailBuild())
   throw new AbortException("Build error forced by plugin settings");
 }

 public String getBuildDelay() {
  return fBuildDelay;
 }
}

Jenkins 4: Unit Tests

Now that our builder plugin is working we should start writing some unit tests for it.

Jenkins Tutorials

For a list of all jenkins related tutorials see Jenkins Tutorials Overview.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online.

Step 1: Wrinting a simple test case

Jenkins tests can be written as JUnit tests. The test instance needed for execution tests can be created using a JUnit Rule.

Create a new JUnit Test Case com.codeandme.jenkins.builder.HelloBuilderTest in the src/test/java folder:
public class HelloBuilderTest {

 @Rule
 public JenkinsRule fJenkinsInstance = new JenkinsRule();
 
 @Test
 public void successfulBuild() throws Exception {
  HelloBuilder builder = new HelloBuilder(false, "JUnit test run");
  
  FreeStyleProject job = fJenkinsInstance.createFreeStyleProject();
  job.getBuildersList().add(builder);
  FreeStyleBuild build = fJenkinsInstance.buildAndAssertSuccess(job);
  
  fJenkinsInstance.assertLogContains("JUnit test run", build);
 }
}
In line 4 we create a test instance for our unit test. This instance is used from line 10 onwards to create and run our test job. The instance provides a set of assertion commands which we use to check the build result and the log output of the job execution.

You can run these tests as JUnit tests right from Eclipse or you can execute them via maven by running
mvn test

Step 2: A test expecting an execution fail

We use the same approach as before. To check for a failed build we need to run the build job a little bit different:
 @Test
 public void failedBuild() throws Exception {
  HelloBuilder builder = new HelloBuilder(true, "JUnit test fail");
  
  FreeStyleProject job = fJenkinsInstance.createFreeStyleProject();
  job.getBuildersList().add(builder);
  QueueTaskFuture<FreeStyleBuild> buildResult = job.scheduleBuild2(0);
  
  fJenkinsInstance.assertBuildStatus(Result.FAILURE, buildResult);
  fJenkinsInstance.assertLogContains("JUnit test fail", buildResult.get());
 }

Wednesday, November 28, 2018

Jenkins 3: Text Input & Validation

Our builder UI is progressing: today we will add a text box with nice defaults and add input validation to it.

Jenkins Tutorials

For a list of all jenkins related tutorials see Jenkins Tutorials Overview.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online.

Step 1: The Text Box

Adding a text box is as simple as adding the checkbox before. Add a new entry to the config.jelly file:
 <f:entry title="Custom build message" field="buildMessage">
  <f:textbox />
 </f:entry>
Make sure you use a unique ID for field. Then add the new field to your builder by adding it to the constructor and create a getter for it:
public class HelloBuilder extends Builder implements SimpleBuildStep {

 private String fBuildMessage;

 @DataBoundConstructor
 public HelloBuilder(boolean failBuild, String buildMessage) {
  fFailBuild = failBuild;
  fBuildMessage = buildMessage;
 }

 @Override
 public void perform(Run<?, ?> run, FilePath workspace, Launcher launcher, TaskListener listener)
   throws InterruptedException, IOException {
  listener.getLogger().println(getBuildMessage());
  
  if (isFailBuild())
   throw new AbortException("Build error forced by plugin settings");
 }

 public String getBuildMessage() {
  return fBuildMessage;
 }
}
All done, give it a try!

Step 2: Default Value

Some example data might help our users when filling out the buid parameters. Therefore lets provide a nice default value. This is done by adding a default attribute to the textbox:
<f:textbox default="${descriptor.getDefaultBuildMessage()}" />
We could have provided the default text directly in the attribute data. Instead we decided to fetch the default dynamically from the Descriptor class. No magic data binding here, so we need to implement the method:
 public static final class Descriptor extends BuildStepDescriptor<Builder> {
  
  public String getDefaultBuildMessage() {
   return "This is a great build";
  }
 }

Step 3: Input Validation

Having no build message would result in an empty log file, which is not what we want. With input validation we can force users to enter some text to the input box. Validation is done by providing a validator in the Descriptor class:
 public static final class Descriptor extends BuildStepDescriptor<Builder> {

  public FormValidation doCheckBuildMessage(@QueryParameter String buildMessage) {
   if (buildMessage.isEmpty())
    return FormValidation.error("Please provide a build message.");
   else if (buildMessage.trim().isEmpty())
    return FormValidation.error("White space is not sufficient for a build message.");
   else
    return FormValidation.ok();
  }
 }
The method name needs to stick to the pattern doCheck<Parameter>. Normally you would only provide the parameter in question to that method (again the parameter name needs to match your field ID) but if needed you could add parameters for other fields of the builder. This comes in handy when parameters depend on each other.

Jenkins 2: A Builder Plugin & Some Jelly

In the previous tutorial we did the basic setup for jenkins plugin development. Now we will try to create a plugin that actually runs a build step.

Jenkins Tutorials

For a list of all jenkins related tutorials see Jenkins Tutorials Overview.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online.

Step 1: The Basic Builder

Maven allows to create a nice template for a builder plugin by calling
mvn archetype:generate -Dfilter=io.jenkins.archetypes:
then select the hello-world-plugin. But as we want to do it the hard way, we will add every single bit on our own and continue with the empty project from our previous tutorial.

The first thing we need is a class to implement our builder. Lets create a simple one. Create a new class com.codeandme.jenkins.builder.HelloBuilder:
package com.codeandme.jenkins.builder;

import java.io.IOException;

public class HelloBuilder extends Builder implements SimpleBuildStep {

 @DataBoundConstructor
 public HelloBuilder() {
 }

 @Override
 public void perform(Run<?, ?> run, FilePath workspace, Launcher launcher, TaskListener listener)
   throws InterruptedException, IOException {
  listener.getLogger().println("This is the Hello plugin!");
 }

 @Symbol("hello")
 @Extension
 public static final class Descriptor extends BuildStepDescriptor<Builder> {

  @Override
  public boolean isApplicable(Class<? extends AbstractProject> aClass) {
   return true;
  }

  @Override
  public String getDisplayName() {
   return "Code & Me - Hello World";
  }
 }
}

Jenkins expects the constructor to be augmented with the @DataBoundConstructor annotation. Later we will add our build parameters to it.

The perform() method is the heart of our implementation. This is where we define what the build step should actually do. In this tutorial we focus on the definition, not the execution so we are just printing some log message to detect that our build step got triggered.

Now lets put our focus on the Descriptor class. It actually describes what our plugin looks like, what parameter it uses and whether the user input is valid or not. You need to use a static class as a descriptor and augment it with the @Extension annotation to allow jenkins to detect it automatically.

isApplicable() might be the most important one as it denotes if ou plugin is usable for the current project type.

Start a jenkins test server using
mvn hpi:run -Djetty.port=8090
Create a new Freestyle Project and add your custom build step to it. Then execute the job and browse the log for our log message.

Step 2: Basic UI

Next we need a *.jelly file to describe how the UI should look like. Therefore create a new package in src/main/resources named com.codeandme.jenkins.builder.HelloBuilder. That is right, the package equals the class name of our builder class. Then create a config.jelly file inside that package:

<?jelly escape-by-default='true'?>
<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout" xmlns:t="/lib/hudson" xmlns:f="/lib/form">

 <f:block>
  <h1>Code &amp; Me Productions</h1>
  <p>We build the best <i>hellos</i> in the world</p>
 </f:block>

</j:jelly>

Inside the jelly definition we can use plain HTML code. Run your test instance again to see your changes in your project configuration view.

Step 3: Checkbox Input

Time to add some input. All build parameters need at least 2 steps of implementation: first we need to define the UI in the config.jelly file, then we need to define the parameters in the java class. Optionally we may add additional checks in the Descriptor class.

To define the UI for the checkbox we add following code to our jelly file:
 <f:entry title="Fail this build" field="failBuild">
  <f:checkbox />
 </f:entry>
This will create a label using the title field and a checkbox on the right side of the label. The field name is important as this is the ID of our field which we now use in the Java code:
public class HelloBuilder extends Builder implements SimpleBuildStep {

 private boolean fFailBuild;

 @DataBoundConstructor
 public HelloBuilder(boolean failBuild) {
  fFailBuild = failBuild;
 }

 @Override
 public void perform(Run<?, ?> run, FilePath workspace, Launcher launcher, TaskListener listener)
   throws InterruptedException, IOException {
  listener.getLogger().println("This is the Hello plugin!");

  if (isFailBuild())
   throw new AbortException("Build error forced by plugin settings");
 }

 public boolean isFailBuild() {
  return fFailBuild;
 }
}

The new parameter needs to be added to our constructor. Make sure you use the same name as in the jelly file. Additionally we need a getter for our parameter. It will be queried to populate the UI when you configure your job and when the job gets executed. Jenkins expects the name of the getter to match the field name of your jelly file.

During the build we evaluate our parameter and throw an AbortException in case our builder is expected to fail.

Step 4: Adding Help

Lots of parameters in Jenkins plugins show a help button on the righthand side of the form. These buttons automatically appear when corresponding help files exist in the right location.

A general help file for the builder named help.html needs to be placed next to the config.jelly file. You may add any arbitrary HTML content there, with no need to use <html> or <body> tags.

To provide help for our checkbox we create another help file named help-failBuild.html. See the pattern? We again use the field ID and Jenkins figures out the rest.

Instead of a plain HTML files we could also provide jelly files following the same name pattern.

Changes like adding help or beautifying jelly files can be done without restarting our test instance. Simply change the file and reload the page in your webbrowser of your test instance.

Further reading

Good documentation on writing forms seems to be rare on the internet. To me the best choice seems to be to find an existing plugin and browse the source code for reference. At least a list of all control types is available online.


Tuesday, November 27, 2018

Jenkins 1: IDE Setup and an Empty Plugin

I recently started to write my first Jenkins plugin and I thought I share the experience with you. For that reason I started a new series of tutorials.

Today we are building a simple Jenkins plugin using Eclipse. The basic steps of this tutorial are extracted from the Jenkins Plugin Tutorial.

Jenkins Tutorials

For a list of all jenkins related tutorials see Jenkins Tutorials Overview.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online.

Prerequisites

As you are interested in writing a plugin for Jenkins I expect that you have a rough idea what Jenkins is used for and how to administer it.

While our build environment allows to run a test instance of Jenkins with our plugin enabled I also liked to have a 'real' Jenkins instance available to test my plugins. Therefore I use docker to quickly get started with a woking Jenkins installation.

Once you have installed docker (extended tutorial for debian), you can download the latest Jenkins container by using
docker pull jenkins/jenkins:lts

Now run your container using
docker run -p 8080:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts

After the startup process your server is reachable via http://localhost:8080/.

To manage containers use commands
docker container ls
docker container stop <container name>

Step 1: Maven configuration

We will need maven installed and ready for creating the project, building and testing it, so make sure you have set it up correctly.

Maven needs some configuration ready to learn about jenkins plugins, therefore you need to adapt the configuration file slightly. On linux change ~/.m2/settings.xml, on windows modify/create %USERPROFILE%\.m2\settings.xml and set following content:
<settings>
 <pluginGroups>
  <pluginGroup>org.jenkins-ci.tools</pluginGroup>
 </pluginGroups>

 <profiles>
  <!-- Give access to Jenkins plugins -->
  <profile>
   <id>jenkins</id>
   <activation>
    <activeByDefault>true</activeByDefault>
   </activation>
   <repositories>
    <repository>
     <id>repo.jenkins-ci.org</id>
     <url>https://repo.jenkins-ci.org/public/</url>
    </repository>
   </repositories>
   <pluginRepositories>
    <pluginRepository>
     <id>repo.jenkins-ci.org</id>
     <url>https://repo.jenkins-ci.org/public/</url>
    </pluginRepository>
   </pluginRepositories>
  </profile>
 </profiles>
 <mirrors>
  <mirror>
   <id>repo.jenkins-ci.org</id>
   <url>https://repo.jenkins-ci.org/public/</url>
   <mirrorOf>m.g.o-public</mirrorOf>
  </mirror>
 </mirrors>
</settings>

Hint: On windows I had to remove the settings file <maven install folder>\conf\settings.xml as it was used in favor of my profile settings.

Step 2: Create the plugin skeleton

To create the initial project open a shell and execute:
mvn archetype:generate -Dfilter=io.jenkins.archetypes:empty-plugin
You will be asked some questions how your plugin should be configurated:
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] >>> maven-archetype-plugin:3.0.1:generate (default-cli) > generate-sources @ standalone-pom >>>
[INFO] 
[INFO] <<< maven-archetype-plugin:3.0.1:generate (default-cli) < generate-sources @ standalone-pom <<<
[INFO] 
[INFO] --- maven-archetype-plugin:3.0.1:generate (default-cli) @ standalone-pom ---
[INFO] Generating project in Interactive mode
[INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype:
1: remote -> io.jenkins.archetypes:empty-plugin (Skeleton of a Jenkins plugin with a POM and an empty source tree.)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1
Choose io.jenkins.archetypes:empty-plugin version: 
1: 1.0
2: 1.1
3: 1.2
4: 1.3
5: 1.4
Choose a number: 5: 
[INFO] Using property: groupId = unused
Define value for property 'artifactId': builder.hello
Define value for property 'version' 1.0-SNAPSHOT: : 
[INFO] Using property: package = unused
Confirm properties configuration:
groupId: unused
artifactId: builder.hello
version: 1.0-SNAPSHOT
package: unused
 Y: : 
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: empty-plugin:1.4
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: unused
[INFO] Parameter: artifactId, Value: builder.hello
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Parameter: package, Value: unused
[INFO] Parameter: packageInPathFormat, Value: unused
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Parameter: package, Value: unused
[INFO] Parameter: groupId, Value: unused
[INFO] Parameter: artifactId, Value: builder.hello
[INFO] Project created from Archetype in dir: ~/Eclipse/codeandme.blogspot.com/ws/builder.hello
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 36.202 s
[INFO] Finished at: 2018-11-27T20:34:15+01:00
[INFO] Final Memory: 16M/169M
[INFO] ------------------------------------------------------------------------

We just created the basic skeleton files and could start working right away. But as we want to do it the eclipse way we need to convert the project to a proper eclipse project. Therefore change into the created project directory and execute:
mvn -DdownloadSources=true -DdownloadJavadocs=true -DoutputDirectory=target/eclipse-classes eclipse:eclipse

The first run might take some time as maven has to fetch tons of dependencies. So sit back and enjoy the show...

Once this step is done we can import our project using the Eclipse import wizard using File / Import... and then select General/Existing Projects into Workspace. On the following page select the project folder that was created by maven.

Step 3: Update configuration files

The created pom.xml file for our plugin provides a good starting point for development. Typically you might want to update it a little before you actually start coding. Fields like name, description, license should be pretty clear. More interesting is
    <properties>
        <jenkins.version>2.7.3</jenkins.version>
        <java.level>7</java.level>
    </properties>
Upgrading the java level to 8 should be pretty safe these days. Further Jenkins 2.7.3 is really outdated. To check out which versions are available you may browse the jenkins artifactory server. Open the jenkins-war node and search for a version you would like to use.

I further adapt the .project file and remove the groovyNature as I am going to write some java code later.

Step 4: Build & Deploy the Plugin

To build your plugin simply run
mvn package
This will build and test your package. Further it creates an installable *.hpi package in the com.codeandme.jenkins.helloworld/target folder.

Step 5: Test the plugin in a test instance

To see your plugin in action you might want to execute it in a test instance of Jenkins. Maven will help us to set this up:
mvn hpi:run -Djetty.port=8090
After the boot phase, open up your browser and point to http://localhost:8090/jenkins to access your test instance.

Debugging is also quite simple, just add environment settings to setup your remote debugger. On Windows this would be:
set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=n
On Linux use
export MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=n
Then you should be able to setup a Remote Java Application debug configuration in Eclipse.

Writing a simple builder will be our next step, stay tuned for the next tutorial.

Thursday, December 14, 2017

Debugger 11: Watch expressions

Now that we have variables working, we might also want to include watch expressions to dynamically inspect code fragments.

Debug Framework Tutorials

For a list of all debug related tutorials see Debug Framework Tutorials Overview.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online.

Step 1: Provide the Watch Expression Delegate

Watch points are implemented via an extension point. So switch to your plugin.xml and add a new extension point for org.eclipse.debug.core.watchExpressionDelegates.
The new delegate simply points to our debugModel identifier: com.codeandme.debugModelPresentation.textinterpreter and provides a class implementation:
public class TextWatchExpressionDelegate implements IWatchExpressionDelegate {

 @Override
 public void evaluateExpression(String expression, IDebugElement context, IWatchExpressionListener listener) {
  if (context instanceof TextStackFrame)
   ((TextStackFrame) context).getDebugTarget().fireModelEvent(new EvaluateExpressionRequest(expression, listener));
 }
}
Delegates can decide on which context they may operate. For our interpreter we could evaluate expressions on StackFrames, Threads or the Process, but typically evaluations do take place on a dedicated StackFrame.

Step 2: Evaluation

Now we apply the usual pattern: send an event, let the debugger process it and send some event back to the debug target. Once the evaluation is done we then will inform the provided listener of the outcome of the evaluation.
public class TextDebugTarget extends TextDebugElement implements IDebugTarget, IEventProcessor {

 @Override
 public void handleEvent(final IDebugEvent event) {

   [...]

   } else if (event instanceof EvaluateExpressionResult) {
    IWatchExpressionListener listener = ((EvaluateExpressionResult) event).getOriginalRequest().getListener();
    TextWatchExpressionResult result = new TextWatchExpressionResult((EvaluateExpressionResult)event, this);
    listener.watchEvaluationFinished(result);    
   }
 }
The TextWatchExpressionResult uses a TextValue to represent the evaluation result. As before with variables we may support nested child variables within the value. In case the evaluation failed for some reason we may provide error messages which do get displayed in the Expressions view.