The Jenkins Warnings plugin collects compiler warnings or issues reported by static analysis tools and visualizes the results. It has built-in support for numerous static analysis tools (including several compilers), see the list of supported report formats.
Starting with release 5.x the Warnings plugin has support for the following Jenkins project types:
- Freestyle Project
- Maven Project
- Matrix Project
- Scripted Pipeline (sequential and parallel steps)
- Declarative Pipeline (sequential and parallel steps)
- Multi-branch Pipeline
The Warnings plugin provides the following features when added as a post build action (or step) to a job:
- The plugin scans the console log of a Jenkins build or files in the workspace of your job for any kind of issues.
There are almost one hundred report formats supported. Among the problems it can detect:
- errors from your compiler (C, C#, Java, etc.)
- warnings from a static analysis tool (CheckStyle, StyleCop, SpotBugs, etc.)
- duplications from a copy-and-paste detector (CPD, Simian, etc.)
- vulnerabilities
- open tasks in comments of your source files
- The plugin publishes a report of the issues found in your build, so you can navigate to a summary report from the
main build page. From there you can also dive into the details:
- distribution of new, fixed and outstanding issues
- distribution of the issues by severity, category, type, module, or package
- list of all issues including helpful comments from the reporting tool
- annotated source code of the affected files
- trend charts of the issues
Previously the Warnings plugin was part of the static analysis suite that provided the same set of features through several plugins (CheckStyle, PMD, Static Analysis Utilities, Analysis Collector etc.). In order to simplify the user experience and the development process, these plugins and the core functionality have been merged into the Warnings plugin. All other plugins are not required anymore and will not be supported in the future. If you currently use one of these plugins you should migrate to the new recorders and steps as soon as possible. I will still maintain the old code for a while, but the main development effort will be spent into the new code base.
Note: the dependency to the old API (analysis-core) is still required so that all existing jobs will work correctly. In future versions of the Warnings plugin I will make this dependency optional.
Pipelines calling the old static analysis steps (e.g., findbugs, checkstyle, etc.) need to call the new recordIssues
step now. The same step is used for all static analysis tools, the actual parser is selected
by using the step property tools. For more details on the set of available parameters please see section
Configuration.
Freestyle, Matrix or Maven Jobs using the old API used a so called Post Build Action that was provided by each individual plugin. E.g., the FindBugs plugin did provide the post build action "Publish FindBugs analysis results". These old plugin specific actions are not supported anymore, they are now marked with [Deprecated] in the user interface. Now you need to add a new post build step - this step now is called "Record static analysis results" for all static analysis tools. The selection of the tool is part of the configuration of this post build step. Note: the warnings produced by a post build step using the old API cannot not be read by the new post build actions. I.e., you can't see a combined history of the old and new results - you simply see two unrelated results. There is also no automatic conversion of results stored in the old format available.
The following plugins have been integrated into this version of the Warnings plugin:
- Android-Lint Plugin
- CheckStyle Plugin
- CCM Plugin
- Dry Plugin
- PMD Plugin
- FindBugs Plugin
All other plugins still need to be integrated or need to be refactored to use the new API.
The configuration of the plugin is the same for all Jenkins job types. It is enabled in the UI by adding
the post build action "Record static analysis results" to your job. In pipelines the plugin will be activated
by adding the step recordIssues. Note that for scripted pipelines some additional features are available to
aggregate and group issues, see section Advanced Pipeline Configuration for details.
In the following sections, both the graphical configuration and the pipeline configuration are shown side by side.
The basic configuration of the plugin is shown in the image above:
First of all you need to specify the tool that should be used to parse the console log or the report file. Depending on the selected tool you might configure some additional parameters as well. E.g., the SpotBugs configuration (see image above) provides a flag to select the priority mapping.
Each tool is identified by an ID that is used as URL to the results of the analysis. For each tool, a default URL (and name) is provided that can be changed if required. E.g., if you are going to use a parser multiple times then you need to specify different IDs for each of the invocations.
Then you need to specify the pattern of the report files that should be parsed and scanned for issues. If you do not specify a pattern, then the console log of your build will be scanned. For several popular tools a default pattern has been provided: in this case the default pattern will be used if the pattern is empty.
You can specify multiple tools (and patterns) that will be used with the same configuration. Due to a technical (or marketing) limitation of Jenkins it is not possible to select different configurations by using multiple post build actions.
One new feature is available by using the checkbox "Aggregate Results": if this option is selected, then one result is created that contains an aggregation of all issues of the selected tools. This is something the Static Analysis Collector Plugin provided previously. When this option is activated you get a unique entry point for all of your issues. The following screenshot shows this new behavior:
If this option is not enabled, then for each tool a separate result will be created. This result has a unique URL and icon, so you quickly see the difference between the created reports:
In the basic configuration section you can additionally choose if the step should run for failed builds as well. This option is disabled by default, since analysis results might be inaccurate if the build failed.
An example pipeline with these options is shown in the following snippet:
recordIssues
enabledForFailure: true, aggregatingResults : true,
tools: [
[tool: [$class: 'Java']],
[pattern: 'checkstyle-result.xml', tool: [$class: 'CheckStyle']]
]
If none of the built-in tools works in your project you have two ways to add additional tools.
The most flexible way is to define a new tool by writing a Java class that will be deployed in your own small Jenkins plugin, see Providing support for a custom static analysis tool for details.
If the format of your log messages is quite simple then you can define support for your tool by creating a simple tool configuration in Jenkins' user interface. This configuration takes a regular expression that will be used to match the console log. If the expression matches, then a Groovy script will be invoked that converts the matching text into a warning instance. Here is an example of such a Groovy based parser:
In order to let the scanner parse correctly your reports and source code files it is required to set the encodings:
An example pipeline with these options is shown in the following snippet:
recordIssues
sourceCodeEncoding: 'UTF-8', reportEncoding : 'ISO-8859-1', tools: [[tool: [$class: 'Java']]]
One important feature of the Warnings plugin is the classification of issues as new, outstanding and fixed:
- new: all issues, that are part of the current report but have not been shown up in the reference report
- fixed: all issues, that are part of the reference report but are not present in the current report anymore
- outstanding: all issues, that are part of the current and reference report
In order to compute this classification, the plugin requires a reference build (baseline). New, fixed, and outstanding issues are then computed by comparing the issues in the current build and the baseline. There are three options that control the selection of the reference build.
An example pipeline with these options is shown in the following snippet:
recordIssues
tools: [ [tool: [$class: 'Java']] ],
ignoreQualityGate: false, ignoreFailedBuilds: true, referenceJobName: 'my-project/master'
The created report of issues can be filtered afterwards. You can specify an arbitrary number of include or exclude filters. Currently, there is support for filtering issues by module name, package or namespace name, file name, category or type.
An example pipeline with these options is shown in the following snippet:
recordIssues
tools: [[pattern: '*.log', tool: [$class: 'Java']]],
filters: [includeFile('MyFile.*.java'), excludeCategory('WHITESPACE')]
You can define several quality gates that will be checked after the issues have been reported. These quality gates let you to modify Jenkins' build status so that you immediately see if the desired quality of your product is met. A build can be set to unstable or failed for each of these quality gates. All quality gates use a simple metric: the maximum number of issues that can be found and still pass a given quality gate.
An example pipeline with these options is shown in the following snippet:
recordIssues
tools: [[pattern: '*.log', tool: [$class: 'Java']]], unstableTotalHigh: 10, unstableNewAll: 1
The plugin can participate in the health report of your project. You can change the number of issues that change the health to 0% and 100%, respectively. Additionally, the severities that should be considered when creating the health report can be selected.
An example pipeline with these options is shown in the following snippet:
recordIssues
tools: [[pattern: '*.log', tool: [$class: 'Java']]],
healthy: 10, unhealthy: 100, minimumSeverity: 'HIGH'
This job adjusts the build health based on all warnings with severity HIGH and errors. If the build has 10 warnings or less then the health is at 100%. If the build has more than 100 warnings, then the health is at 0%.
Requirements for using the Warnings plugin in Jenkins Pipeline can be complex and sometimes controversial. In order to be as flexible as possible I decided to split the main step into two individual parts, which could then be used independently from each other.
The simple pipeline configuration is provided by the step recordIssues, it provides the same properties as
the post build action (see above). This step scans for issues
in a given set of files (or in the console log) and reports these issues in your build. You can use the
snippet generator to create a working snippet that calls this step. A typical example of this step
is shown in the following example:
recordIssues
enabledForFailure: true,
tools: [[pattern: '*.log', tool: [$class: 'Java']]],
filters: [includeFile('MyFile.*.java'), excludeCategory('WHITESPACE')]
In this example, the files '*.log' are scanned for Java issues. Only issues with a file name matching the pattern 'MyFile.*.java' are included. Issues with category 'WHITESPACE' will be excluded. The step will be executed even if the build failed.
In order to see all configuration options you can investigate the step implementation.
Configuration of the plugin in Declarative Pipeline jobs is the same as in Scripted Pipelines, see the following example that builds the analysis-model library on Jenkins:
pipeline {
agent 'any'
tools {
maven 'mvn-default'
jdk 'jdk-default'
}
stages {
stage ('Build') {
steps {
sh '${M2_HOME}/bin/mvn --batch-mode -V -U -e clean verify -Dsurefire.useFile=false -Dmaven.test.failure.ignore'
}
}
stage ('Analysis') {
steps {
sh '${M2_HOME}/bin/mvn --batch-mode -V -U -e checkstyle:checkstyle pmd:pmd pmd:cpd findbugs:findbugs spotbugs:spotbugs'
}
}
}
post {
always {
junit testResults: '**/target/surefire-reports/TEST-*.xml'
recordIssues enabledForFailure: true,
tools: [[tool: [$class: 'MavenConsole']],
[tool: [$class: 'Java']],
[tool: [$class: 'JavaDoc']]]
recordIssues enabledForFailure: true, tools: [[tool: [$class: 'CheckStyle']]]
recordIssues enabledForFailure: true, tools: [[tool: [$class: 'FindBugs']]]
recordIssues enabledForFailure: true, tools: [[tool: [$class: 'SpotBugs']]]
recordIssues enabledForFailure: true, tools: [[pattern: '**/target/cpd.xml', tool: [$class: 'Cpd']]]
recordIssues enabledForFailure: true, tools: [[pattern: '**/target/pmd.xml', tool: [$class: 'Pmd']]]
}
}
}
Sometimes publishing and reporting issues using a single step is not sufficient. E.g., if you build your product using several parallel steps and you want to combine the issues from all of these steps into a single result. Then you need to split scanning and aggregation. The plugin provides the following two steps:
scanForIssues: this step scans a report file or the console log with a particular parser and creates an intermediate AnnotatedReport object that contains the report. See step implementation for details.publishIssues: this step publishes a new report in your build that contains the aggregated results of severalscanForIssuessteps. See step implementation for details.
Example:
node {
stage ('Checkout') {
git branch:'5.0', url: 'git@github.com:jenkinsci/warnings-plugin.git'
}
stage ('Build') {
def mvnHome = tool 'mvn-default'
sh "${mvnHome}/bin/mvn --batch-mode -V -U -e clean verify -Dsurefire.useFile=false"
junit testResults: '**/target/*-reports/TEST-*.xml'
def java = scanForIssues tool: [$class: 'Java']
def javadoc = scanForIssues tool: [$class: 'JavaDoc']
publishIssues issues:[java, javadoc],
filters:[includePackage('io.jenkins.plugins.analysis.*')]
}
stage ('Analysis') {
def mvnHome = tool 'mvn-default'
sh "${mvnHome}/bin/mvn -batch-mode -V -U -e checkstyle:checkstyle pmd:pmd pmd:cpd findbugs:findbugs"
def checkstyle = scanForIssues tool: [$class: 'CheckStyle'], pattern: '**/target/checkstyle-result.xml'
publishIssues issues:[checkstyle]
def pmd = scanForIssues tool: [$class: 'Pmd'], pattern: '**/target/pmd.xml'
publishIssues issues:[pmd]
def cpd = scanForIssues tool: [$class: 'Cpd'], pattern: '**/target/cpd.xml'
publishIssues issues:[cpd]
def findbugs = scanForIssues tool: [$class: 'FindBugs'], pattern: '**/target/findbugsXml.xml'
publishIssues issues:[findbugs]
def maven = scanForIssues tool: [$class: 'MavenConsole']
publishIssues issues:[maven]
publishIssues id:'analysis', name:'White Mountains Issues',
issues:[checkstyle, pmd, findbugs],
filters:[includePackage('io.jenkins.plugins.analysis.*')]
}
The most important new features are described in the following sections.
One highlight of the plugin is the ability to categorize issues of subsequent builds as new, fixed and outstanding.
Using this feature makes it a lot easier to keep the quality of your project under control: you can focus only on those warnings that have been introduced recently.
Note: the detection of new warnings is based on a complex algorithm that tries to track the same warning in two two different versions of the source code. Depending on the extend of the modification of the source code it might produce some false positives, i.e., you might still get some new and fixed warnings even if there should be none. The accuracy of this algorithm is still ongoing research and will be refined in the next couple of months.
The plugin shows the distribution of the severities of the issues in a chart. It defines the following default severities, but additional ones might be added by plugins that extend the Warnings plugin.
- Error: Indicates an error that typically fails the build
- Warning (High, Normal, Low): Indicates a warning of the given priority. Mapping to the priorities is up to the individual parsers.
Note that not every parser is capable of producing warnings with a different severity. Some of the parsers simply use the same severity for all issues.
In order to see the trend of the analysis results, a chart showing the number of issues per build is also shown. This chart is used in the details page as well as in the job overview. Currently, type and configuration of the chart is fixed. This will be enhanced in future versions of the plugin.
You can get a fast and efficient overview of the reported set of issues in several aggregation views. Depending on the number or type of issues you will see the distribution of issues by
- Static Analysis Tool
- Module
- Package or Namespace
- Severity
- Category
- Type
Each of these detail views are interactive, i.e. you can navigate into a subset of the categorized issues.
The set of reported issues is shown in a modern and responsive table. The table is loaded on demand using an Ajax call. It provides the following features:
- Pagination: the number of issues is subdivided into several pages which can be selected by using the provided page links. Note that currently the pagination is done on the client side, i.e. it may take some time to obtain the whole table of issues from the server.
- Sorting: the table content can be sorted by clicking on ony of the table columns.
- Filtering, Searching: you can filter the shown issues by entering some text in the search box.
- Content Aware: columns are only shown if there is something useful to display. I.e., if a tool does not report an issues category, then the category will be automatically hidden.
- Responsive: the layout should adapt to the actual screen size.
- Details: the details message for an issue (if provided by the corresponding static analysis tool) is shown as child row within the table.
If not disabled in the job configuration, the plugin will execute git blame to determine who is the responsible
'author' of an issue. In the corresponding Source Control view all issues will be listed with author name, email and
commit ID.
In order to disable the blame feature, set the property blameDisabled to true, see the following example:
recordIssues
blameDisabled: true, tools: [[pattern: '*.log', tool: [$class: 'Java']]]
TBD.
The Warnings plugin is compatible with the Configuration as Code plugin. You can import parser configurations (see section Creating a new tool in the user interface) into Jenkins' system configuration using a YAML configuration in the form of the following example:
unclassified:
warningsParsers:
parsers:
- name: "Name of Parser (used in all user interface labels)"
id: "id" # the ID must be a valid URL
regexp: ".*"
script: "Groovy Script"The plugin provides two REST API endpoints.
You can obtain a summary of a particular analysis report by using the URL [tool-id]/api/xml
(or [tool-id]/api/json). The summary contains the number of issues, the quality gate status, and all
info and error messages.
Here is an example XML report:
<analysisResultApi _class='io.jenkins.plugins.analysis.core.restapi.AnalysisResultApi'>
<totalSize>3</totalSize>
<fixedSize>0</fixedSize>
<newSize>0</newSize>
<noIssuesSinceBuild>-1</noIssuesSinceBuild>
<successfulSinceBuild>-1</successfulSinceBuild>
<qualityGateStatus>WARNING</qualityGateStatus>
<owner _class='org.jenkinsci.plugins.workflow.job.WorkflowRun'>
<number>46</number>
<url>http://localhost:8080/view/White%20Mountains/job/Full%20Analysis%20-%20Model/46/</url>
</owner>
<infoMessage>Searching for all files in '/tmp/node1/workspace/Full Analysis - Model' that match the pattern
'**/target/spotbugsXml.xml'
</infoMessage>
<infoMessage>-> found 1 file</infoMessage>
<infoMessage>Successfully parsed file /tmp/node1/workspace/Full Analysis - Model/target/spotbugsXml.xml</infoMessage>
<infoMessage>-> found 3 issues (skipped 0 duplicates)</infoMessage>
<infoMessage>Post processing issues on 'node1' with encoding 'UTF-8'</infoMessage>
<infoMessage>Resolving absolute file names for all issues</infoMessage>
<infoMessage>-> affected files for all issues already have absolute paths</infoMessage>
<infoMessage>Copying affected files to Jenkins' build folder /Users/hafner/Development/jenkins/jobs/Full Analysis -
Model/builds/46
</infoMessage>
<infoMessage>-> 2 copied, 0 not in workspace, 0 not-found, 0 with I/O error</infoMessage>
<infoMessage>Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)</infoMessage>
<infoMessage>-> all issues already have a valid module name</infoMessage>
<infoMessage>Resolving package names (or namespaces) by parsing the affected files</infoMessage>
<infoMessage>-> all affected files already have a valid package name</infoMessage>
<infoMessage>Creating fingerprints for all affected code blocks to track issues over different builds</infoMessage>
<infoMessage>No filter has been set, publishing all 3 issues</infoMessage>
<infoMessage>No valid reference build found - all reported issues will be considered outstanding</infoMessage>
<infoMessage>Evaluating quality gates</infoMessage>
<infoMessage>-> WARNING - Total number of issues: 3 - Quality Gate: 1</infoMessage>
<infoMessage>-> Some quality gates have been missed: overall result is WARNING</infoMessage>
<infoMessage>Health report is disabled - skipping</infoMessage>
</analysisResultApi>The reported issues are also available as REST API. You can either query all issues or only the new, fixed, or outstanding issues. The corresponding URLs are:
[tool-id]/all/api/xml: lists all issues[tool-id]/fixed/api/xml: lists all fixed issues[tool-id]/new/api/xml: lists all new issues[tool-id]/outstanding/api/xml: lists all outstanding issues
Here is an example JSON report:
{
"_class" : "io.jenkins.plugins.analysis.core.restapi.ReportApi",
"issues" : [
{
"baseName" : "AbstractParser.java",
"category" : "EXPERIMENTAL",
"columnEnd" : 0,
"columnStart" : 0,
"description" : "",
"fileName" : "/private/tmp/node1/workspace/Full Analysis - Model/src/main/java/edu/hm/hafner/analysis/AbstractParser.java",
"fingerprint" : "be18f803030f2af690fbeef09eafa5c9",
"lineEnd" : 59,
"lineStart" : 59,
"message" : "edu.hm.hafner.analysis.AbstractParser.parse(File, Charset, Function) may fail to clean up java.io.InputStream",
"moduleName" : "Static Analysis Model and Parsers",
"origin" : "spotbugs",
"packageName" : "edu.hm.hafner.analysis",
"reference" : "46",
"severity" : "LOW",
"type" : "OBL_UNSATISFIED_OBLIGATION"
},
{
"baseName" : "ReportTest.java",
"category" : "STYLE",
"columnEnd" : 0,
"columnStart" : 0,
"description" : "",
"fileName" : "/private/tmp/node1/workspace/Full Analysis - Model/src/test/java/edu/hm/hafner/analysis/ReportTest.java",
"fingerprint" : "331d509297fad027813365ad0fb37e69",
"lineEnd" : 621,
"lineStart" : 621,
"message" : "Return value of Report.get(int) ignored, but method has no side effect",
"moduleName" : "Static Analysis Model and Parsers",
"origin" : "spotbugs",
"packageName" : "edu.hm.hafner.analysis",
"reference" : "46",
"severity" : "LOW",
"type" : "RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT"
},
{
"baseName" : "ReportTest.java",
"category" : "STYLE",
"columnEnd" : 0,
"columnStart" : 0,
"description" : "",
"fileName" : "/private/tmp/node1/workspace/Full Analysis - Model/src/test/java/edu/hm/hafner/analysis/ReportTest.java",
"fingerprint" : "1e641f9c0b35ed97140d639695e8ce18",
"lineEnd" : 624,
"lineStart" : 624,
"message" : "Return value of Report.get(int) ignored, but method has no side effect",
"moduleName" : "Static Analysis Model and Parsers",
"origin" : "spotbugs",
"packageName" : "edu.hm.hafner.analysis",
"reference" : "46",
"severity" : "LOW",
"type" : "RV_RETURN_VALUE_IGNORED_NO_SIDE_EFFECT"
}
],
"size" : 3
}The Warnings plugin provides the token ANALYSIS_ISSUES_COUNT that could be used in additional post build processing
steps, e.g. in the mailer. In order to use this tokens you need to install the latest release of the
Token Macro plugin.
The token has an optional parameter tool that could be used to select a particular analysis result.
Examples:
${ANALYSIS_ISSUES_COUNT}: expands to the aggregated number of issues of all analysis tools${ANALYSIS_ISSUES_COUNT, tool='checkstyle'}: expands to the total number of CheckStyle issues
Some of the existing features of the Warnings plugin are not yet ported to the API. These will be added one by one after the first 5.x release of the Warnings plugin:
- Visualization of open tasks
- Portlets for Jenkins' dashboard view
- View column that shows the number of issues in a job
- Quality gate based on the total number of issues