Home

Awesome

Violation Comments to GitLab Plugin

This is a Jenkins plugin for Violation Comments to GitLab Lib. This plugin will find report files from static code analysis and comment GitLab pull requests with the content.

An alternative is also to have violations-command-line convert the violations to CodeClimate format and upload that to GitLab. It is explained in readme of violations-command-line.

Available in Jenkins here.

Example of supported reports are available here.

There is a complete running example available here: https://github.com/tomasbjerre/jenkins-configuration-as-code-sandbox

You can also do this with a command line tool.

A number of parsers have been implemented. Some parsers can parse output from several reporters.

ReporterParserNotes
ARM-GCCCLANG
AndroidLintANDROIDLINT
Ansible-LaterANSIBLELATERWith json format
AnsibleLintFLAKE8With -p
BanditCLANGWith bandit -r examples/ -f custom -o bandit.out --msg-template "{abspath}:{line}: {severity}: {test_id}: {msg}"
CLangCLANG
CPDCPD
CPPCheckCPPCHECKWith cppcheck test.cpp --output-file=cppcheck.xml --xml
CPPLintCPPLINT
CSSLintCSSLINT
CheckstyleCHECKSTYLE
CloudFormation LinterJUNITcfn-lint . -f junit --output-file report-junit.xml
CodeClimateCODECLIMATE
CodeNarcCODENARC
CoverityCOVERITY
DartMACHINEWith dart analyze --format=machine
Dependency CheckSARIFUsing --format SARIF
DetektCHECKSTYLEWith --output-format xml.
DocFXDOCFX
DoxygenCLANG
ERBCLANGWith erb -P -x -T '-' "${it}" | ruby -c 2>&1 >/dev/null | grep '^-' | sed -E 's/^-([a-zA-Z0-9:]+)/${filename}\1 ERROR:/p' > erbfiles.out.
ESLintCHECKSTYLEWith format: 'checkstyle'.
FindbugsFINDBUGS
Flake8FLAKE8
FxCopFXCOP
GCCCLANG
GHSGHS
GendarmeGENDARME
Generic reporterGENERICWill create one single violation with all the content as message.
GoLintGOLINT
GoVetGOLINTSame format as GoLint.
GolangCI-LintCHECKSTYLEWith --out-format=checkstyle.
GoogleErrorProneGOOGLEERRORPRONE
HadoLintCHECKSTYLEWith -f checkstyle
IARIARWith --no_wrap_diagnostics
InferPMDFacebook Infer. With --pmd-xml.
JACOCOJACOCO
JCReportJCREPORT
JSHintJSLINTWith --reporter=jslint or the CHECKSTYLE parser with --reporter=checkstyle
JUnitJUNITIt only contains the failures.
KTLintCHECKSTYLE
KlocworkKLOCWORK
KotlinGradleKOTLINGRADLEOutput from Kotlin Gradle Plugin.
KotlinMavenKOTLINMAVENOutput from Kotlin Maven Plugin.
LintLINTA common XML format, used by different linters.
MSBuildLogMSBULDLOGWith -fileLogger use .*msbuild\\.log$ as pattern or -fl -flp:logfile=MyProjectOutput.log;verbosity=diagnostic for a custom output filename
MSCppMSCPP
MccabeFLAKE8
MyPyMYPY
NullAwayGOOGLEERRORPRONESame format as Google Error Prone.
PCLintPCLINTPC-Lint using the same output format as the Jenkins warnings plugin, details here
PHPCSCHECKSTYLEWith phpcs api.php --report=checkstyle.
PHPPMDPMDWith phpmd api.php xml ruleset.xml.
PMDPMD
Pep8FLAKE8
PerlCriticPERLCRITIC
PiTestPITEST
ProtoLintPROTOLINT
Puppet-LintCLANGWith -log-format %{fullpath}:%{line}:%{column}: %{kind}: %{message}
PyDocStylePYDOCSTYLE
PyFlakesFLAKE8
PyLintPYLINTWith pylint --output-format=parseable.
ReSharperRESHARPER
RubyCopCLANGWith rubycop -f clang file.rb
SARIFSARIFv2.x. Microsoft Visual C# can generate it with ErrorLog="BuildErrors.sarif,version=2".
SbtScalacSBTSCALAC
ScalastyleCHECKSTYLE
SemgrepSEMGREPWith --json.
SimianSIMIAN
SonarSONARWith mvn sonar:sonar -Dsonar.analysis.mode=preview -Dsonar.report.export.path=sonar-report.json. Removed in 7.7, see SONAR-11670 but can be retrieved with: curl --silent 'http://sonar-server/api/issues/search?componentKeys=unique-key&resolved=false' | jq -f sonar-report-builder.jq > sonar-report.json.
SpotbugsFINDBUGS
StyleCopSTYLECOP
SwiftLintCHECKSTYLEWith --reporter checkstyle.
TSLintCHECKSTYLEWith -t checkstyle
ValgrindVALGRINDWith --xml=yes.
XMLLintXMLLINT
XUnitXUNITIt only contains the failures.
YAMLLintYAMLLINTWith -f parsable
ZPTLintZPTLINT

52 parsers and 79 reporters.

Missing a format? Open an issue here!

There is also:

Notify Jenkins from GitLab

Merge

You must perform the merge before build. If you don't perform the merge, the reported violations will refer to other lines then those in the pull request. The merge can be done with a shell script like this.

echo ---
echo --- Merging from $FROM in $FROMREPO to $TO in $TOREPO
echo ---
git clone $TOREPO
cd *
git reset --hard $TO
git status
git remote add from $FROMREPO
git fetch from
git merge $FROM
git --no-pager log --max-count=10 --graph --abbrev-commit

Your build command here!

Screenshots

Comments can be made on the diff with one comment per violation createSingleFileComments.

Example comment on diff

Or one big comment can be made, createCommentWithAllSingleFileComments.

Example comment

Job DSL Plugin

This plugin can be used with the Job DSL Plugin. Here is an example using Generic Webhook Trigger plugin, HTTP Request Plugin and Conditional BuildStep Plugin.

job('GitLab_MR_Builder') {
 concurrentBuild()
 quietPeriod(0)
 parameters {
  stringParam('MERGE_REQUEST_TO_URL', '')
  stringParam('MERGE_REQUEST_FROM_URL', '')
  stringParam('MERGE_REQUEST_TO_BRANCH', '')
  stringParam('MERGE_REQUEST_FROM_BRANCH', '')
 }
 scm {
  git {
   remote {
    name('origin')
    url('$MERGE_REQUEST_TO_URL')
   }
   remote {
    name('upstream')
    url('$MERGE_REQUEST_FROM_URL')
   }
   branch('$MERGE_REQUEST_FROM_BRANCH')
   extensions {
    mergeOptions {
     remote('upstream')
     branch('$MERGE_REQUEST_TO_BRANCH')
    }
   }
  }
 }
 triggers {
  genericTrigger {
   genericVariables {
    genericVariable {
     key("MERGE_REQUEST_TO_URL")
     value("\$.object_attributes.target.git_http_url")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MERGE_REQUEST_FROM_URL")
     value("\$.object_attributes.source.git_http_url")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MERGE_REQUEST_TO_BRANCH")
     value("\$.object_attributes.target_branch")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MERGE_REQUEST_FROM_BRANCH")
     value("\$.object_attributes.source_branch")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("PROJECT_ID")
     value("\$.object_attributes.target_project_id")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MERGE_REQUST_ID")
     value("\$.object_attributes.id")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MR_OBJECT_KIND")
     value("\$.object_kind")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MR_OLD_REV")
     value("\$.object_attributes.oldrev")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MR_ACTION")
     value("\$.object_attributes.action")
     expressionType("JSONPath")
     regexpFilter("")
    }
   }
   regexpFilterText("\$MR_OBJECT_KIND \$MR_ACTION \$MR_OLD_REV")
   regexpFilterExpression("^merge_request\\s(update\\s.{40}\$|open.*)")
  }
 }
 steps {
  httpRequest {
   url("http://gitlab:880/api/v3/projects/\$PROJECT_ID/merge_requests/\$MERGE_REQUST_ID/notes?private_token=AvAkp6HtUvzpesPypXSk")
   consoleLogResponseBody(true)
   httpMode("POST")
   requestBody('body=Building... %20\$BUILD_URL')
   }
 
  shell('./gradlew build')
 
  conditionalBuilder {
   runCondition {
    statusCondition {
     worstResult('SUCCESS')
     bestResult('SUCCESS')
    }
    runner {
     runUnstable()
    }
    conditionalbuilders {
     httpRequest {
      url('http://gitlab:880/api/v3/projects/\$PROJECT_ID/merge_requests/\$MERGE_REQUST_ID/notes?private_token=AvAkp6HtUvzpesPypXSk')
      consoleLogResponseBody(true)
      httpMode('POST')
      requestBody('body=SUCCESS%20\$BUILD_URL')
     }
    }
   }
  }
 
  conditionalBuilder {
   runCondition {
    statusCondition {
     worstResult('FAILURE')
     bestResult('FAILURE')
    }
    runner {
     runUnstable()
    }
    conditionalbuilders {
     httpRequest {
      url('http://gitlab:880/api/v3/projects/\$PROJECT_ID/merge_requests/\$MERGE_REQUST_ID/notes?private_token=AvAkp6HtUvzpesPypXSk')
      consoleLogResponseBody(true)
      httpMode('POST')
      requestBody('body=FAIL%20\$BUILD_URL')
     }
    }
   }
  }
 }
 publishers {
  violationsToGitLabRecorder {
   config {
    gitLabUrl("http://gitlab:880/")
    projectId("\$PROJECT_ID")
    mergeRequestIid("\$MERGE_REQUST_IID")
    
    // Only specify proxy if you need it
    proxyUri('')
    proxyCredentialsId('') // A username/password credential
 
    commentOnlyChangedContent(true)
    commentOnlyChangedContentContext(0)
    commentOnlyChangedFiles(true)
    createSingleFileComments(true)
    createCommentWithAllSingleFileComments(true)
    minSeverity('INFO')
    maxNumberOfViolations(99999)
    
    //You may want this when troubleshooting things
    enableLogging(true)
    
 
    apiTokenCredentialsId("gitlabtoken") // A secret text credential
    apiTokenPrivate(true)
    authMethodHeader(true)
    ignoreCertificateErrors(true)
    
    commentTemplate("""
    **Reporter**: {{violation.reporter}}{{#violation.rule}}
    
    **Rule**: {{violation.rule}}{{/violation.rule}}
    **Severity**: {{violation.severity}}
    **File**: {{violation.file}} L{{violation.startLine}}{{#violation.source}}
    
    **Source**: {{violation.source}}{{/violation.source}}
    
    {{violation.message}}
    """)
 
    violationConfigs {
     violationConfig {
      parser("FINDBUGS")
      reporter("Findbugs")
      pattern(".*/findbugs/.*\\.xml\$")
     }
     violationConfig {
      parser("CHECKSTYLE")
      reporter("Checkstyle")
      pattern(".*/checkstyle/.*\\.xml\$")
     }
    }
   }
  }
 }
}

Pipeline Plugin

Here is an example pipeline that will merge, run unit tests, run static code analysis and finally report back to GitLab. It requires the GitLab Plugin.

pipelineJob("merge-request-pipeline") {
 concurrentBuild()
 quietPeriod(0)
 authenticationToken("thetoken")
 triggers {
  genericTrigger {
   genericVariables {
    genericVariable {
     key("MERGE_REQUEST_TO_URL")
     value("\$.object_attributes.target.git_http_url")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MERGE_REQUEST_FROM_URL")
     value("\$.object_attributes.source.git_http_url")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MERGE_REQUEST_TO_BRANCH")
     value("\$.object_attributes.target_branch")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MERGE_REQUEST_FROM_BRANCH")
     value("\$.object_attributes.source_branch")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("PROJECT_ID")
     value("\$.object_attributes.target_project_id")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("PROJECT_PATH")
     value("\$.object_attributes.target.path_with_namespace")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MERGE_REQUST_IID")
     value("\$.object_attributes.iid")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MR_OBJECT_KIND")
     value("\$.object_kind")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MR_OLD_REV")
     value("\$.object_attributes.oldrev")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MR_ACTION")
     value("\$.object_attributes.action")
     expressionType("JSONPath")
     regexpFilter("")
    }
    genericVariable {
     key("MR_TITLE")
     value("\$.object_attributes.title")
     expressionType("JSONPath")
     regexpFilter("")
    }
   }
   regexpFilterText("\$MR_OBJECT_KIND \$MR_ACTION \$MR_OLD_REV")
   regexpFilterExpression("^merge_request\\s(update\\s.{40}\$|open.*)")
  }
 }
 
 definition {
  cps {
   script(readFileFromWorkspace('merge_request_pipeline.pipeline'))
   sandbox()
  }
 }
}

And the merge_request_pipeline.pipeline contains

def commentMr(projectId, mergeRequestIid, comment) {
 def body = comment
 .replaceAll(" ","%20")
 .replaceAll("/","%2F")
 def project = projectId
 .replaceAll("/","%2F")
 sh "curl http://gitlab:80/api/v4/projects/$project/merge_requests/$mergeRequestIid/notes -H 'PRIVATE-TOKEN: 6xRcmSzPzzEXeS2qqr7R' -X POST -d \"body="+body+"\""
}
  
node {
 deleteDir()
 currentBuild.description = "$MR_TITLE from $MERGE_REQUEST_FROM_BRANCH to $MERGE_REQUEST_TO_BRANCH"
  
 commentMr(env.PROJECT_PATH,env.MERGE_REQUST_IID,"Verifierar $MERGE_REQUEST_FROM_BRANCH... ${env.BUILD_URL}")
  
 stage('Merge') {
  sh "git init"
  sh "git fetch --no-tags $MERGE_REQUEST_TO_URL +refs/heads/*:refs/remotes/origin/* --depth=200"
  sh "git checkout origin/${env.MERGE_REQUEST_TO_BRANCH}"
  sh "git config user.email 'je@nkins.domain'"
  sh "git config user.name 'jenkins'"
  sh "git merge origin/${env.MERGE_REQUEST_FROM_BRANCH}"
  sh "git log --graph --abbrev-commit --max-count=10"
 }
  
 stage('Compile') {
  sh "./gradlew assemble"
 }
  
 stage('Unit test') {
  sh "./gradlew test"
  commentMr(env.PROJECT_PATH,env.MERGE_REQUST_IID,"Test ok in $MERGE_REQUEST_FROM_BRANCH =) ${env.BUILD_URL}")
 }
  
 stage('Regression test') {
  sh "echo regtest"
  commentMr(env.PROJECT_PATH,env.MERGE_REQUST_IID,"Regression test ok in $MERGE_REQUEST_FROM_BRANCH =) ${env.BUILD_URL}")
 }
  
 stage('Static code analysis') {
 sh "./gradlew check"
 step([
 $class: 'ViolationsToGitLabRecorder',
 config: [
  gitLabUrl: 'http://gitlab:80/',
  projectId: env.PROJECT_PATH,
  mergeRequestIid: env.MERGE_REQUST_IID,
  commentOnlyChangedContent: true,
  commentOnlyChangedContentContext: 0,
  commentOnlyChangedFiles: true,
  createSingleFileComments: true,
  createCommentWithAllSingleFileComments: true,
  minSeverity: 'INFO',
  maxNumberOfViolations: 99999,
 
  // You may want this when troubleshooting things
  enableLogging: true,
 
  // Only specify proxy if you need it
  proxyUri: '',
  proxyCredentialsId: '', // A username/password credential
 
  apiTokenCredentialsId: 'id', // A secret text credential
  apiTokenPrivate: true,
  authMethodHeader: true,
  ignoreCertificateErrors: true,
  keepOldComments: false,
  shouldSetWip: true,
 
  commentTemplate: """
 **Reporter**: {{violation.reporter}}{{#violation.rule}}
 
 **Rule**: {{violation.rule}}{{/violation.rule}}
 **Severity**: {{violation.severity}}
 **File**: {{violation.file}} L{{violation.startLine}}{{#violation.source}}
 
 **Source**: {{violation.source}}{{/violation.source}}
 
 {{violation.message}}
  """,
 
  violationConfigs: [
   [ pattern: '.*/checkstyle/.*\\.xml$', parser: 'CHECKSTYLE', reporter: 'Checkstyle' ],
   [ pattern: '.*/findbugs/.*\\.xml$', parser: 'FINDBUGS', reporter: 'Findbugs' ],
   [ pattern: '.*/pmd/.*\\.xml$', parser: 'PMD', reporter: 'PMD' ],
  ]
 ]
 ])
 }
}

Encoding

The commentTemplate parameter can be used to fix encoding problems, or just adjust what is being commented. See README in violation-comments-lib.

Plugin development

More details on Jenkins plugin development is available here.

There is a /build.sh that will perform a full build and test the plugin.

If you have release-permissions this is how you do a release:

mvn release:prepare release:perform