We announced the code insights feature as part of Bitbucket 5.15. However, this feature doesn't provide any insights itself - it is only an API to surface the insights of other tools. While there are some ready-made integrations available that can be found on the Atlassian Marketplace, it is also possible to create your own integration and run it as part of your normal build.
If you do not already understand how the code insights feature works, please first have a read of our how-to guide which explains what reports and annotations are, how they are displayed to a user on the pull request, and what kind of data can be displayed.
This tutorial will be using the JavaScript static analysis tool ESLint to detect code that doesn't match the JavaScript style guidelines as well as code that may result in errors. While of course the approach that this tutorial describes will work for any static analysis tool, we will assume you are working with a repository that already uses ESLint. If you are starting with an empty repository for this tutorial, then a quick way to get started with a default configuration is to run the following:
1 2npm init -f npm install eslint npx eslint --init
It will also assume that you have some JavaScript code in a folder called lib/.
While the general approach of this tutorial will work on any CI system, this tutorial will assume that you have a build configured in Bamboo which will have the repository checked out and ready to run the static analysis and post results back to Bitbucket Data Center. The tutorial will assume that Bamboo agent is Unix-based and has Curl, Git, Node and Python installed.
We will build a script which will run as a script task
in your Bamboo job. If you use a separate repository for your plan configuration then this is a good place to put this
script. Another good option is to put the script in the repository that will be analyzed. For this tutorial, the script
will live in the repository being analyzed, and will be called run_insights.sh. Create a script task in your Bamboo
job that runs this script.
run_insights.sh in your repository. Remember to make it executable.run_insights.shThe first step the script must perform is to run ESLint and generate a list of violations. Of course the command you run
for invoking ESLint may differ slightly to the one below, however for ease of parsing the output this tutorial assumes
you will use the --format=json option and output the violation to eslint.out.
You can run ESLint by executing the following
1 2npm install npx eslint --format=json -o eslint.out lib/**
run_insights.sh, run ESLint and send output to eslint.out in JSON form.Since parsing the output is a little more complicated, we will use Python instead of bash. Create a Python file containing the following code, or write your own code to convert the ESLint output into a report and annotation format.
This code goes through each error reported in the output file and creates an annotation on the line which the error
occurred. For the purposes of the tutorial, lets call this script parse.py
1 2import json, os severities = { 0: 'LOW', 1: 'MEDIUM', 2: 'HIGH' } with open('eslint.out') as eslint_output: # The error and warning counts are reported per-file, so lets aggregate them across files total_error_count = 0 total_warning_count = 0 annotations = [] for file in json.load(eslint_output): total_error_count += file['errorCount'] total_warning_count += file['warningCount'] # The path is absolute, but Bitbucket Data Center requires it to be relative to the git repository relativePath = file['filePath'].replace(os.getcwd() + '/', '') for message in file['messages']: annotations.append({ 'path': relativePath, 'line': message['line'], 'message': message['message'], 'severity': severities[message['severity']] }) with open('report.json', 'w') as report_file: report = { 'title': 'ESLint report', 'vendor': 'ESLint', 'logoUrl': 'https://eslint.org/img/logo.svg', 'data': [ { 'title': 'Error Count', 'value': total_error_count }, { 'title': 'Warning Count', 'value': total_warning_count } ] } # Write the report json to file json.dump(report, report_file) with open('annotations.json', 'w') as annotation_file: # Write the annotations json to file json.dump({'annotations': annotations}, annotation_file)
Note that the above script creates report.json for the report and annotations.json for the annotations.
These files will be used later when doing the REST call to create the report and annotations. For more details on what
can be included in the report and annotations JSON, see the REST documentation
or the how-to guide.
Run the Python script as the second step in run_insights.sh.
parse.py in your repositoryparse.py in run_insights.shOur script will use curl to create insights in Bitbucket Data Center, so we need the URL parameters for the endpoints
described in the REST documentation.
We will do this by creating bash variables in create_insights.sh.
We need variables for
When viewing a repository in the browser, the URL will be in the form <base_url>/projects/<PROJECT_KEY>/repos/<repo_slug>/....
If you would like, you can hard code these variables in the script.
1 2BBS_URL="http://url.to.bitbucket.server.here" BBS_PROJECT="MY_PROJECT" BBS_REPO="my_repo"
Another option is to set these variables as Bamboo environment variables.
The commit hash can be obtained by the script by running git rev-parse HEAD. The script should keep track of this value
in the form of a bash variable.
1 2COMMIT_ID=`git rev-parse HEAD`
The report key is a string that represents the analysis that was done. It should be a unique string chosen by the integration and must not clash with report keys from other integrations. We recommend using reverse DNS namespacing or a similar standard to ensure that collision is avoided.
For the purposes of this tutorial, we will use the report key my.example.eslint.report:
1 2REPORT_KEY="my.example.eslint.report"
Set the following variables in run_insights.sh:
BBS_URLBBS_PROJECTBBS_REPOCOMMIT_IDREPORT_KEYNow that the report has been parsed and converted into the format required by Bitbucket Data Center, we need to let Bamboo know the credentials for a user with repository read access. In order to create insights, the user performing the http REST call must be an authenticated Bitbucket Data Center user with Repository read permission. There are a number of ways to give Bamboo the information required to perform REST calls on behalf of a user. As with the last step, the credential data will be stored in Bamboo as environment variables.
We recommend creating one dedicated user in Bitbucket that can be used for all Bamboo calls. However, if this is not feasible then it is possible to use an existing user that has read access to the repository. Instead of using the user's password directly (which would pose a security risk), we recommend instead that you create a personal access token.
Create a personal access token and set it as a Bamboo environment variable. For the purpose of this tutorial, lets
call this variable token_password. In the script, this will be made available as an environmental variable called
bitbucket_token_password. With personal access tokens, we are able to perform a REST call without providing the username
using bearer authentication, so setting the username environment variable is not required.
bitbucket_token_password environment variable for the Bamboo jobTo create the report, do a PUT to the report endpoint: {baseUrl}/rest/insights/latest/projects/{projectKey}/repos/{repositorySlug}/commits/{commitId}/reports/{reportKey}.
The report body was created by parse.py and stored in report.json, and we have variables for the URL parameters
and the credentials.
1 2curl \ -H "Content-type: application/json" \ -H "Authorization: Bearer $bitbucket_token_password" \ -X PUT \ -d @report.json \ "$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY"
Note that if a different user has already created a report for this commit and report key then the request will be rejected. However if the existing report was created by the same user as this request then the existing report will be replaced by the new report. This can be useful in the cases of rerunning a build.
curl command in run_insights.sh to create a reportAfter creating the report, annotations can be added to the report. To add annotations to a report, do a POST to the
annotations endpoint for the given report:
{baseUrl}/rest/insights/latest/projects/{projectKey}/repos/{repositorySlug}/commits/{commitId}/reports/{reportKey}/annotations.
The annotations body was created by parse.py and stored in annotations.json, and we have variables for the URL
parameters and the credentials.
Note that if annotations already exist on the report, then posting additional annotations will not alter any existing
ones. In the case of rerunning a build, this could mean that duplicate annotations get created. Because of this, we
recommend deleting all the annotations for a report first by doing a DELETE to {baseUrl}/rest/insights/latest/{projectKey}/repos/{repositorySlug}/commits/{commitId}/reports/{key}/annotations
before creating the new annotations.
1 2# Delete old annotations from the report (they may not exist but it is better to be safe) curl \ -H "Authorization: Bearer $bitbucket_token_password" \ -H "X-Atlassian-Token: no-check" \ -X DELETE \ "$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY/annotations" # Create the annotations curl \ -H "Content-type: application/json" \ -H "Authorization: Bearer $bitbucket_token_password" \ -X POST \ -d @annotations.json \ "$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY/annotations"
curl command in run_insights.sh to add annotations to the reportBy this stage you should have completed the following:
run_insights.sh to store the script commandsparse.py, to convert the output of the static analysis tool into the format required by BitbucketBBS_URL, BBS_PROJECT, BBS_REPO, COMMIT_ID, REPORT_KEYtoken_password)If the above steps are complete, then your script, run_insights.sh, should look something like this:
1 2set -e # Make the Bamboo job fail if one of the commands fails # bitbucket_token_password needs to be set as a Bamboo environment variable. # If testing this locally, set the value manually #bitbucket_token_password="OTUwNTIxNTY5MTQwOjdH3JClEHyutj59QTNseDNp2hzt" # Set up the variables needed for the URL BBS_URL="http://url.to.bitbucket.server.here" BBS_PROJECT="MY_PROJECT" BBS_REPO="my_repo" COMMIT_ID=`git rev-parse HEAD` REPORT_KEY="my.example.eslint.report" # Run the analysis and parse the output echo "Running ESLint" npm install npx eslint --format=json -o eslint.out lib/** || true # Make sure that eslint doesn't make the Bamboo job fail echo "Done" echo "Parsing ESLint output" python parse.py # This will parse eslint.out and create report.json and annotations.json echo "Done" # Create the report or replace the existing one echo "Creating insight report" curl \ -H "Content-type: application/json" \ -H "Authorization: Bearer $bitbucket_token_password" \ -X PUT \ -d @report.json \ "$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY" echo "Done" # Delete old annotations from the report (they may not exist but it is better to be safe) echo "Deleting any existing annotations" curl \ -H "Authorization: Bearer $bitbucket_token_password" \ -H "X-Atlassian-Token: no-check" \ -X DELETE \ "$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY/annotations" echo "Done" # Create the annotations echo "Adding annotations to report" curl \ -H "Content-type: application/json" \ -H "Authorization: Bearer $bitbucket_token_password" \ -X POST \ -d @annotations.json \ "$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY/annotations" echo "Done"
And you're done! Commit this script to your repository and ensure that Bamboo is configured correctly to run it as a script task in your normal pull request build. You can view the code insights on a pull request.
Rate this page: