In Azure DevOps, it’s possible to configure a pipeline to run every time a Pull Request is created. You can make it a requirement so that a Pull Request can only be completed, when the pipeline runs without any errors. This is a great feature, and you can use that pipeline to build and validate all the relevant Source files and ARM / Bicep / Terraform templates.
Using such a pipeline ensures that nothing is committed to your main or develop branch that would otherwise break your Build or Deployment pipelines somewhere. Approvers can also see these errors and know that they don’t have to check the Pull Request, until these issues are resolved.
However, this isn’t the case with warnings. Warnings help maintain code standards and adhere to best practices. While these warnings are visible in the Pipeline run details, there is no way to see them directly in the Pull Request or prevent the Pull Request from being completed. These warnings can help the approvers of the Pull Requests to detect these issues or can, at least, start a discussion.
Of course, most build, linting or validation tools allow you to configure the severity of the specific warnings and treat them as an error. This would solve these problems but would also take away the possibility to purposely disregard these warnings, because they currently cannot be solved, or would take too much time.
The solution is to add comments to the Pull Request. For each individual warning a comment will be added to the Pull Request. This allows for a discussion per warning, if needed, and the comment can still be resolved, allowing the Pull Request to be completed, if necessary, despite these warnings.
I found some examples online, but none of them were complete, or even correct, because of changed property and variable names, so I figured it could be helpful to write a blog about this, describing all the necessary steps:
Step 1: Create a YAML file
In your project create a YAML file, with at least 1 stage, that builds, tests and validates your source code. Don’t forget to also include the deployment templates (ARM / Bicep / Terraform), when the related tools have the possibility to validate them. Also make sure the pipeline isn’t triggered by anything else by setting the ‘trigger: none’ at the top of the file.
Then add the following code:
stages:
- stage: 'Validate'
... Your build code
- stage: 'Check'
displayName: 'Check for warnings'
dependsOn: [Validate]
condition: in(dependencies.Validate.result, 'Succeeded')
jobs:
- job: Build
workspace:
clean: all
pool:'YourAgentPool'
displayName: 'Check for Warnings'
dependsOn: []
steps:
- task: AzureCLI@2
displayName: 'Add comments for warnings'
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
inputs:
scriptType: pscore
azureSubscription: 'YourServiceConnection'
scriptLocation: inlineScript
inlineScript: |
$orgUri = $env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI
$projectName = $env:SYSTEM_TEAMPROJECTID
$repositoryName = $env:BUILD_REPOSITORY_NAME
$pullRequestId = $env:SYSTEM_PULLREQUEST_PULLREQUESTID
$token = $env:SYSTEM_ACCESSTOKEN
$buildId = $env:BUILD_BUILDID
$v = "?api-version=6.0"
$authHeader = @{Authorization = "Bearer $token" }
$buildUrl = "$($orgUri)$($projectName)/_apis/build/builds/$($buildId)/timeline$($v)"
$apiUrl = "$($orgUri)$($projectName)/_apis/git/repositories/$($repositoryName)/pullRequests/$($pullRequestId)/threads$($v)"
$response = (Invoke-RestMethod -Uri $buildUrl -Headers $authHeader -Method GET -ContentType application/json)
foreach ($rec in $response.records){
if (-not $null -eq $rec.issues)
{
foreach ($iss in $rec.issues){
if ($iss.type -eq "warning")
{
$body = @{
comments = @(
@{
parentCommentId = 0
content = "Warning: $($iss.message)"
commentType = 1
}
)
status = 1
}
$bodyJson = $body | ConvertTo-Json
Invoke-RestMethod -Uri $apiUrl -Method Post -Headers $authHeader -Body $bodyJson -ContentType "application/json"
}
}
}
}
Line 2
This is where you can put your code. This can be any number of stages, but we assume it’s a single stage in this scenario.
Line 6 + 7
This indicates this stage will only run if the former stage(s) have run without errors. If you have multiple stages, you will need to adjust here. Because the code above checks the timeline of the build rather than individual build results, and stages and jobs can run in parallel in Azure DevOps, you need to make sure the code runs only after all the Build stages are finished.
Line 20
Most variables related to your build and pull requests, are automatically available in your agents as environment variables. This is not the case with the access token. The access token allows you to run code on DevOps and Azure on behalf of the Service connection. It’s available in DevOps in the variable $(System.AccessToken), but you must explicitly request it in your pipeline to gain access to it. Here we create an Environment variable SYSTEM_ACCESSTOKEN, that will now be available in the PowerShell script.
Line 22
I have chosen PowerShell Core here. You can also do this in bash, or any other type that is supported by the @AzureCli task, but it’s a matter of preference.
Line 26 – 31
These env:* variables, with the exception of the SYSTEM_ACCESSTOKEN that we supplied in line 20, are automatically available in every script you run from the pipeline. To improve readability, the code is organized into separate variables with descriptive names.
Line 34
This token is sent in the Authorization header of every request to the DevOps REST API,
Line 36
This is the URL for the Build timeline REST API. It provides the same information used by the DevOps portal to display the progress of your pipeline. We use this to extract the warnings.
Line 37
This is the URL of Pull Request Thread Comments REST API. This is used to post comments to the Pull Requests.
Line 46
Here, the body of the message we want to send to the REST API is created. Most properties can be used as they are, but the status property indicates the status of the comment: 1 is active and 3 is resolved. This format isn’t in the documentation, but it works, and also allows to set a status from the start.
Commit your changes so Azure Devops can see them.
Step 2: Create DevOps Pipeline
In Azure DevOps create a Pipeline and select the yaml file(s) you created in step 1.
Step 3: configure your branch(es)
Pull request policies don’t exist. Instead, you set the branch policies of the branches that require a Pull Request. Select the respective branch, click the ellipses (…) at the end of the row and select ‘Branch policies’. Make sure the ‘Requires a minimum number of reviewers’ is selected, to ensure a Pull Request is needed for this branch.
Scroll down to the ‘Build Validation’ section and add a new one or edit the existing one. Make sure your settings are configured as shown below. The build pipeline is the pipeline, you created in step 2.
This pipeline will now run every time a Pull Request is created.
Step 4: Add permissions
By default Azure Devops agents can only read your repositories. As a result you are not allowed to post comments on a Pull Request. To fix this, follow these steps:
- Go to Project Settings and Repositories
- Select the Repository you’d like to configure permissions for
- Go to Security
- If Azure Pipelines is set up, you should see two standard users:
- The admin who set up the organisation
- A build service user named ‘[PROJECT NAME] Build Service ([ORGANISATION NAME]))
- Select the Build Service user to modify its permissions. Set Contribute to pull requests to Allow.
That’s it. If everything is configured correctly, you should now see comments appear on your Pull Request when the pipeline has warnings.