YAML Templates — making pipelines smarter

Introduction

Why you should be using YAML templates? To make your life easier.
When it comes to creating YAML pipelines for either infrastructure as code (IAC), testers or developers, it’s always beneficial for everyone to make things as easy as possible by requiring less overhead of work on repetitive tasks. This is where templating comes to play. Below we will look at an example of this before templates and after.

Current test YAML

Here is how our current unit test YAML looks:

pool:
 name: selfHosted001
steps:
- task: NuGetToolInstaller@1
 displayName: 'Use NuGet'
 
- task: NuGetCommand@2
 displayName: 'NuGet restore'
 inputs:
 restoreSolution: austin.sln
 vstsFeed: 'xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx'
- task: MSBuild@1
 displayName: 'Build unit Tests'
 inputs:
 solution: austin/my.test.files.locations.csproj
 configuration: Release
 msbuildArguments: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"'
 clean: true
- task: VSTest@2
 inputs:
 testSelector: 'testAssemblies'
 testAssemblyVer2: |
 **\*test*.dll
 !**\*TestAdapter.dll
 !**\obj\**
 searchFolder: '$(System.DefaultWorkingDirectory)'

In this example you can see a standard YAML build where it calls the required tasks, adds any custom arguments along with any custom versions and where to build from, in our case a self-hosted agent. We had 13 of these builds required and growing, each had different unit test location and some tweaks on arguments, but the majority was the same.

Templates allow us to:

• Reuse YAML
• Avoid replicating.
• Reduce complexity of pipelines.
• Allows others to create own pipelines referencing our templates.
• Amend one thing or many in one place.

pool:
 name: selfHosted001
 
extends:
 template: /pipelines/unitTestTemplate.yml
 parameters:
 projectUnderTest: austin

Simple and easy. State your template location and parameters, which in our case was the project folder containing the test file.
Here is how our template looks:

parameters:
- name: projectUnderTest
 type: string
variables:
assemblyUnderTest: ${{ parameters.projectUnderTest }}
UnitTests.Project.Name: ${{ parameters.projectUnderTest }}
UnitTests.Directory: $(Build.SourcesDirectory)/$(UnitTests.Project.Name)
UnitTests.Project.Path: $(UnitTests.Directory)/$(UnitTests.Project.Name).csproj
steps:
- task: NuGetToolInstaller@1
 displayName: 'Use NuGet'
- task: NuGetCommand@2
 displayName: 'NuGet restore'
 inputs:
 restoreSolution: Austin.sln
 vstsFeed: 'xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx'
- task: MSBuild@1
 displayName: 'Build $(UnitTests.Project.Path) Test'
 inputs:
 solution: $(UnitTests.Project.Path)
 configuration: Release
 msbuildArguments: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"'
 clean: true
- task: VSTest@2
 inputs:
 testSelector: 'testAssemblies'
 testAssemblyVer2: |
 **\*test*.dll
 !**\*TestAdapter.dll
 !**\obj\**
 searchFolder: '$(System.DefaultWorkingDirectory)'

Here you can see how we have shifted our repetitive tasks for the 13 builds to one location and then have it called per each build. The only thing we require then is one parameter to be sent back to the template, projectUnderTest, which in our case is based on the folder name of the test within the repository.
A second huge benefit of this is getting the developers’ involvement as it grows. As we have this set up for them now, when new testing or projects come in, the pipelines are easily set up where they can clone a previous YAML build file into the new test location and rename it to suit the folder structure. This will still call the same template source for them and produce the results they want, it also lets them play with various triggers if some units tests are linked. Removing the need to run every unit test (takes approx 60mins) down to a fraction of that time.

Where else can this be useful?

If you take Terraform pipelines, a structured terraform CI/CD could have the following:
1. Terraform pull request validate.
2. Security checks.
3. Terraform validation.
4. Terraform plan.
5. Terraform apply (pending environment dev/uat/pp/prod).

Putting all this in one YAML file will create a long list of tasks, especially when deploying to various environments.
You can easily create a template for each of them tasks and have one pipeline calling them each to do a series of checks and deployments. You just need to use this if pipeline based:

extends:
 template: /templatelocation.yml

or if stage based:

Stages:
- template: /templatelocation.yml

Anything else we can add on top?

Have you considered pull request checks to run before someone can approve your code?
Traditionally, you would have your checks done locally with either unit tests for a developer or terraform validate for IAC. You might then have your pull request done and either one big test build to run across everything or a manual test to check something specific.
Once you have a pull request created and sent to a colleague for peer review, you could have your checks done before your colleague gets to the pull request. Which would save them time and take out any risks of something being overlooked.

With Azure DevOps, that can be done in the Build validation section of branch policies. Here you can:

• Set a policy to trigger a pipeline reflective of project test name required.
• Configure a path filter to only run when a specific location is amended in the code.
• Make it a requirement to run.

Conclusion

Pipelines have come on leaps and bounds the last number of years. Where people still would use classic build creator in Azure DevOps, YAML pipelines is the best solution which creating templates to build the code, test the code to deploy the code.