Continuous Deployment of BizTalk Applications using Visual Studio Team Services

Introduction

With the recent rollout of the public preview of Release vNext as part of Visual Studio Team Services (formerly Visual Studio Online), making use of Continuous Deployment as part of the development process for BizTalk applications should become normal practice for development teams.

The benefits range from rapidly exposing coding issues with the use of automated end-to-end system tests, through to problems with the deployment or configuration of the BizTalk applications that are being developed.

This post will provide a high-level overview of what is involved in getting up and running with a development process that takes full advantage of the features offered by Visual Studio Team Services (VSTS), and with the option of using Azure for hosting the IAAS environments required to fulfil this model.

Please note that this is a rapidly changing area – the information and screenshots contained within this post are correct as of mid-January 2016, but could soon change.

Overview

The requirements that we trying to meet are that for every check-in made to the repository a build is automatically triggered using VSTS Build vNext. This build will:

1. Get the source code from the repository at the changeset revision that was checked in. A Get Latest cannot be used in case multiple developers check in almost simultaneously. Batched changes are not what is wanted here.
2. The source code is compiled.
3. Unit tests are executed and results collated.
4. The BizTalk Deployment Framework (BTDF) installers are generated.
5. The build output is published.

If all these steps complete successfully then a Release vNext Release Definition will be automatically triggered that will:

1. Copy the necessary output from the build that has just completed to the “System Test” server.
2. Undeploy any existing deployment from a previous run, and then install the new version.
3. Run the end-to-end System Tests

Infrastructure Requirements

At least two servers will be needed to support these requirements:

• Build Server
• System Test Server

Additional servers can be added as more environments are needed or introduced. Typically these would all be AD joined servers, but you can also use standalone servers with local accounts and groups if needed. This does introduce an extra degree of complexity as you will need to add the build server to the WinRM service TrustedHosts list on all the servers you want to push deployments to. This is outside the scope of this post.

Build Server

The Hosted Build Agent provided with VSTS cannot be used for BizTalk builds as it does not have the necessary libraries installed, so you will need to stand up your own build server. This can be provisioned in Azure using the standard Windows Server 2012 R2 Datacenter image. Once up and running, you will need to install the following:

• Visual Studio 2013 with Update 5, and the following optional components:
  • Microsoft SQL Server Data Tools
  • Microsoft Web Developer Tools
• BizTalk Server 2013 R2 – not a full installation, just these components:
  • Developer Tools and SDK
  • Project Build Component
• Deployment Framework for BizTalk 6.0 Beta 1
• VSTS build agent – available from the Agent queues tab within your VSTS collection administration pages
  

Once downloaded, you will need to configure the agent to connect to your VSTS collection and link it to a new Queue that you will need to create. Instructions for this can be found here.

System Test Server

This server will be used for deploying the built BizTalk applications to, and so needs to be a fully configured BizTalk server. The actual version of software that will need to be installed on here will vary depending on your requirements, but the following serves as an example:

• SQL Server 2014
• BizTalk Server 2013 R2

Once installed, BizTalk Server will need to be fully configured, ready to have the BizTalk Applications deployed.

Finally the System Test server will need to have PowerShell remoting enabled and the WinRM port configured for HTTP communication.

To enable PS remoting, run this command in an Administator PowerShell session: “Enable-PSRemoting –Force”
To configure WinRM for HTTP communication, run this command in Administrator PowerShell session: “winrm quickconfig -transport:http”

Build

Now that the necessary infrastructure is in place the build definition can be configured. This will be a new vNext build definition type (rather than the older XAML definition) and is configured through the Build tab of your VSTS Team Project.

Create a new Build Definition

To create a new vNext build definition select the Build tab, then click the green + symbol. Choose the Empty definition then click Next and complete the configuration as required.

Triggers

In order for the build to be triggered automatically on every check in click the Triggers tab and ensure the “Continuous integration” box is checked, and that the “Batch changes” box is unchecked, as shown here.

Build steps

Return to the Build tab and configure the build steps that you want to execute as part of the build. Once configured, you should end up with a build definition with steps that look similar to this:

Running through these steps in turn we have:

1. Updating versioning. This modifies all the AssemblyInfo.cs files and BTDF project files to patch in the build number so as the artefacts and installers can be traced back to a specific build and changeset. The PowerShell script this executed here is based upon the UpdateVersionInfo.ps1 script from the BizTalk ALM CodePlex site.
2. Build solutions. In this demo this just wildcard matches “**\*.sln” files. As your project becomes more complex you may need to have multiple MSBuild steps and explicitly detail which solution files to build, and in which order.
3. Build BTDF Packages. This again is a wildcard match, but this time for “**\*.btdfproj”. By the time this runs all your artefacts should have been built by the previous step, so the order these get built in should not matter.
4. Run Unit Tests. This runs all the tests that do NOT require the BizTalk applications to be deployed. This relies on TestCategories being defined when declaring your tests so as specific test categories can be excluded as shown here:
   
5. Create Deployment Package. This calls the PowerShell script CreateDeployment.ps1, which again is based upon the script available from the BizTalk ALM CodePlex site. This produces a zip file containing all the MSIs that have been generated by the previous step, along with PowerShell scripts that will be used to automatically deploy these packages.
6. Publish Artifacts: Binaries. This copies all the build artifacts to a drop location to be used (if required) by automated tests, or for developers for diagnosing issues.
7. Publish Artifacts: Package. This copies the deployment package generated above to the build drop location, ready to be picked up by the Release Management process to deploy through the environments.

At the end of a successful build we have a package like this in our drop location.

On the build summary page you will also get a breakdown of the build steps that have been executed, along with the test results for the unit tests that have been executed as part of the build.

Release

Now that we have something to deploy we can configure the Release Definition. In this case I want the Release to my System Test environment to be automatically triggered when the Dev.ContinuousBuild completes successfully.

All the configuration for the Release Definitions is done within the new Release tab of your VSTS Team Project, which is still in Public Preview.

Create new Release Definition

First you need to start by creating a new empty Release Definition. Select the Release tab, then click the green + symbol. I’ve called mine Dev.ContinuousDeploy – as it will be a continuous deploy of what has been built from my Dev branch.

Rename the Default Environment that has been created for you to “SystemTest”.

Configure Agent Options

Click the ellipses within the SystemTest environment box, and choose “Agent Options”.

Change the Default queue to use the new queue that you configured earlier. The agent running on this server will need to have access to your build drop location, and all the environment servers that you want to deploy to.

Configure linked build and artifacts

Select the Artifacts tab and click “Link to a build definition”.

Unless you have created more than one build definition you should be able to leave the options as default in here and click Link.

The Dev.Continuous Build should now be listed as a Source alias. This means that you will be able to pull in artefacts published by this build into your Release tasks.

Configure Triggers

Choose the Triggers tab option and check the box for “Continuous deployment”.  Set the trigger on artefact source to be the build definition name you have just linked to in Artifacts. Then select the SystemTest environment to set this as the environment that will be automatically deployed to.

Configure Environment Tasks

Return to the Environments tab, ensure the SystemTest environment is selected and add the tasks that you want to be executed in this environment, similar to the following:

Running through these steps in turn we have:

1. Copy Package artefact to SystemTest. This copies the artefact called “Package” that was generated as part of the Build definition to our SystemTest server.
2. Copy Binaries artefact to SystemTest. This copies the artefact called “Binaries” to our SystemTest server, which will be used for the test run.
3. Deploy BizTalk Applications on to System Test. This executes a script which is based upon the AutoDeploy.ps1 script from the BizTalk ALM CodePlex site.
4. Deploy TestAgent on System Test. This installs the “Agent for Microsoft Visual Studio 2015” which is required on the System Test server in order to execute the System Tests.
5. Run SystemTests. This runs all the tests that have a TestCategory of “SystemTest” that are in the Binaries artefact.

At the end of a successful Release the Release summary page will show the System Test results.

In addition, after a short period of time, these System Test results will also be recognised by the original build that triggered the release. The following screenshot shows the same build summary page as shown above, except it is now showing 9 test passes instead of 4.

Next Steps

Now that you have the initial steps working you can start to add more environments into your Deployment Pipeline, to include Test, UAT, even all the way through to Production. Deployments don’t have to be fully automated, you can choose to define environment owners who have to approve releases before they are allowed to progress.

You are likely to find that you will need to introduce more agents to be able to handle the build and release requests that will be generated as the size of your team, project and environments grow. In the example we have shown here there is only one build agent, so whilst a Release is being deployed all Build requests will be queued, and vice versa.

More changes are on the way for Release Management.  Keep an eye on the Release notes for more information on what’s new, and what’s coming.