Enterprise Jenkins on Azure, Part 1

I’ve been trying out Enterprise Jenkins on Azure. The inspiration was a webinar, Achieving Continuous Delivery on Microsoft Azure, about the “four quadrants of DevOps maturity”, and how to address the needs of different teams in the delivery pipeline using tools from Azure and Enterprise Jenkins. The webinar includes a demo of a cloud-based Jenkins Pipeline job, and the source code is freely available.

I tried out the demo and got it working with minor changes and a few workarounds. This post is part 1 of a series:

Part 1: Overview and Deploy Azure Resources
Part 2: Deploy Jenkins Operations Center, Master and Shared Agent
Part 3: The Jenkinsfile, tweaks and tips

 

Demo on Azure

The Pipeline job builds software that you’ve probably seen before: Conway’s Game of Life. The implementation is a Java web app, hosted in a Jetty web container. Pipeline lets you structure a job in stages; this one has four:

1. Get code from version control into a Jenkins build workspace.
2. Build the code, run automated acceptance tests.
3. Deploy the app to an application server.
4. Verify that the app was deployed correctly.

During or after a build, the job’s status page shows the stage view, a graphic of build history that links to commits and logs for a given build. The screenshot below shows that build #17 succeeded after some failed builds:

 

Stages are defined by Groovy code in a Jenkinsfile. Both web app source code and Jenkinsfile are in a GitHub repository. Note the branch: azure-pipeline. I forked the repository and made minor changes to Jenkinsfile (I’ll describe the changes in a future post).

To implement the demo you’ll need the following:

• Locally installed: Git, Bash
• A GitHub account
• An Azure subscription with 10 or more cores available

The diagram and notes below describe relationships between local and cloud-hosted resources:

• Laptop: develop code, push code to GitHub, manage Azure resources.
• Azure:
  • CloudBees Jenkins Operations Center (CJOC) to manage a client master (CJE) and agent. CJOC leases the agent to the master on demand.
  • CloudBees Jenkins Enterprise (CJE), or client master: on receiving a request from GitHub, start a build job on the agent.
  • Agent: accept jobs from client master, get code from GitHub, build code, deploy build artifact to the App Server.
  • Azure App Service set up with Jetty web server to host web applications.
• GitHub to host a Git repository and trigger a webhook (on push changes) to call a web method on the Jenkins master.

 

Set up Azure Resources

The point of the demo is to implement both CI and CD on cloud infrastructure which provides essential functions for modern enterprise-level DevOps, including isolation for networking security, and scalability.

To start, I prepared an SSH key which is required when creating an Azure VM resource. I used the Azure Portal to create some resources:

• Linux VM, template for provisioning CloudBees Jenkins Operations Center (CJOC).
• Linux VM, template for provisioning CloudBees Jenkins Enterprise (CJE).
• Linux VM, Ubuntu 14.04: to use as Jenkins build agent.
• Azure App Service, template for provisioning a Jetty web container.

Except for the App Service (see below), the resources were trivial to create. For CJOC and CJE, Azure templates install the Java runtime and Jenkins, configure firewall, connect CJE to CJOC.

Azure App Service

Azure offers some flexibility for deploying a web application:

• IaaS: Azure VM resource. Allows for maximum control but you must do all  setup, like installing an HTTP application server and other dependencies.
• PaaS: Azure App Service resource. Allows for simplicity at the cost of less control. There are several options for this, each with its own trade-offs.

To focus on Jenkins I chose the simplest PaaS option: from the Azure Marketplace, a template to create and configure a Java web app server. There are several to choose from, including Tomcat and Jetty (Game of Life requires Jetty). The Portal has a series of screens to create a template.

For the Web app property I chose the name gboys-gameoflife; as a result I have the publicly-accessible web app server URL below:

http://gboys-gameoflife.azurewebsites.net

For Jetty this page shows a few Java properties, including the file system location of the web app server (property jetty.base in the screenshot below). We need that location in order to deploy web apps.

The source code produces a Java web archive named gameoflife.war. When deployed, it should be accessible at the URL below:

http://gboys-gameoflife.azurewebsites.net/gameoflife/

Having created the App Service resource, I found Deployment credentials on its Portal blade and set a username and password:

At this point the web app server is ready to host web applications. In the screenshot below, the Portal blade’s Properties item shows values that determine the deployment location for applications. These values will be used in the Jenkins job.

For FTPS deployment, we need to build a URL string. From Properties on the App Service we get the FTPS host  name:

ftps://waws-prod-sg1-021.ftp.azurewebsites.windows.net

Some developer tools are available on the Portal blade. The Console tool is not wonderful – copy/paste is unsupported (at least on Chrome) – but it can be a handy alternative to FTP and other commands from a local Bash terminal. Using the Console I set the directory to the server’s webapps directory:

D:\home\site\wwwroot\bin\jetty-distribution-9.1.2.v20140210\webapps

From there, the pwd command converts to a UNIX path:

/d/home/site/wwwroot/bin/jetty-distribution-9.1.2.v20140210/webapps

Extract the path relative to the home directory:

/site/wwwroot/bin/jetty-distribution-9.1.2.v20140210/webapps

For FTPS authentication the URL must include an authority part (deployment credentials) in this form:

username:password@

Here, username must be the fully-qualified FTP/deployment user:

gboys-gameoflife/gboy

Finally, concatenate the parts below:

ftps://
gboys-gameoflife/gboy:********@
waws-prod-sg1-021.ftp.azurewebsites.windows.net
/site/wwwroot/bin/jetty-distribution-9.1.2.v20140210/webapps

The Jenkins job will build this URL.

That’s all for this post. In the next part I’ll set up a Jenkins environment on three VMs.