Jenkins for Pull Requests and Promoted Builds Deploying to Multiple Environments

This is a step-by-step to setup Jenkins to build, test and report status on Github pull requests and to deploy builds of a production branch, usually master, to multiple environments.

This is a continuation of the previous post which went over the setup of a basic Jenkins server deployed on AWS with Github authentication.

Make sure you have the following plugins installed (Manage Jenkins > Manage Plugins):

  • Github Authentication Plugin
  • GitHub Plugin
  • GitHub Pull Request Builder
  • Copy Artifact Plugin
  • Notification Plugin
  • Parameterized Trigger Plugin
  • Promoted Builds Plugin
  • SBT Plugin (if building Scala projects)

In this setup, we'll be creating four jobs:

  • Master Branch Job that will be building master (or whatever the production branch is) with deployment.
  • Pull Requests Job will be doing the same for Pull Requests.
  • Deploy to Staging Job
  • Deploy to Production Job

I like to have different jobs for master and pull requests to make sure build history is clean, that's specially important on master. Deployment will only be available for master builds, not pull request ones.

Here's a quick overview of the basic workflow:

  • Feature branches are created whenever changes are needed on the repository. I personally like to name branches $developername-$featurename (for example: felipe-circuitbreaker).
  • Once the work is done and the pull request is created, Pull Requests Job triggers a build, running all the unit and integration tests. Pull Request Builder Plugin reports the build status on the pull request, pretty neat feature.
  • Pull Requests Job might trigger multiple builds, every time there's a new commit to the pull request, since the developer will be fixing bugs, addressing code review comments, etc.
  • Once the code is reviewed, PR comments addressed and build is executed successfully, all the commits from the feature branch are squashed and merged to master. Squashing to a single commit helps keeping the repository's commit history clean.
  • The Master Branch Job then triggers a build for that commit.
  • Once a build is ready to be deployed, you should go to the build page and click on Promotion Status. There will be two promotion types available, Deploy to Staging and Deploy to Production, hit Approve or Force promotion.

Ok it's time to create the jobs!


Master Branch Job Setup

Enter basic properties for the job:

Name: Master Branch Job  
GitHub project: <url to project on github>  
Permission to Copy Artifact: checked  
Projects to allow copy artifacts: Deploy to Staging, Deploy to Production  
Source Code Management: Git  
Repository URL: git@github.com:company-name/project-name.git  
Branch Specifier (blank for 'any'): */master  
Build when a change is pushed to GitHub: checked  

Build

This is what tells Jenkins how to build and test the project. In the case of Scala projects, it calls SBT.

Post-build Actions

Three post-build actions are needed: Archive the Artifacts, E-mail Notification and Send build status to Github on commit.

Under Archive the Artifacts, make sure to enter the list of files needed by the deployment jobs. Normally it's contains the application package (like a zip or tar.gz file) and scripts needed to be executed during deployment.

On E-mail Notification, enter the list of emails that need to be notified when the build breaks.

Build Promotion

First of all, enable Promote builds when...

Two promotion processes are needed, one to call Deploy to Staging Job and another to call Deploy to Production Job.

Here's the basic information you need on each promotion step:

Name: Deploy to [Staging|Production]  
Only when manually approved: checked  

Then an action, Trigger parametized build on other projects, needs to be added for each promotion process. Look for the Add action button. The action is what actually triggers the deployment jobs.

Here's the information needed for each action:

Projects to build: Deploy to [Staging|Production]

Predefined Parameters:  
job=${PROMOTED_JOB_NAME}  
build_id=${PROMOTED_NUMBER}  


Pull Requests Job Setup

Enter basic properties for the job:

Name: Pull Requests Job  
GitHub project: <url to project on github>  
Build when a change is pushed to GitHub: checked  

Please enable This build is parameterized and add parameter with name sha1 and default value master.

Source Code Management

  • Select Git and then hit Advanced.
  • Please fill out Repository URL which it should be pretty standard, something like git@github.com:company-name/project-name.git.
  • Refspec should be +refs/pull/*:refs/remotes/origin/pr/* to make sure only pull requests are built on this job.
  • Branch Specifier should be ${sha1}.

Build Promotion

No promotion steps or artifact copy is needed because there will be no deployment from this job, only from Master Branch Job.

Build Triggers

  • Make sure to enable GitHub Pull Request Builder and then hit the Advanced button.
  • Please enter the list of all Github usernames that are allowed to trigger a build from a pull request on Admin List field. If pull requests from everyone on the team should trigger builds, enter the organization name on List of organizations. Their members will be whitelisted. and enable Allow members of whitelisted organisations as admins.
  • Enable Use github hooks for build triggering.

Build

It looks just like the build section on the Master Branch Job.

Post-build Actions

You should add Set build status on GitHub commit so Jenkins pushes the build status to the pull request on GitHub.


Deploy to [Staging|Production] Job

  • Enter Deploy to Staging or Deploy to Production under Project Name.
  • On Source Code Management, pick None.

Build

Look for add build step button and define two build steps:

  • Copy artifacts from another projects which will make the files created by Master Branch Job available to the deployment job.
    • Under Project Name, you should enter ${job} which was defined as a pre-defined parameter in Master Branch Job.
    • On Artifacts to copy, enter whatever value you defined on Archive the Artifacts on Master Branch Job.

  • Execute shell which will execute shell command(s) to do the actual deployment.


GitHub Webkhooks (Github Project > Settings > Webhooks & Services)

On the Github project, you need to make sure the service and webhook are created and active under the project settings. That's what will be sending push notifications to Jenkins that will trigger builds when appropriate.

Jenkins (GitHub plugin) Service

  • First create the service

  • Fill out Jenkins hook url and activate it.

Jenkins Pull Request Builder Webhook

  • Payload URL should be something like http://jenkins.mydomain.com/ghprbhook/.
  • Content type should be application/x-www-form-urlencoded.
  • Make sure events Pull Request and Issue comment are enabled.
  • Activate it.