Continuous integration for iOS app with Cucumber, Frank and Bamboo

It's been a while since I had written latest post and it's just because I have too little time and too little readers, so I wasn't sure it worthies spending time. But after this post by Marlena Compton I've decided to keep it on.

So, this post is about building continuous integration for iOS application with Cucumber + Frank as acceptance testing framework and Bamboo as CI server.

Before you read everything else, I want to warn you that after a month of running CI, I come to the conclusion that Frank may be not the best solution for UAT, but there was nothing comparable at the time I've started. Now, however, you can try Ruby bindings for NativeDriver which were written by Jari Bakken (if anybody doesn't know, he's an awesome author of Ruby bindings for Selenium WebDriver and Watir-WebDriver) and me.

Regardless your choice, here is the flow which should be catered:

1. Developer pushes some changes to develop branch of application repository.
2. Bamboo fetches the changes and apply them.
3. Application is built with Xcode.
4. Acceptance tests are run on new build.
5. Each test produces a video with itself running.
6. The results of the tests are parsed and if there are failures, developer receive email notification on that.

Step one - frankified application

Proceeding from assumption you already have Xcode project of your iOS application the following steps are required

Make custom "frankified" target of your application which will be built with Frank code. It's described good at Frank tutorial while it's rather nontrivial (especially for people like me which have never worked with Xcode and even MacOS X). You may get some errors even after following the tutorial step-by-step, but don't give up.

Step two - writing Frank tests

When you have created "frankified" target, try writing some basic Frank tests for your app and see if it's running. Tutorial has everything necessary for this step.

Step three - getting MacOS-powered hosting

In my case, Bamboo was installed and running on Linux server and iOS app wouldn't build on it, so we had to make all this stuff run on separate Mac server. We've selected http://www.macminicloud.net/. Nice thing was that it has MacOS X 10.7 Lion and you're provided with both SSH and VNC access.

Step four - adding MacMini as remote agent to Bamboo

The tests should be run on MacMini, but the results and all this stuff should be shown in Bamboo itself which is on Linux server. So, you have to create a "remote agent" for Bamboo.

1. Make sure your Bamboo license allows it. Go to Administration → License Details and look for "Number of remote agents supported". Unless it's 0, you can go on. Otherwise, make sure you upgrade your license.
2. Go to Administration → Agents and click "Install Remote Agent".
3. Copy the "Running a Remote Agent" command, you'll need it.
4. There will be "Remote agent JAR" for downloading. Get it.
5. Connect to MacMini and place downloaded JAR to home folder on MacMini.
6. Run it with "Running a Remote Agent" command.
7. Wait until it appears on Agents page in Bamboo.
8. When it does, open it by clicking on its name.
9. In "Agent-Specific Capabilities" click to "Add Custom" and create new custom capability like isMacMini key and true value. This is needed to ensure the plan is built on MacMini remote agent and not on local agent.

Step five - create your iOS app plan

Create simple plan just like you usually do and job for it. The only difference is to create requirment for MacMini. Go to job configuration and Requirements, then select "isMacMini" from Requirement list and click "Add". It will ensure that job is running on MacMini.

N.B. If you have any different plans which should be run on local agent, add custom capability to it (e.g. isLocal = true) and add it as requirement to your plans' jobs. Otherwise, the jobs may run on MacMini too.

Step six - configure plan's tasks

I have two tasks in job: one for running tests which call simple Bash script (Command) and one for parsing test results (JUnit parser). The following script builds application and run tests:

#!/bin/bash

# Compiles and runs Frank tests for iOS.
#
# Written by Alex Rodionov (p0deje@gmail.com)

# Xcode variables
target="myapp Frankified"
configuration="Debug"
sdk="iphonesimulator4.3"

# Path to compiled application
app_path="/Users/cloud/bamboo-agent-home/xml-data/build-dir/MOBILE-IOS-JOB1/iPhone/build/Debug-iphonesimulator/myapp Frankified.app"

# Compilation
echo 'Building application...'
rm -rf "$app_path" # Remove old build
cd iPhone/
xcodebuild -configuration $configuration -target "$target" -sdk $sdk

# Running test
echo 'Running Frank user-acceptance tests...'
cd Frank/
rm -rf test_results/ # Remove old results
# Remove old videos
rm -f /Users/cloud/Movies/*.mov
rm -rf videos/
# run tests
export APP_BUNDLE_PATH="$app_path"
cucumber -f pretty -f junit -o test_results/

# Copy videos
mkdir videos
cp /Users/cloud/Movies/*.mov videos/

This script works with recorded videos but you can remove this code if you don't record them in Frank tests.

(optional) Step seven - produce video of tests as artifcats

If your Frank tests record videos, you can publish them as artifacts of the build.
To record videos, copy start_recording and stop_recording methods to your env.rb from frank_helper.rb (or require it) and call first in Before hook and latest in After hook.
To publish videos as artifacts, go to job configuration, click "Artifacts" and add new one with Location of videos directory and Copy Pattern *.mov.

Step eight - have some beer and chill out, you've done great

What else?
It's still not finished yet. We also need to publish ipa file as an artifact and push it to www.testflight.com.