Unity3D and Automated Builds II

In the previous post, we automated the build process for Unity3D game development using Jenkins.
Here we built and packaged game code to streamline the deployment process to multiple devices.

While this approach works well, it is somewhat inflexible: whatever source code that is checked-in, it is built verbatim. That is, it is not possible to configure Jenkins build jobs to target different environments.

Also, during development, we may like to produce Test builds internally plus build and distribute Beta builds externally using TestFlight. All of which can be done by extending the Automated Build process.

Let’s check it out!

Pre-requisites
Ensure you have all the pre-requisites installed and setup for Jenkins automation as per previous post.

Example
As an example, let’s extend Angry Bots and configure a Patch Server to target different environments:

TEST	Used for internal development and testing.
BETA	Used for external development and testing.
PROD	Used for the Live Production environment.

Jenkins
The existing Angry Bots build job executes PerformBuild CommandLineBuildOnCheckinIOS method as it is.
Now we would like to configure the execute method to accept arguments to target different environments.

Unfortunately, Jenkins does not allow arguments to be passed to the execute method; we must build this infrastructure ourselves. Here is a great code sample that demonstrates how to implement this manually.

In our example we would like to override the following information as configured by default in Unity3D:
Target build environment (Test / Beta / Prod), Build version number and finally the Bundle Identifier.

Note: it may be necessary to override the Bundle ID to distinguish Beta and Prod side-by-side builds.

Implementation
Add the following "CommandLineReader.cs" file to the "Editor" folder to parse command line arguments:

using System;
using System.Collections.Generic;
using System.Linq;
using UnityEngine;

public class CommandLineReader
{
 public static string GetCustomArgument(string argumentName)
 {
  var customArgsDict = GetCustomArguments();
  string argumentValue;

  if (customArgsDict.ContainsKey(argumentName))
  {
   argumentValue = customArgsDict[argumentName];
  }
  else
  {
   Debug.Log("GetCustomArgument() Can't retrieve any custom argument named [" + argumentName + "] in command line [" + GetCommandLine() + "].");
   argumentValue = String.Empty;
  }

  Debug.Log("CommandLineReader " + argumentName + "=\"" + argumentValue + "\"");
  return argumentValue;
 }

 //Config
 private const string CUSTOM_ARGS_PREFIX = "-CustomArgs:";
 private const char CUSTOM_ARGS_SEPARATOR = ';';

 private static string[] GetCommandLineArgs()
 {
  return Environment.GetCommandLineArgs();
 }

 private static string GetCommandLine()
 {
  string[] args = GetCommandLineArgs();
  if (args.Length > 0)
  {
   return string.Join(" ", args);
  }
  else
  {
   Debug.LogError("GetCommandLine() - Can't find any command line arguments!");
   return String.Empty;
  }
 }

 private static Dictionary<string, string> GetCustomArguments()
 {
  var customArgsDict = new Dictionary<string, string>();
  string[] commandLineArgs = GetCommandLineArgs();

  string customArgsStr;
  try
  {
   customArgsStr = commandLineArgs.SingleOrDefault(row => row.Contains(CUSTOM_ARGS_PREFIX));
   if (String.IsNullOrEmpty(customArgsStr))
   {
    return customArgsDict;
   }
  }
  catch (Exception e)
  {
   Debug.LogError("GetCustomArguments() - Can't retrieve any custom arguments in command line [" + commandLineArgs + "]. Exception: " + e);
   return customArgsDict;
  }

  customArgsStr = customArgsStr.Replace(CUSTOM_ARGS_PREFIX, String.Empty);
  string[] customArgs = customArgsStr.Split(CUSTOM_ARGS_SEPARATOR);

  foreach (string customArg in customArgs)
  {
   string[] customArgBuffer = customArg.Split('=');
   if (customArgBuffer.Length == 2)
   {
    customArgsDict.Add(customArgBuffer[0], customArgBuffer[1]);
   }
   else
   {
    Debug.LogWarning("GetCustomArguments() - The custom argument [" + customArg + "] seems to be malformed.");
   }
  }
  return customArgsDict;
 }
}

Update the existing "PerformBuild.cs" file in the "Editor" folder to read arguments passed in to the build:

using System;
using System.Collections.Generic;
using System.IO;
using UnityEditor;
using UnityEngine;
using Object = UnityEngine.Object;

public class PerformBuild
{
 [MenuItem("Automated/Automated iOS Build")]
 static void CommandLineBuildOnCheckinIOS()
 {
  const BuildTarget target = BuildTarget.iPhone;
  
  string[] levels = GetBuildScenes();
  const string locationPathName = "iOSbuild";
  const BuildOptions options = BuildOptions.None;

  DeleteStreamingAssets();
  BuildPipelineBuildAssetBundle(target);

  string environment = CommandLineReader.GetCustomArgument("Environment");
  string version = CommandLineReader.GetCustomArgument("Version");
  string bundleID = CommandLineReader.GetCustomArgument("BundleID");

  string bundleIdentifier = (0 == bundleID.Length) ? "com.studiosstevepro.angrybots" : bundleID;
  BuildPipelineBuildPlayer(levels, locationPathName, target, options, environment, version, bundleIdentifier);
 }

 // similar code as previous post.

 private static void BuildPipelineBuildPlayer(string[] levels, string locationPathName, BuildTarget target,
  BuildOptions options, string environment, string version, string bundleIdentifier)
 {  
  PlayerSettings.productName = "Angry Bots";
  PlayerSettings.bundleIdentifier = bundleIdentifier;

  string bundleVersion = (0 == version.Length) ? "1.0" : version;
  PlayerSettings.bundleVersion = bundleVersion;

  if (0 != environment.Length)
  {   
   string fullPath = Application.streamingAssetsPath + "/Environment.txt";
   File.WriteAllText(fullPath, environment);  
  }
  
  String error = BuildPipeline.BuildPlayer(levels, locationPathName, target, options);
  if (!String.IsNullOrEmpty(error))
  {
   throw new System.Exception("Build failed: " + error);
  }
 }
}

Finally, write Unity3D client code that consumes the Patch Server build environment variable accordingly:

public class Utilities
{ 
 internal static string PATCH_SERVER_ROOT
 {
  get
  {
   if (String.IsNullOrEmpty(_patchServerRootData))
   {
    SetBuildEnvironmentVariables();
   }

   return _patchServerRootData;
  }
 }

 private static void SetBuildEnvironmentVariables()
 {
  TargetEnviroment target = TargetEnviroment.Test;
  string environment = target.ToString();

  string fullPath = Application.streamingAssetsPath + "/Environment.txt";
  if (File.Exists(fullPath))
  {
   environment = File.ReadAllText(fullPath);
  }

  try
  {
   target = (TargetEnviroment) Enum.Parse(typeof (TargetEnviroment), environment, true);
  }
  catch
  {
   // Must do this was as Enum.TryParse not available in .NET 2.0.
  }

  switch (target)
  {
   case TargetEnviroment.Test:
    _patchServerRootData = "http://TestPatchServer/"; break;

   case TargetEnviroment.Beta:
    _patchServerRootData = "http://BetaPatchServer/"; break;

   case TargetEnviroment.Prod:
    _patchServerRootData = "http://ProdPatchServer/"; break;
  }
 }

 private static string _patchServerRootData;

 enum TargetEnviroment
 {
  Test, Beta, Prod
 }
}

From here, any reference to the Patch Server from the Unity3D client code can be made simply like this:

  string patchServerRoot = Utilities.PATCH_SERVER_ROOT;

Note: ensure Optimization Api Compatibility Level is updated to accommodate Enum.Parse();

Navigate to File menu | Build Settings | Platform | Switch Platform | Other Settings
Change Optimization Api Compatibility Level from .NET 2.0 Subset to .NET 2.0.

Build Job
Make one small change to the existing AngryBotsIOS build job: pass arguments to the executeMethod.

Unity3d installation name	Unity3D
Editor command line arguments:  -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinIOS  -CustomArgs:Environment=Beta;Version=2.0;BundleID=com.studiosstevepro.angrybotsbeta

Click Save button | Click Build Now. Deploy Angry Bots to iOS device as per previous instructions.
Congratulations! Angry Bots should be installed as before but now target Beta build environment.

TestFlight
TestFlight Beta Testing makes it easy to invite users to test pre-release versions of iOS apps before they are released to the App Store. Up to 1,000 beta testers can be invited using just their email addresses.

Here is a good tutorial on how to setup TestFlight and how to use TestFlight to distribute the beta app:
1. Create an account on TestFlight with "Developer" option checked.
2. Log in to the TestFlight web site and "Add a team" for your app.
3. Add teammates, either testers or developers or available public.
4. Ask teammates to register iOS devices following the instructions.
5. Once accepted / registered, device information will be available.
6. Add devices / UDIDs to Devices section of Apple web site portal.
7. Update provisioning profile and install onto Jenkins build server.

Upload API
Extend the Automated Build process to upload latest revision to TestFlight via Upload API.

API Token
API Token gives access to Upload API and any endpoints for future TestFlight releases.
In TestFlight, click User icon (top right) | Account Settings | Upload API | API Token.

Team Token
Team Token is the token used for the Upload API "Team" parameter (in Jenkins).
In TestFlight, click Team name (top right) | Edit Info | Team Info | Team Token.

Distribution List
Create distribution list to notify testers once latest revision upload to TestFlight:
In TestFlight, click People tab | click "+" icon next to "Distribution Lists" to add.

Apple
Add same device information to the Apple portal and include in Distribution provisioning profile.
Navigate to Apple portal | iOS Apps | Log In | Certificates, Identifiers & Profiles | Devices.
Click the "+" icon (top right) to Register new device(s): Enter a valid name and the UDID.

Once all the device(s) have been registered, include in the Distribution provisioning profile:
Provisioning Profiles | Distribution | Click profile | Edit | Select All devices | Click Generate.

Download the updated Distribution provisioning profile and install onto Jenkins build server:
Copy /Users/username/Downloads/*.mobileprovision file to here:

1. /Users/username/Library/MobileDevice/Provisioning\ Profiles/
2. /Users/Shared/Jenkins/MobileDevice/Provisioning\ Profiles/
3. /System/Library/MobileDevice/Provisioning\ Profiles/

Upgrade
Mac OS/X El Capitan 10.11 upgrade magically removes the MobileDevice folder under /System/Library/
Thus, without provisioning profiles stored there, Jenkins system process will fail to sign the iOS builds;

Therefore provisioning profiles must be copied back to /System/Library/MobileDevice folder as before.
However, when you attempt to copy them back you may receive the "Operation not permitted" error.

Solution: Reboot Mac | Press cmd + R on boot | OS X Utilities prompt | choose Utilities menu | Terminal
Type csrutil disable | Type reboot. Now when back in to you can manually copy the MobileDevice folder
Launch Terminal window. Type cp /Users/Shared/Jenkins/MobileDevice /System/Library/MobileDevice

Jenkins
Finally, update Jenkins automation: install the TestFlight plugin and add new nightly build job.

Manage Plugins
Main dashboard | Click Manage Jenkins | Click Manage Plugins | Click "Available" tab.
Ensure the following plugin is installed: Testflight plugin (upload to testflightapp.com)
Restart Plugins
Manually stop daemon: sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
Manually start daemon: sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

Configure System
Main dashboard | Click Manage Jenkins | Click Configure System. Enter the following: Test Flight | Test Flight Tokens

Token Pair Name	AngryBotsIOSTestFlight
API Token	Enter API Token from Test Flight
Team Token	Enter Team Token from Test Flight

New Item
Main dashboard | Click New Item | Enter Item name: AngryBotsIOSTestFlight.
Click radio button next to "Copy existing Item". Enter "AngryBotsIOS". Build Triggers
For example, build every night at 8pm Sunday-Thursday, ready for next working day.

Build periodically	Checked
Schedule	H 20 * * 0,1,2,3,4

Build | Add build step | Invoke Unity3d Editor

Unity3d installation name	Unity3D
Editor command line arguments:  -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinIOS  -CustomArgs:Environment=Beta;Version=2.0;BundleID=com.studiosstevepro.angrybotsbeta

Xcode | General build settings

Target	Unity-iPhone
Clean before build?	Checked
Configuration	Release
Pack application and build .ipa?	Checked
.ipa filename pattern	AngryBotsIOSTestFlight

Build | Add build step | Execute shell. Enter the following shell script:

# This is the output IPA file.
filename="AngryBotsIOSTestFlight.ipa"

echo "Copying $filename to /mnt/builds now STARTING..."
echo "Running as :"
whoami

# Navigate to destination.
cd /mnt/builds

# Remove current file if exists.
if [ -f "$filename" ]; then rm "$filename"; fi

# Copy latest version of file.
cp ${WORKSPACE}/output/"$filename" .

echo "Copying $filename to /mnt/builds now COMPLETE..."
echo Finished

Post build Actions | Upload to TestFlight

Token Pair	AngryBotsIOSTestFlight
IPA/APK Files (optional)	/mnt/builds/AngryBotsIOSTestFlight.ipa
Build Notes	Angry Bots IOS no. ${SVN_REVISION}
Append changelog to build notes	Checked

Click Advanced button | Update the following:

Distribution List	Enter Distribution List from Test Flight
Notify Team	Checked

Click Save button. Build job will now upload latest revision to TestFlight automatically!
Note: Beta testers will receive notification once latest revision available for download.

Conclusion
In conclusion, open source continuous integration tools like Jenkins make it easy to successfully build Test / Beta / Prod ready binaries. Additional tools like TestFlight make them even easier to distribute!