z88dk Programming Sample

In the previous post, we checked out z88dk Programming Setup. The z88dk is a portable Z80 / C cross compiler that targets hardware such as the 8-bit video games console: the Sega Master System (SMS).

Using z88dk, it is possible to write game code in high level C language rather than pure Z80 assembly.
Therefore, we would now like to extend this knowledge and write more detailed programming sample.

Let's check it out!

Software
Follow all instructions from the previous post: this documents how to setup the pre-requisite software.

Here is a summary of all required software to be installed:

Name  Version  Z80 Editor  ConTEXT  Assembler  WLA DX  Emulators  Fusion, Meka  Disassembler  SMS Examine

Name  Version  C IDE Editor  Crimson IDE  Cross compiler  Z88 Dev Kit  Make files  Cygwin  Hex Editor  HxD

Example
As an example, let's write the main loop of a simple "Shoot 'em up" style game, similar to the previous post, that simply moves a target sprite around the screen based on user input; preferably the joystick.

Graphics
Graphics on the Sega Master System are built up from four things: Palette, Tiles, Tilemap, Sprite table.
In our example, convert target sprite into a programming format the Sega Master System understands.

Here we will use the BMP2Tile utility from Maxim to convert the 16x16 pixel target sprite into SMS data.
Create 16x16 pixel target sprite (as above). Save target sprite as BMP (bitmap) file. Load in MS Paint:

Note: sprite / tile data on the SMS must be max. 4bpp (bits per pixel). Save as type 16 Color Bitmap:

BMP2Tile
Download and launch BMP2Tile. Drag n' drop 16x16 pixel target sprite onto open "Source" tab.

Choose "Tiles" tab. Ensure that "Remove duplicates" and "Planar tile output" are both checked.
Set "Index of first tile" to 1. It seems best practice to leave empty background tile at index 0.

Choose "Tilemap" tab. Leave "Use sprite palette" and "In front of sprites" options unchecked.

Choose "Palette" tab. Leave the "Output hex (SMS)" option checked for Sega Master System.
Click Save button on "Tiles", "Tilemap", "Palette" tabs. All data here is used in the game code.

Target Sprite
As an example, let's build on the existing test.c program found here at C:\z88dk\examples\sms.
First, create new directory: C:\TargetSprite. Next, add the following 2x files: test.s and linkfile.

test.s

.MEMORYMAP
SLOTSIZE $8000
DEFAULTSLOT 0
SLOT 0 $0000
SLOT 1 $8000
.ENDME

.ROMBANKMAP
BANKSTOTAL 1
BANKSIZE $8000
BANKS 1
.ENDRO

.SDSCTAG 1.0, "Target Sprite", "Shoot 'em up game main loop", "StevePro Studios"

.EMPTYFILL $C9                ;ret.
.COMPUTESMSCHECKSUM           ;compute sms checksum for this rom.

.BANK 0 SLOT 0
.ORG 0

.INCBIN "test.bin"

linkfile

[objects]
test.o

Launch Crimson IDE. File | New file. Save test.c. Next, type in the following source code: test.c.

test.c

#include <sms.h>

unsigned char pal1[] = {0x00, 0x02, 0x08, 0x0A, 0x20, 0x22, 0x28, 0x2A, 0x3F, 0x03, 0x0C, 0x0F, 0x30, 0x33, 0x3C, 0x3F};
unsigned char pal2[] = {0x00, 0x02, 0x08, 0x0A, 0x20, 0x22, 0x28, 0x2A, 0x3F, 0x03, 0x0C, 0x0F, 0x30, 0x33, 0x3C, 0x00};

unsigned char all_tiles[] =
{
  // blank tile.
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
 
  // target sprite.
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x39, 0x00, 0x00, 0x39, 0x20, 0x00, 0x00, 0x20, 0x21, 0x00, 0x00, 0x21, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x01, 0x2A, 0x00, 0x00, 0x2A,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x9C, 0x00, 0x00, 0x9C, 0x04, 0x00, 0x00, 0x04, 0x84, 0x00, 0x00, 0x84, 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x80, 0x54, 0x00, 0x00, 0x54,
  0x2A, 0x00, 0x00, 0x2A, 0x01, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x21, 0x00, 0x00, 0x21, 0x20, 0x00, 0x00, 0x20, 0x39, 0x00, 0x00, 0x39, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x54, 0x00, 0x00, 0x54, 0x80, 0x00, 0x00, 0x80, 0x00, 0x00, 0x00, 0x00, 0x84, 0x00, 0x00, 0x84, 0x04, 0x00, 0x00, 0x04, 0x9C, 0x00, 0x00, 0x9C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
};

void setSprite(int x, int y)
{
  // priority, x, y, tile.
  set_sprite(0, x+0, y+0, 1);
  set_sprite(1, x+8, y+0, 2);
  set_sprite(2, x+0, y+8, 3);
  set_sprite(3, x+8, y+8, 4); 
}

void main()
{
  int idx, off, x, y; 
  set_vdp_reg(VDP_REG_FLAGS1, VDP_REG_FLAGS1_SCREEN);

  // array, start, count, bpp.
  load_tiles(all_tiles, 0, 5, 4);

  // array, start, count.
  load_palette(pal1,  0, 16);
  load_palette(pal2, 16, 16);

  x=120, y=88; 
  setSprite(x, y);
  for (;;)
  {
    off = 2;
    idx = read_joypad1();
    if (idx & JOY_FIREB) { off = 4; }
    if (idx & JOY_LEFT)
    {      
      x -= off;
      if (x < 0)      { x = 0; }
    }
    if (idx & JOY_RIGHT)
    {
      x += off;
      if (x > 240)    { x = 240; }
    }
    if (idx & JOY_UP)
    {      
      y -= off;
      if (y < 0)      { y = 0; }
    }
    if (idx & JOY_DOWN)
    {
      y += off;
      if (y > 176)    { y = 176; }
    }
    wait_vblank_noint();
    setSprite(x, y);
  }
}
 

If you have C:\PerfomBuild.bat setup as per previous post then simply hit Ctrl+1 hot key to run!
Note: move target sprite left, right, up or down. Press "Fire2" button to double movement speed.

Disassemble
Finally, disassemble the compiled binary file test.sms generated from the Crimson IDE batch script.
In directory C:\TargetSprite, copy the 2x files Opcodes.dat + smsexamine.exe from SMS Examine zip.

Launch command prompt, change directory cd C:\TargetSprite. Type: smsexamine.exe test.sms.
This action will generate test.sms.asm and any data files. Launch ConTEXT. Open test.sms.asm:

Follow instructions from the previous post to setup hotkeys: F9 (compile) F10 (run) F11 (debug).

Press F9 to compile and link "test.sms.asm". This generates the "output.sms" binary file.
Press F10 to run program in the Fusion emulator. Press F11 to debug in Meka emulator:

Compare machine code in the debugger with the raw binary output file from test.sms.

Summary
This simple programming sample creates building blocks necessary to produce an 8-bit video game for Sega Master System. The next steps would be to build an 8-bit Homebrew video game to completion!

z88dk Programming Setup

In 2013, we checked out Sega Console Programming, specifically for the Sega Master System (SMS).
The SMS was the last ever 8-bit video game console manufactured by Sega built using the Z80 chip.

Here, we built a simple Hello World program similar to how 8-bit video games were written 30 years ago!
(Cartridge-based video games built for 8-bit Sega Retro Gaming consoles were written in pure assembly)

Therefore, the process to write Z80 assembly language from scratch for the SMS can be very daunting.
Fortunately, there is another way: write game code in C language using Z88 Development Kit (z88dk).
Let’s check it out!

z88dk
The z88dk is a Z80 / C cross compiler supplied with an assembler, linker and a set of C standard libraries.
The compiler is portable: targets Amiga OS, HP-UX 9, Linux, Mac OS/X, MS-DOS, Windows and the SMS.

Follow all instructions from the previous post: this documents how to setup the pre-requisite software.

Text Editor
Download and install ConTEXT. Download the syntax highlighter file. Save to the Highlighters folder.

Assembler
Recommended is the WLA DX assembler. Download compiled binaries and extract to C:\wlaz80win32.
Add C:\wlaz80win32 to PATH environment variable for custom batch scripts to compile Z80 / C code.

Emulators
If you followed the Sega Retro Gaming series then you would have both Fusion and Meka installed:
- Fusion: popular emulator used to play Sega retro games. Download Fusion364.zip and extract.
- Meka: emulator with some excellent debugging features. Download mekaw073.zip and extract.

Disassembler
Recommended is the SMS Examine disassembler. Download smsexamine1_2a.zip and extract.
Note: output is recompilable by WLA DX; thus perfect to reverse engineer binary output files.

Hex Editor
Also, install Hex Editor to inspect binary data generated from the assembler:
Download and extract HxD Hex Editor. Double click the setup.exe to install.

Setup
Follow all instructions from All Things Micro: this documents how to setup the Z80 dev environment.

Cygwin
Download and install Cygwin to C:\cygwin. Accept the default packages location C:\cygwin\packages.
Ensure you install devel package category. This is required to use Linux commands like cp and make.
Finally, add C:\cygwin\bin to PATH environment variable to use Linux commands on Windows:
Right click Computer | Properties | Advanced system settings | Environment Variables | Path
Start | Run | cmd. Enjoy Linux commands on Windows like cp and make!

z88dk
Download and install z88dk-1.8.0-setup.exe to C:\z88dk. Choose full installation all options.
Add C:\z88dk\bin to PATH environment variable. Update Temp directory to C:\Temp\Temp.
Also, ensure the z88dk installation configures the following User environment variables:

Z80_OZFILES	c:\z88dk\lib
ZCCCFG	c:\z88dk\lib\config

Start | Run | cmd. Type cd C:\z88dk\libsrc | Type zcc. The following should display:
Type make clean. Deletes all pre-compiled libraries and objects needed for current platforms.
Type make all. Makes all libraries for all the current platforms; this process may take a while!

Crimson IDE
Recommended is Crimson Emerald Editor. Download and install cedt-286M-setup.exe.
Configure the IDE with custom batch script to compile, link, and run all Z80 / C code.

Hello World
As an example, let's work through the libctest.c "Hello" program found in C:\z88dk\examples\sms.
First, create new directory: C:\HelloWorld. Next, add the following 3x files: test.c, test.s and linkfile.

test.c

#include <sms.h>
#include <stdio.h>

unsigned char pal1[] = {0x00, 0x20, 0x08, 0x28, 0x02, 0x22, 0x0A, 0x2A, 0x15, 0x35, 0x1D, 0x3D, 0x17, 0x37, 0x1F, 0x3F};
unsigned char pal2[] = {0x00, 0x03, 0x08, 0x28, 0x02, 0x22, 0x0A, 0x2A, 0x15, 0x35, 0x1D, 0x3D, 0x17, 0x37, 0x1F, 0x3F};

void main()
{
 set_vdp_reg(VDP_REG_FLAGS1, VDP_REG_FLAGS1_SCREEN);
 load_tiles(standard_font, 0, 255, 1);
 load_palette(pal1, 0, 16);
 load_palette(pal2, 16, 16);

 printf("Hello!");
 for (;;)
 {
  wait_vblank_noint();
 }
}
 

Important: ensure the final blank line is included otherwise an "Unexpected end of file error" may occur.

test.s

.MEMORYMAP
SLOTSIZE $8000
DEFAULTSLOT 0
SLOT 0 $0000
SLOT 1 $8000
.ENDME

.ROMBANKMAP
BANKSTOTAL 1
BANKSIZE $8000
BANKS 1
.ENDRO

.SDSCTAG 1.0, "Hello World", "Obligatory Hello World program", "StevePro Studios"

.EMPTYFILL $C9                ;ret.
.COMPUTESMSCHECKSUM           ;compute sms checksum for this rom.

.BANK 0 SLOT 0
.ORG 0

.INCBIN "test.bin"

linkfile

[objects]
test.o

Note: before continuing, ensure you make the following changes to the C:\wlaz80win32\Compile.bat file:

set WLAPATH=%~dp0

rem Cleanup to avoid confusion
if exist test.o del test.o

rem Compile
"%WLAPATH%wla-z80.exe" -o %1 test.o

rem Make linkfile
echo [objects]>linkfile
echo test.o>>linkfile

rem Link
"%WLAPATH%wlalink.exe" -drvs linkfile output.sms

rem Fixup for eSMS
if exist output.sms.sym del output.sms.sym
ren output.sym output.sms.sym

rem Cleanup to avoid mess
if exist test.o del test.o

Build
Manually compile, assemble and link the Hello program. Launch command prompt: Start | Run | cmd.
Change directory cd C:\HelloWorld. Next, execute the following 3x commands (in bold):

ACTION	COMMAND	OUTPUT
Compile	zcc +sms test.c -o test.bin -m -create-app	test.bin
Assemble	wla-z80 -o test.s	test.o
Link	wlalink -v linkfile test.sms	test.sms

Finally, type test.sms. The Hello program should launch in the Fusion emulator.
Congratulations! You have just written your first SMS program in C language.

Automate
Let's automate the build process: create PerformBuild.bat script file that contains the commands.
Configure Crimson IDE to perform build process through Tools menu | Preferences... | User Tools.

Create the following file C:\PerfomBuild.bat.

REM PerfomBuild.bat
zcc +sms test.c -o test.bin -m -create-app
wla-z80 -o test.s
wlalink -v linkfile test.sms
test.sms

Launch Crimson IDE. Navigate to Tools menu | Preferences... | User Tools. Configure Ctrl+1 hot key.

Menu Text	PerformBuild
Command	C:\PerformBuild.bat
Initial Dir	$(FileDir)
Hot Key	None
Close on exit	Checked
Save before execute	Checked

Next, modify the existing file test.c. For example, change print statement to "Hello World!". Hit Ctrl+1.

Disassemble
Finally, disassemble the compiled binary file test.sms generated from the Crimson IDE batch script.
In directory C:\HelloWorld, copy the 2x files Opcodes.dat + smsexamine.exe from SMS Examine zip.

Launch command prompt, change directory cd C:\HelloWorld. Type: smsexamine.exe test.sms.
This action will generate test.sms.asm and any data files. Launch ConTEXT. Open test.sms.asm:

Follow instructions from the previous post to setup hotkeys: F9 (compile) F10 (run) F11 (debug).

Recommended also is the following setting in ConTEXT to "Automatically update changed files":
Options menu | Environment Options | General | Automatically update changed files | Checked

Press F9 to compile and link "test.sms.asm". This generates the "output.sms" binary file.
Press F10 to run program in the Fusion emulator. Press F11 to debug in Meka emulator:

Compare machine code in the debugger with the raw binary output file from test.sms.

Summary
Writing source code in C language and using Z88 Development Kit are an alternative way to produce Z80 assembly language required for 8-bit cartridge-based video games consoles like the Sega Master System!

Unity3D and Automated Builds II

In the previous post, we automated the build process for Unity3D game development using Jenkins.
Here we built and packaged game code to streamline the deployment process to multiple devices.

While this approach works well, it is somewhat inflexible: whatever source code that is checked-in, it is built verbatim. That is, it is not possible to configure Jenkins build jobs to target different environments.

Also, during development, we may like to produce Test builds internally plus build and distribute Beta builds externally using TestFlight. All of which can be done by extending the Automated Build process.

Let’s check it out!

Pre-requisites
Ensure you have all the pre-requisites installed and setup for Jenkins automation as per previous post.

Example
As an example, let’s extend Angry Bots and configure a Patch Server to target different environments:

TEST	Used for internal development and testing.
BETA	Used for external development and testing.
PROD	Used for the Live Production environment.

Jenkins
The existing Angry Bots build job executes PerformBuild CommandLineBuildOnCheckinIOS method as it is.
Now we would like to configure the execute method to accept arguments to target different environments.

Unfortunately, Jenkins does not allow arguments to be passed to the execute method; we must build this infrastructure ourselves. Here is a great code sample that demonstrates how to implement this manually.

In our example we would like to override the following information as configured by default in Unity3D:
Target build environment (Test / Beta / Prod), Build version number and finally the Bundle Identifier.

Note: it may be necessary to override the Bundle ID to distinguish Beta and Prod side-by-side builds.

Implementation
Add the following "CommandLineReader.cs" file to the "Editor" folder to parse command line arguments:

using System;
using System.Collections.Generic;
using System.Linq;
using UnityEngine;

public class CommandLineReader
{
 public static string GetCustomArgument(string argumentName)
 {
  var customArgsDict = GetCustomArguments();
  string argumentValue;

  if (customArgsDict.ContainsKey(argumentName))
  {
   argumentValue = customArgsDict[argumentName];
  }
  else
  {
   Debug.Log("GetCustomArgument() Can't retrieve any custom argument named [" + argumentName + "] in command line [" + GetCommandLine() + "].");
   argumentValue = String.Empty;
  }

  Debug.Log("CommandLineReader " + argumentName + "=\"" + argumentValue + "\"");
  return argumentValue;
 }

 //Config
 private const string CUSTOM_ARGS_PREFIX = "-CustomArgs:";
 private const char CUSTOM_ARGS_SEPARATOR = ';';

 private static string[] GetCommandLineArgs()
 {
  return Environment.GetCommandLineArgs();
 }

 private static string GetCommandLine()
 {
  string[] args = GetCommandLineArgs();
  if (args.Length > 0)
  {
   return string.Join(" ", args);
  }
  else
  {
   Debug.LogError("GetCommandLine() - Can't find any command line arguments!");
   return String.Empty;
  }
 }

 private static Dictionary<string, string> GetCustomArguments()
 {
  var customArgsDict = new Dictionary<string, string>();
  string[] commandLineArgs = GetCommandLineArgs();

  string customArgsStr;
  try
  {
   customArgsStr = commandLineArgs.SingleOrDefault(row => row.Contains(CUSTOM_ARGS_PREFIX));
   if (String.IsNullOrEmpty(customArgsStr))
   {
    return customArgsDict;
   }
  }
  catch (Exception e)
  {
   Debug.LogError("GetCustomArguments() - Can't retrieve any custom arguments in command line [" + commandLineArgs + "]. Exception: " + e);
   return customArgsDict;
  }

  customArgsStr = customArgsStr.Replace(CUSTOM_ARGS_PREFIX, String.Empty);
  string[] customArgs = customArgsStr.Split(CUSTOM_ARGS_SEPARATOR);

  foreach (string customArg in customArgs)
  {
   string[] customArgBuffer = customArg.Split('=');
   if (customArgBuffer.Length == 2)
   {
    customArgsDict.Add(customArgBuffer[0], customArgBuffer[1]);
   }
   else
   {
    Debug.LogWarning("GetCustomArguments() - The custom argument [" + customArg + "] seems to be malformed.");
   }
  }
  return customArgsDict;
 }
}

Update the existing "PerformBuild.cs" file in the "Editor" folder to read arguments passed in to the build:

using System;
using System.Collections.Generic;
using System.IO;
using UnityEditor;
using UnityEngine;
using Object = UnityEngine.Object;

public class PerformBuild
{
 [MenuItem("Automated/Automated iOS Build")]
 static void CommandLineBuildOnCheckinIOS()
 {
  const BuildTarget target = BuildTarget.iPhone;
  
  string[] levels = GetBuildScenes();
  const string locationPathName = "iOSbuild";
  const BuildOptions options = BuildOptions.None;

  DeleteStreamingAssets();
  BuildPipelineBuildAssetBundle(target);

  string environment = CommandLineReader.GetCustomArgument("Environment");
  string version = CommandLineReader.GetCustomArgument("Version");
  string bundleID = CommandLineReader.GetCustomArgument("BundleID");

  string bundleIdentifier = (0 == bundleID.Length) ? "com.studiosstevepro.angrybots" : bundleID;
  BuildPipelineBuildPlayer(levels, locationPathName, target, options, environment, version, bundleIdentifier);
 }

 // similar code as previous post.

 private static void BuildPipelineBuildPlayer(string[] levels, string locationPathName, BuildTarget target,
  BuildOptions options, string environment, string version, string bundleIdentifier)
 {  
  PlayerSettings.productName = "Angry Bots";
  PlayerSettings.bundleIdentifier = bundleIdentifier;

  string bundleVersion = (0 == version.Length) ? "1.0" : version;
  PlayerSettings.bundleVersion = bundleVersion;

  if (0 != environment.Length)
  {   
   string fullPath = Application.streamingAssetsPath + "/Environment.txt";
   File.WriteAllText(fullPath, environment);  
  }
  
  String error = BuildPipeline.BuildPlayer(levels, locationPathName, target, options);
  if (!String.IsNullOrEmpty(error))
  {
   throw new System.Exception("Build failed: " + error);
  }
 }
}

Finally, write Unity3D client code that consumes the Patch Server build environment variable accordingly:

public class Utilities
{ 
 internal static string PATCH_SERVER_ROOT
 {
  get
  {
   if (String.IsNullOrEmpty(_patchServerRootData))
   {
    SetBuildEnvironmentVariables();
   }

   return _patchServerRootData;
  }
 }

 private static void SetBuildEnvironmentVariables()
 {
  TargetEnviroment target = TargetEnviroment.Test;
  string environment = target.ToString();

  string fullPath = Application.streamingAssetsPath + "/Environment.txt";
  if (File.Exists(fullPath))
  {
   environment = File.ReadAllText(fullPath);
  }

  try
  {
   target = (TargetEnviroment) Enum.Parse(typeof (TargetEnviroment), environment, true);
  }
  catch
  {
   // Must do this was as Enum.TryParse not available in .NET 2.0.
  }

  switch (target)
  {
   case TargetEnviroment.Test:
    _patchServerRootData = "http://TestPatchServer/"; break;

   case TargetEnviroment.Beta:
    _patchServerRootData = "http://BetaPatchServer/"; break;

   case TargetEnviroment.Prod:
    _patchServerRootData = "http://ProdPatchServer/"; break;
  }
 }

 private static string _patchServerRootData;

 enum TargetEnviroment
 {
  Test, Beta, Prod
 }
}

From here, any reference to the Patch Server from the Unity3D client code can be made simply like this:

  string patchServerRoot = Utilities.PATCH_SERVER_ROOT;

Note: ensure Optimization Api Compatibility Level is updated to accommodate Enum.Parse();

Navigate to File menu | Build Settings | Platform | Switch Platform | Other Settings
Change Optimization Api Compatibility Level from .NET 2.0 Subset to .NET 2.0.

Build Job
Make one small change to the existing AngryBotsIOS build job: pass arguments to the executeMethod.

Unity3d installation name	Unity3D
Editor command line arguments:  -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinIOS  -CustomArgs:Environment=Beta;Version=2.0;BundleID=com.studiosstevepro.angrybotsbeta

Click Save button | Click Build Now. Deploy Angry Bots to iOS device as per previous instructions.
Congratulations! Angry Bots should be installed as before but now target Beta build environment.

TestFlight
TestFlight Beta Testing makes it easy to invite users to test pre-release versions of iOS apps before they are released to the App Store. Up to 1,000 beta testers can be invited using just their email addresses.

Here is a good tutorial on how to setup TestFlight and how to use TestFlight to distribute the beta app:
1. Create an account on TestFlight with "Developer" option checked.
2. Log in to the TestFlight web site and "Add a team" for your app.
3. Add teammates, either testers or developers or available public.
4. Ask teammates to register iOS devices following the instructions.
5. Once accepted / registered, device information will be available.
6. Add devices / UDIDs to Devices section of Apple web site portal.
7. Update provisioning profile and install onto Jenkins build server.

Upload API
Extend the Automated Build process to upload latest revision to TestFlight via Upload API.

API Token
API Token gives access to Upload API and any endpoints for future TestFlight releases.
In TestFlight, click User icon (top right) | Account Settings | Upload API | API Token.

Team Token
Team Token is the token used for the Upload API "Team" parameter (in Jenkins).
In TestFlight, click Team name (top right) | Edit Info | Team Info | Team Token.

Distribution List
Create distribution list to notify testers once latest revision upload to TestFlight:
In TestFlight, click People tab | click "+" icon next to "Distribution Lists" to add.

Apple
Add same device information to the Apple portal and include in Distribution provisioning profile.
Navigate to Apple portal | iOS Apps | Log In | Certificates, Identifiers & Profiles | Devices.
Click the "+" icon (top right) to Register new device(s): Enter a valid name and the UDID.

Once all the device(s) have been registered, include in the Distribution provisioning profile:
Provisioning Profiles | Distribution | Click profile | Edit | Select All devices | Click Generate.

Download the updated Distribution provisioning profile and install onto Jenkins build server:
Copy /Users/username/Downloads/*.mobileprovision file to here:

1. /Users/username/Library/MobileDevice/Provisioning\ Profiles/
2. /Users/Shared/Jenkins/MobileDevice/Provisioning\ Profiles/
3. /System/Library/MobileDevice/Provisioning\ Profiles/

Upgrade
Mac OS/X El Capitan 10.11 upgrade magically removes the MobileDevice folder under /System/Library/
Thus, without provisioning profiles stored there, Jenkins system process will fail to sign the iOS builds;

Therefore provisioning profiles must be copied back to /System/Library/MobileDevice folder as before.
However, when you attempt to copy them back you may receive the "Operation not permitted" error.

Solution: Reboot Mac | Press cmd + R on boot | OS X Utilities prompt | choose Utilities menu | Terminal
Type csrutil disable | Type reboot. Now when back in to you can manually copy the MobileDevice folder
Launch Terminal window. Type cp /Users/Shared/Jenkins/MobileDevice /System/Library/MobileDevice

Jenkins
Finally, update Jenkins automation: install the TestFlight plugin and add new nightly build job.

Manage Plugins
Main dashboard | Click Manage Jenkins | Click Manage Plugins | Click "Available" tab.
Ensure the following plugin is installed: Testflight plugin (upload to testflightapp.com)
Restart Plugins
Manually stop daemon: sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
Manually start daemon: sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

Configure System
Main dashboard | Click Manage Jenkins | Click Configure System. Enter the following: Test Flight | Test Flight Tokens

Token Pair Name	AngryBotsIOSTestFlight
API Token	Enter API Token from Test Flight
Team Token	Enter Team Token from Test Flight

New Item
Main dashboard | Click New Item | Enter Item name: AngryBotsIOSTestFlight.
Click radio button next to "Copy existing Item". Enter "AngryBotsIOS". Build Triggers
For example, build every night at 8pm Sunday-Thursday, ready for next working day.

Build periodically	Checked
Schedule	H 20 * * 0,1,2,3,4

Build | Add build step | Invoke Unity3d Editor

Unity3d installation name	Unity3D
Editor command line arguments:  -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinIOS  -CustomArgs:Environment=Beta;Version=2.0;BundleID=com.studiosstevepro.angrybotsbeta

Xcode | General build settings

Target	Unity-iPhone
Clean before build?	Checked
Configuration	Release
Pack application and build .ipa?	Checked
.ipa filename pattern	AngryBotsIOSTestFlight

Build | Add build step | Execute shell. Enter the following shell script:

# This is the output IPA file.
filename="AngryBotsIOSTestFlight.ipa"

echo "Copying $filename to /mnt/builds now STARTING..."
echo "Running as :"
whoami

# Navigate to destination.
cd /mnt/builds

# Remove current file if exists.
if [ -f "$filename" ]; then rm "$filename"; fi

# Copy latest version of file.
cp ${WORKSPACE}/output/"$filename" .

echo "Copying $filename to /mnt/builds now COMPLETE..."
echo Finished

Post build Actions | Upload to TestFlight

Token Pair	AngryBotsIOSTestFlight
IPA/APK Files (optional)	/mnt/builds/AngryBotsIOSTestFlight.ipa
Build Notes	Angry Bots IOS no. ${SVN_REVISION}
Append changelog to build notes	Checked

Click Advanced button | Update the following:

Distribution List	Enter Distribution List from Test Flight
Notify Team	Checked

Click Save button. Build job will now upload latest revision to TestFlight automatically!
Note: Beta testers will receive notification once latest revision available for download.

Conclusion
In conclusion, open source continuous integration tools like Jenkins make it easy to successfully build Test / Beta / Prod ready binaries. Additional tools like TestFlight make them even easier to distribute!

Unity3D and Automated Builds

In the previous post, we discussed Unity Remote to lessen the time taken to view constant changes on target platforms. Now, we would like to package game code once to be deployed to multiple devices.

In this post, we use Jenkins: an open source continuous integration tool to perform automated builds.
At the time of this writing, Jenkins is available for Android on PC and both Android / iOS on the Mac.

Let's check it out!

Windows PC
Ensure you have the following pre-requisites installed and setup as per previous post.

The Java SDK	Environment Variables
Android SDK	USB Drivers
Unity 3D

Also, ensure you have an SVN client installed on localhost, for example, Tortoise SVN.

Example
As an example, let's upload Angry Bots into source control (SVN) to demonstrate automated builds.

Add the following "PerformBuild.cs" file to the "Editor" folder to help automate the Android build:
Note: upgrade to Unity Pro licence if you would like to programmatically Build the Player Pipeline.

using System;
using System.Collections.Generic;
using System.IO;
using UnityEditor;
using UnityEngine;
using Object = UnityEngine.Object;

public class PerformBuild
{ 
  [MenuItem("Automated/Automated Android Build")]
  static void CommandLineBuildOnCheckinAndroid()
  {
    const BuildTarget target = BuildTarget.Android;

    string[] levels = GetBuildScenes();
    const string locationPathName = "AngryBotsAndroid.apk";
    const BuildOptions options = BuildOptions.None;

    DeleteStreamingAssets();
    BuildPipelineBuildAssetBundle(target);
    BuildPipelineBuildPlayer(levels, locationPathName, target, options);
  }
 
  private static string[] GetBuildScenes()
  {
    List<string> names = new List<string>();
    foreach (EditorBuildSettingsScene e in EditorBuildSettings.scenes)
    {
      if (e == null) { continue; }
      if (e.enabled) { names.Add(e.path); }
    }
    return names.ToArray();
  }
 
  private static void DeleteStreamingAssets()
  {
    // Delete streaming assets (downloaded from source control).
    string[] filesToDelete = Directory.GetFiles(Application.streamingAssetsPath, "*.unity3d*");
    foreach (string file in filesToDelete) 
    {
      File.Delete(file);
    }
  }

  private static void BuildPipelineBuildAssetBundle(BuildTarget buildTarget)
  {
    string[] assetPaths = AssetDatabase.GetAllAssetPaths();
  
    string pathName = Application.streamingAssetsPath;
    foreach (string f in assetPaths)
    {
      if (!f.Contains("Master Assets")) { continue; }
      Object a = Resources.LoadAssetAtPath(f, typeof(Object));
      if (a == null) { continue; }

      Object[] asset = new Object[1];
      asset[0] = a;
      string assetType = a.GetType().Name;
      if (assetType.Equals("Object")) { continue; }

      string assetName = assetType + "_" + asset[0].name + ".unity3d";
      string fullName = pathName + "/" + assetName;

      const BuildAssetBundleOptions options = BuildAssetBundleOptions.CollectDependencies | BuildAssetBundleOptions.CompleteAssets | BuildAssetBundleOptions.UncompressedAssetBundle;
 
      BuildPipeline.BuildAssetBundle(a, asset, fullName, options, buildTarget);
    }
  }

  private static void BuildPipelineBuildPlayer(string[] levels, string locationPathName, BuildTarget target, BuildOptions options)
  {
    PlayerSettings.productName = "Angry Bots";
    PlayerSettings.bundleIdentifier = "com.studiosstevepro.angrybots";
    PlayerSettings.bundleVersion = "1.0";

    String error = BuildPipeline.BuildPlayer(levels, locationPathName, target, options);
    if (!String.IsNullOrEmpty(error))
    {
      throw new System.Exception("Build failed: " + error);
    }  
  }
}

Jenkins
Download Jenkins Binary Installer and run. Restart computer. Navigate to http://localhost:8080.

Manage Plugins
Main dashboard | Click Manage Jenkins | Click Manage Plugins | Click "Available" tab.
Ensure the following plugins are installed: Subversion Plugin and Unity3d plugin.
Restart Plugins
Start | run | services.msc. Scroll down to Jenkins. Right click. Restart.
Configure System
Main dashboard | Click Manage Jenkins | Click Configure System. Enter the following:
Unity3d | Add Unity3d | Unity3d installations

Name	Unity3D
Installation directory	C:\Program Files (x86)\Unity\

Jenkins Location

Jenkins URL	http://buildserver:8080
System Admin e-mail address	build@mycompany.com

Subversion

Subversion Workspace Version

1.7

E-mail Notification

SMTP server

smtp@mycompany.com

New Item
Main dashboard | Click New Item | Enter Item name: AngryBotsAndroid.
Click radio button next to "Build a free-style software project". Click OK.
Configuration

Project Name

AngryBotsAndroid

Source Code Management

Subversion Modules	Checked
Repository URL	https://mycompany.com/svn/AngryBots
Check-out Strategy	Use 'svn update', with 'svn revert'

Note: you may be prompted to enter SVN source control credentials at this point.
Credentials | Click Add button | Add SVN source control Username and Password.

Build | Add build step | Invoke Unity3d Editor

Unity3d installation name	Unity3D
Editor command line arguments:  -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinAndroid

Build | Add build step | Execute shell

Command

echo ${WORKSPACE}

Post-build Actions | E-mail Notification

Recipients

Enter email distribution list

Click Apply button | Click Save button | Click Build Now.

Note: during build job, Unity may prompt you to select Android SDK path. Choose D:\Android\sdk

Deploy
Main dashboard | Click AngryBotsAndroid | Build History | Click DateTime link | Console Output.

Scroll to bottom: note the absolute path to AngryBotsAndroid build job workspace entered there.
For example, it may be C:\Program Files (x86)\Jenkins\jobs\AngryBotsAndroid\workspace

Connect Android device to PC. Launch command prompt, navigate to Android APK file and install:
Start | run | cmd | cd "C:\Program Files (x86)\Jenkins\jobs\AngryBotsAndroid\workspace"
adb install -r AngryBotsAndroid.apk

Congratulations! Angry Bots should be installed on your Android device ready to play.

Apple Mac
Ensure you have the following pre-requisites installed and setup as per previous post.

The Java SDK	Environment Variables
Android SDK	USB Drivers
Unity 3D	Xcode

Also, ensure you have an SVN client installed on localhost, for example, svnx for Mac.

In order to deploy code to an iOS device, this post assumes you have an Apple Developer ID.
This post also assumes you have a Provisioning Profile setup via the Apple Developer portal.

Archive
Automated builds may actually benefit from archiving final packaged products to a network file share.
That way, packages can be deployed from mounted network file share on to multiple devices directly.

Mount Drive
For example: there is a Shared/builds folder on ARCHIVE-SERVER for automated builds.

Finder | Go menu | Connect to Server... Enter the following Server Address: smb://username:password@ARCHIVE-SERVER/Shared. Click Connect button.

Launch Terminal window to manually mount drive. Type the following (as root):
sudo su | cd /mnt | mkdir builds | cd builds
mount_smbfs //ARCHIVE-SERVER/Shared/builds /mnt/builds

Root User
Important: Jenkins must run as root user in order to have copy permission to mount drive.
Launch Terminal window to elevate Jenkins privileges as root user. Modify configuration: cd /Library/LaunchDaemons | vi org.jenkins-ci.plist | Find <key>UserName</key>
Change <string>Jenkins</string> TO <string>root</string> | Type !wq to Save.

Example
Update the "PerformBuild.cs" file that exists in the "Editor" folder to help automate the iOS build:
Note: upgrade to Unity Pro licence if you would like to programmatically Build the Player Pipeline.

public class PerformBuild
{
  [MenuItem("Automated/Automated iOS Build")]
  static void CommandLineBuildOnCheckinIOS()
  {
    const BuildTarget target = BuildTarget.iPhone;

    string[] levels = GetBuildScenes();
    const string locationPathName = "iOSbuild";
    const BuildOptions options = BuildOptions.None;

    DeleteStreamingAssets();
    BuildPipelineBuildAssetBundle(target);
    BuildPipelineBuildPlayer(levels, locationPathName, target, options);
  }

  // same code as before.
}

Jenkins
Download Jenkins Binary Installer and run. Restart computer. Navigate to http://localhost:8080.

Note: if you receive warning "Jenkins.pkg can't be opened because from unidentified developer"
Choose: System Preferences | Security & Privacy | Click "Open anyway" button | Install.

Manage Plugins
Main dashboard | Click Manage Jenkins | Click Manage Plugins | Click "Available" tab.
Ensure the following plugins are installed: Subversion, Unity3d and the Xcode plugin.

Restart Plugins
The Jenkins Installer sets up a launch daemon listening on port 8080 for all incoming requests.
Therefore, launch Terminal window to manually restart daemon. Type the following (as root): Manually stop daemon: sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
Manually start daemon: sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

Configure System
Main dashboard | Click Manage Jenkins | Click Configure System. Enter the following: Configuration

Home directory

/Users/Shared/Jenkins/Home

Unity3d | Add Unity3d | Unity3d installations

Name	Unity3D
Installation directory	/Applications/Unity/Unity.app

Xcode Builder

xcodebuild executable path	/usr/bin/xcodebuild
agvtool executable path	/usr/bin/agvtool
xcrun executable path	/usr/bin/xcrun

Jenkins Location

Jenkins URL	http://buildserver:8080
System Admin e-mail address	build@mycompany.com

Subversion

Subversion Workspace Version

1.7

E-mail Notification

SMTP server

smtp@mycompany.com

New Item
Main dashboard | Click New Item | Enter Item name: AngryBotsIOS.
Click radio button next to "Build a free-style software project". OK.
Configuration

Project Name

AngryBotsIOS

Source Code Management

Subversion Modules	Checked
Repository URL	https://mycompany.com/svn/AngryBots
Check-out Strategy	Use 'svn update', with 'svn revert'

Note: you may be prompted to enter SVN source control credentials at this point.
Credentials | Click Add button | Add SVN source control Username and Password.

Build Triggers
For example, poll build server every 10 minutes for any source code revisions.

Poll SCM	Checked
Schedule	H/10 * * * *

Build | Add build step | Invoke Unity3d Editor

Unity3d installation name	Unity3D
Editor command line arguments:  -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinIOS

Xcode | General build settings

Target	Unity-iPhone
Clean before build?	Checked
Configuration	Release
Pack application and build .ipa?	Checked
.ipa filename pattern	AngryBotsIOS

Xcode | Code signing & OS X keychain options

Code signing & OS X keychain options

iPhone Distribution

Note: leave blank if you do not wish to override the code signing identity in project.

Xcode | Advanced Xcode build options

Xcode Project Directory	iOSbuild
Build output directory	${WORKSPACE}/output

Build | Add build step | Execute shell. Enter the following shell script:

# This is the output IPA file.
filename="AngryBotsIOS.ipa"

echo "Copying $filename to /mnt/builds now STARTING..."
echo "Running as :"
whoami

# Navigate to destination.
cd /mnt/builds

# Remove current file if exists.
if [ -f "$filename" ]; then rm "$filename"; fi

# Copy latest version of file.
cp ${WORKSPACE}/output/"$filename" .

# Create archive directory if necessary.
testdate=${BUILD_ID}
filedate=`echo "$testdate" | cut -d '_' -f1`

if [ -d "$filedate" ]; then echo "$filedate" exists; else mkdir "$filedate"; fi
cd "$filedate"

# Remove archive file if exists.
archfile=${SVN_REVISION}_"$filename"
if [ -f "$archfile" ]; then rm "$archfile"; fi

# Copy archive version of file.
cp ${WORKSPACE}/output/"$filename" ./"$archfile"

echo "Copying $filename to /mnt/builds now COMPLETE..."
echo Finished

Click Apply button | Click Save Button | Click Build Now.

Errors
iOS automated builds may encounter errors if the build server is not completely setup.

Code Sign Errors
There are no valid certificate/private key pairs in the default keychain
Open "Keychain Access" application. Finder | Applications | Utilities | Keychain Access

Click login tab (top left) | Right click "iPhone Developer YOUR NAME" | Choose Copy
Click System tab | Right click in area | Choose Paste 2 items

Note: you may need to repeat with the "iPhone Distribution YOUR NAME" certificate.

Provisioning profile 'xxxxx-xxxx-xxxx-xxxxx' can't be found
Copy your username mobile device provisioning profile to the shared Jenkins folder:
Launch Terminal window to copy provisioning profile. Type the following (as root):

Copy /Users/username/Library/MobileDevice folder TO /Users/Shared/Jenkins
Copy /Users/username/Library/MobileDevice folder TO /System/Library

Upgrade
Mac OS/X El Capitan 10.11 upgrade magically removes the MobileDevice folder under /System/Library/
Thus, without provisioning profiles stored there, Jenkins system process will fail to sign the iOS builds;

Therefore provisioning profiles must be copied back to /System/Library/MobileDevice folder as before.
However, when you attempt to copy them back you may receive the "Operation not permitted" error.

Solution: Reboot Mac | Press cmd + R on boot | OS X Utilities prompt | choose Utilities menu | Terminal
Type csrutil disable | Type reboot. Now when back in to you can manually copy the MobileDevice folder
Launch Terminal window. Type cp /Users/Shared/Jenkins/MobileDevice /System/Library/MobileDevice

/usr/bin/codesign SecKey API returned: unknown error -1=ffffffffffffffff
Open "Keychain Access" application. Finder | Applications | Utilities | Keychain Access

Click login tab (top left) | Expand "iPhone Distribution" | Right click private key | Get Info
Click the "Access Control" button (top right). Ensure the following options are set here:
Click System tab (mid left) | Expand "iPhone Distribution" | Right click private key | Get Info
Click the "Access Control" button (top right). Ensure the following options are set here:

FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL
If you encounter this error then Jenkins wants to run as its own private user instead of Mac OS/X.
Create slave node on Jenkins thus the Unity process can connect as Mac OS/X user accordingly:

Main dashboard | Click Manage Jenkins | Click Manage Nodes | New Node

Node name	Unity3D
Dumb slave	Checked

Configure

Name	Unity3D
# of executors	2
Remote root directory	/Users/username/JenkinsNode
Usage	Leave this machine for tied Jobs only
Launch method	Launch slave agents via Java Web Start
Availability	Keep this slave on-line as much as possible

UnityException: Launching iOS project via Xcode4 failed. Check editor log for details
If you are using Unity 4.3.* and Xcode 5.1.* then you may possibly receive this error.

Manually edit Xcode plugin metadata Info.plist file for Unity and add the missing key:
Launch Terminal window. Type the following (as root):

cd /Applications/Unity/Unity.app/Contents/BuildTargetTools
cd iPhonePlayer/Unity4XC.xcplugin/Contents

vi Info.plist | Search for DVTPlugInCompatibilityUUIDs | Add key:
<string>A2E4D43F-41F4-4FB9-BB94-7177011C9AED</string>

UnityException: Launching iOS project via Xcode4 failed. Check editor log for details
If you are using Unity 4.5.* and Xcode 6.0.* then you may possibly receive this error.

Manually edit Xcode plugin metadata Info.plist file for Unity and add the missing key:
Launch Terminal window. Type the following (as root):

cd /Applications/Unity/Unity.app/Contents/PlaybackEngines
cd iOSSupport/Tools/OSX/Unity4XC.xcplugin/Contents

vi Info.plist | Search for DVTPlugInCompatibilityUUIDs | Add key:
<string>C4A681B0-4A26-480E-93EC-1218098B9AA0</string>

UnityException: Launching iOS project via Xcode4 failed. Check editor log for details
If you are using Unity 4.6.* and Xcode 6.2.* then you may possibly receive this error.

Manually edit Xcode plugin metadata Info.plist file for Unity and add the missing key:
Launch Terminal window. Type the following (as root):

cd /Applications/Unity/Unity.app/Contents/PlaybackEngines
cd iOSSupport/Tools/OSX/Unity4XC.xcplugin/Contents

vi Info.plist | Search for DVTPlugInCompatibilityUUIDs | Add key:
<string>A16FF353-8441-459E-A50C-B071F53F51B7</string>

Method
In order to determine which GUID must be added follow steps as outlined here:
Finder | Applications | Right click Xcode | Show Package Contents | Contents
Double click Info.plist. Choose DVT PluginCompatibility UUID | Copy GUID

Next
Finder | Applications | Unity | Right click Unity.app | Show Package Contents
Contents | PlaybackEngines | iOSsupport | Tools | OSX
Right click Unity4XC.xcplugin | Contents | Double click Info.plist
Choose DVT PluginCompatibility UUID | Click + | Paste GUID

Deploy
After the iOS build is successful, deploy packaged game code once to multiple devices:

Connect an iOS device to your PC | Launch iTunes | Click the top left drop down arrow
Add File to Library... | Navigate to \\ARCHIVE-SERVER\Shared | Double click AngryBotsIOS.ipa Click iPhone button | Click Apps button | Click Install button (next to AngryBots) | Click Apply button. Congratulations! Angry Bots should be installed on your iOS device ready to play.

Summary
The only automated build option outstanding here is an Android build deployed on the Mac.
Fortunately, this is simple: clone iOS build job, remove Xcode build step and modify script.

Clone iOS build
Main dashboard | Click New Item | Enter Item name: AngryBotsAndroid.
Click radio button next to "Copy existing Item". Enter "AngryBotsIOS".

Build | Add build step | Invoke Unity3d Editor

Unity3d installation name	Unity3D
Editor command line arguments: -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinAndroid

Remove Xcode build step
Scroll down to Xcode | General build settings | Click Delete button.

Modify script.
Make the following changes to shell script to accommodate Android packaged game code:

# This is the output APK file.
filename="AngryBotsAndroid.APK"

echo "Copying $filename to /mnt/builds now STARTING..."
echo "Running as :"
whoami

# Navigate to destination.
cd /mnt/builds

# Remove current file if exists.
if [ -f "$filename" ]; then rm "$filename"; fi

# Copy latest version of file.
cp ${WORKSPACE}/"$filename" .

# Create archive directory if necessary.
testdate=${BUILD_ID}
filedate=`echo "$testdate" | cut -d '_' -f1`

if [ -d "$filedate" ]; then echo "$filedate" exists; else mkdir "$filedate"; fi
cd "$filedate"

# Remove archive file if exists.
archfile=${SVN_REVISION}_"$filename"
if [ -f "$archfile" ]; then rm "$archfile"; fi

# Copy archive version of file.
cp ${WORKSPACE}/"$filename" ./"$archfile"

echo "Copying $filename to /mnt/builds now COMPLETE..."
echo Finished

Conclusion
In conclusion, Continuous Integration has many advantages; one of which of course is stability: an automated build process constantly checks state of the code and immediately reports build errors.

The timeliness of this feedback is critical for maintaining stable code. Especially for software being regularly deployed to multiple devices; most of which avail to frequent online mobile store updates!