Monday, September 15, 2014

z88dk Programming Setup

In 2013, we checked out Sega Console Programming, specifically for the Sega Master System (SMS).
The SMS was the last ever 8-bit video game console manufactured by Sega built using the Z80 chip.

Here, we built a simple Hello World program similar to how 8-bit video games were written 30 years ago!
(Cartridge-based video games built for 8-bit Sega Retro Gaming consoles were written in pure assembly)

Therefore, the process to write Z80 assembly language from scratch for the SMS can be very daunting.
Fortunately, there is another way: write game code in C language using Z88 Development Kit (z88dk).
     

Let’s check it out!

z88dk
The z88dk is a Z80 / C cross compiler supplied with an assembler, linker and a set of C standard libraries.
The compiler is portable: targets Amiga OS, HP-UX 9, Linux, Mac OS/X, MS-DOS, Windows and the SMS.

Follow all instructions from the previous post: this documents how to setup the pre-requisite software.

Text Editor
Download and install ConTEXT. Download the syntax highlighter file. Save to the Highlighters folder.

Assembler
Recommended is the WLA DX assembler. Download compiled binaries and extract to C:\wlaz80win32.
Add C:\wlaz80win32 to PATH environment variable for custom batch scripts to compile Z80 / C code.

Emulators
If you followed the Sega Retro Gaming series then you would have both Fusion and Meka installed:
- Fusion: popular emulator used to play Sega retro games. Download Fusion364.zip and extract.
- Meka: emulator with some excellent debugging features. Download mekaw073.zip and extract.

Disassembler
Recommended is the SMS Examine disassembler. Download smsexamine1_2a.zip and extract.
Note: output is recompilable by WLA DX; thus perfect to reverse engineer binary output files.

Hex Editor
Also, install Hex Editor to inspect binary data generated from the assembler:
Download and extract HxD Hex Editor. Double click the setup.exe to install.

Setup
Follow all instructions from All Things Micro: this documents how to setup the Z80 dev environment.

Cygwin
Download and install Cygwin to C:\cygwin. Accept the default packages location C:\cygwin\packages.
Ensure you install devel package category. This is required to use Linux commands like cp and make.
Finally, add C:\cygwin\bin to PATH environment variable to use Linux commands on Windows:
Right click Computer | Properties | Advanced system settings | Environment Variables | Path
Start | Run | cmd. Enjoy Linux commands on Windows like cp and make!

z88dk
Download and install z88dk-1.8.0-setup.exe to C:\z88dk. Choose full installation all options.
Add C:\z88dk\bin to PATH environment variable. Update Temp directory to C:\Temp\Temp.
Also, ensure the z88dk installation configures the following User environment variables:
 Z80_OZFILES c:\z88dk\lib
 ZCCCFG c:\z88dk\lib\config

Start | Run | cmd. Type cd C:\z88dk\libsrc | Type zcc. The following should display:
Type make clean. Deletes all pre-compiled libraries and objects needed for current platforms.
Type make all. Makes all libraries for all the current platforms; this process may take a while!

Crimson IDE
Recommended is Crimson Emerald Editor. Download and install cedt-286M-setup.exe.
Configure the IDE with custom batch script to compile, link, and run all Z80 / C code.

Hello World
As an example, let's work through the libctest.c "Hello" program found in C:\z88dk\examples\sms.
First, create new directory: C:\HelloWorld. Next, add the following 3x files: test.c, test.s and linkfile.

test.c
#include <sms.h>
#include <stdio.h>

unsigned char pal1[] = {0x00, 0x20, 0x08, 0x28, 0x02, 0x22, 0x0A, 0x2A, 0x15, 0x35, 0x1D, 0x3D, 0x17, 0x37, 0x1F, 0x3F};
unsigned char pal2[] = {0x00, 0x03, 0x08, 0x28, 0x02, 0x22, 0x0A, 0x2A, 0x15, 0x35, 0x1D, 0x3D, 0x17, 0x37, 0x1F, 0x3F};

void main()
{
 set_vdp_reg(VDP_REG_FLAGS1, VDP_REG_FLAGS1_SCREEN);
 load_tiles(standard_font, 0, 255, 1);
 load_palette(pal1, 0, 16);
 load_palette(pal2, 16, 16);

 printf("Hello!");
 for (;;)
 {
  wait_vblank_noint();
 }
}
 
Important: ensure the final blank line is included otherwise an "Unexpected end of file error" may occur.

test.s
.MEMORYMAP
SLOTSIZE $8000
DEFAULTSLOT 0
SLOT 0 $0000
SLOT 1 $8000
.ENDME

.ROMBANKMAP
BANKSTOTAL 1
BANKSIZE $8000
BANKS 1
.ENDRO

.SDSCTAG 1.0, "Hello World", "Obligatory Hello World program", "StevePro Studios"

.EMPTYFILL $C9                ;ret.
.COMPUTESMSCHECKSUM           ;compute sms checksum for this rom.

.BANK 0 SLOT 0
.ORG 0

.INCBIN "test.bin"
linkfile
[objects]
test.o

Note: before continuing, ensure you make the following changes to the C:\wlaz80win32\Compile.bat file:
set WLAPATH=%~dp0

rem Cleanup to avoid confusion
if exist test.o del test.o

rem Compile
"%WLAPATH%wla-z80.exe" -o %1 test.o

rem Make linkfile
echo [objects]>linkfile
echo test.o>>linkfile

rem Link
"%WLAPATH%wlalink.exe" -drvs linkfile output.sms

rem Fixup for eSMS
if exist output.sms.sym del output.sms.sym
ren output.sym output.sms.sym

rem Cleanup to avoid mess
if exist test.o del test.o

Build
Manually compile, assemble and link the Hello program. Launch command prompt: Start | Run | cmd.
Change directory cd C:\HelloWorld. Next, execute the following 3x commands (in bold):
 ACTION  COMMAND  OUTPUT
 Compile  zcc +sms test.c -o test.bin -m -create-app  test.bin
 Assemble  wla-z80 -o test.s  test.o
 Link  wlalink -v linkfile test.sms  test.sms

Finally, type test.sms. The Hello program should launch in the Fusion emulator.
Congratulations! You have just written your first SMS program in C language.

Automate
Let's automate the build process: create PerformBuild.bat script file that contains the commands.
Configure Crimson IDE to perform build process through Tools menu | Preferences... | User Tools.

Create the following file C:\PerfomBuild.bat.
REM PerfomBuild.bat
zcc +sms test.c -o test.bin -m -create-app
wla-z80 -o test.s
wlalink -v linkfile test.sms
test.sms
Launch Crimson IDE. Navigate to Tools menu | Preferences... | User Tools. Configure Ctrl+1 hot key.
 Menu Text PerformBuild
 Command C:\PerformBuild.bat
 Initial Dir $(FileDir)
 Hot Key None
 Close on exit Checked
 Save before execute Checked

Next, modify the existing file test.c. For example, change print statement to "Hello World!". Hit Ctrl+1.

Disassemble
Finally, disassemble the compiled binary file test.sms generated from the Crimson IDE batch script.
In directory C:\HelloWorld, copy the 2x files Opcodes.dat + smsexamine.exe from SMS Examine zip.

Launch command prompt, change directory cd C:\HelloWorld. Type: smsexamine.exe test.sms.
This action will generate test.sms.asm and any data files. Launch ConTEXT. Open test.sms.asm:

Follow instructions from the previous post to setup hotkeys: F9 (compile) F10 (run) F11 (debug).

Recommended also is the following setting in ConTEXT to "Automatically update changed files":
Options menu | Environment Options | General | Automatically update changed files | Checked

Press F9 to compile and link "test.sms.asm". This generates the "output.sms" binary file.
Press F10 to run program in the Fusion emulator. Press F11 to debug in Meka emulator:

Compare machine code in the debugger with the raw binary output file from test.sms.

Summary
Writing source code in C language and using Z88 Development Kit are an alternative way to produce Z80 assembly language required for 8-bit cartridge-based video games consoles like the Sega Master System!

Sunday, August 31, 2014

Unity3D and Automated Builds II

In the previous post, we automated the build process for Unity3D game development using Jenkins.
Here we built and packaged game code to streamline the deployment process to multiple devices.

While this approach works well, it is somewhat inflexible: whatever source code that is checked-in, it is built verbatim. That is, it is not possible to configure Jenkins build jobs to target different environments.

Also, during development, we may like to produce Test builds internally plus build and distribute Beta builds externally using TestFlight. All of which can be done by extending the Automated Build process.

Let’s check it out!

Pre-requisites
Ensure you have all the pre-requisites installed and setup for Jenkins automation as per previous post.

Example
As an example, let’s extend Angry Bots and configure a Patch Server to target different environments:
 TEST  Used for internal development and testing.
 BETA  Used for external development and testing.
 PROD  Used for the Live Production environment.

Jenkins
The existing Angry Bots build job executes PerformBuild CommandLineBuildOnCheckinIOS method as it is.
Now we would like to configure the execute method to accept arguments to target different environments.

Unfortunately, Jenkins does not allow arguments to be passed to the execute method; we must build this infrastructure ourselves. Here is a great code sample that demonstrates how to implement this manually.

In our example we would like to override the following information as configured by default in Unity3D:
Target build environment (Test / Beta / Prod), Build version number and finally the Bundle Identifier.

Note: it may be necessary to override the Bundle ID to distinguish Beta and Prod side-by-side builds.

Implementation
Add the following "CommandLineReader.cs" file to the "Editor" folder to parse command line arguments:
using System;
using System.Collections.Generic;
using System.Linq;
using UnityEngine;

public class CommandLineReader
{
 public static string GetCustomArgument(string argumentName)
 {
  var customArgsDict = GetCustomArguments();
  string argumentValue;

  if (customArgsDict.ContainsKey(argumentName))
  {
   argumentValue = customArgsDict[argumentName];
  }
  else
  {
   Debug.Log("GetCustomArgument() Can't retrieve any custom argument named [" + argumentName + "] in command line [" + GetCommandLine() + "].");
   argumentValue = String.Empty;
  }

  Debug.Log("CommandLineReader " + argumentName + "=\"" + argumentValue + "\"");
  return argumentValue;
 }

 //Config
 private const string CUSTOM_ARGS_PREFIX = "-CustomArgs:";
 private const char CUSTOM_ARGS_SEPARATOR = ';';

 private static string[] GetCommandLineArgs()
 {
  return Environment.GetCommandLineArgs();
 }

 private static string GetCommandLine()
 {
  string[] args = GetCommandLineArgs();
  if (args.Length > 0)
  {
   return string.Join(" ", args);
  }
  else
  {
   Debug.LogError("GetCommandLine() - Can't find any command line arguments!");
   return String.Empty;
  }
 }

 private static Dictionary<string, string> GetCustomArguments()
 {
  var customArgsDict = new Dictionary<string, string>();
  string[] commandLineArgs = GetCommandLineArgs();

  string customArgsStr;
  try
  {
   customArgsStr = commandLineArgs.SingleOrDefault(row => row.Contains(CUSTOM_ARGS_PREFIX));
   if (String.IsNullOrEmpty(customArgsStr))
   {
    return customArgsDict;
   }
  }
  catch (Exception e)
  {
   Debug.LogError("GetCustomArguments() - Can't retrieve any custom arguments in command line [" + commandLineArgs + "]. Exception: " + e);
   return customArgsDict;
  }

  customArgsStr = customArgsStr.Replace(CUSTOM_ARGS_PREFIX, String.Empty);
  string[] customArgs = customArgsStr.Split(CUSTOM_ARGS_SEPARATOR);

  foreach (string customArg in customArgs)
  {
   string[] customArgBuffer = customArg.Split('=');
   if (customArgBuffer.Length == 2)
   {
    customArgsDict.Add(customArgBuffer[0], customArgBuffer[1]);
   }
   else
   {
    Debug.LogWarning("GetCustomArguments() - The custom argument [" + customArg + "] seems to be malformed.");
   }
  }
  return customArgsDict;
 }
}
Update the existing "PerformBuild.cs" file in the "Editor" folder to read arguments passed in to the build:
using System;
using System.Collections.Generic;
using System.IO;
using UnityEditor;
using UnityEngine;
using Object = UnityEngine.Object;

public class PerformBuild
{
 [MenuItem("Automated/Automated iOS Build")]
 static void CommandLineBuildOnCheckinIOS()
 {
  const BuildTarget target = BuildTarget.iPhone;
  
  string[] levels = GetBuildScenes();
  const string locationPathName = "iOSbuild";
  const BuildOptions options = BuildOptions.None;

  DeleteStreamingAssets();
  BuildPipelineBuildAssetBundle(target);

  string environment = CommandLineReader.GetCustomArgument("Environment");
  string version = CommandLineReader.GetCustomArgument("Version");
  string bundleID = CommandLineReader.GetCustomArgument("BundleID");

  string bundleIdentifier = (0 == bundleID.Length) ? "com.studiosstevepro.angrybots" : bundleID;
  BuildPipelineBuildPlayer(levels, locationPathName, target, options, environment, version, bundleIdentifier);
 }

 // similar code as previous post.

 private static void BuildPipelineBuildPlayer(string[] levels, string locationPathName, BuildTarget target,
  BuildOptions options, string environment, string version, string bundleIdentifier)
 {  
  PlayerSettings.productName = "Angry Bots";
  PlayerSettings.bundleIdentifier = bundleIdentifier;

  string bundleVersion = (0 == version.Length) ? "1.0" : version;
  PlayerSettings.bundleVersion = bundleVersion;

  if (0 != environment.Length)
  {   
   string fullPath = Application.streamingAssetsPath + "/Environment.txt";
   File.WriteAllText(fullPath, environment);  
  }
  
  String error = BuildPipeline.BuildPlayer(levels, locationPathName, target, options);
  if (!String.IsNullOrEmpty(error))
  {
   throw new System.Exception("Build failed: " + error);
  }
 }
}
Finally, write Unity3D client code that consumes the Patch Server build environment variable accordingly:
public class Utilities
{ 
 internal static string PATCH_SERVER_ROOT
 {
  get
  {
   if (String.IsNullOrEmpty(_patchServerRootData))
   {
    SetBuildEnvironmentVariables();
   }

   return _patchServerRootData;
  }
 }

 private static void SetBuildEnvironmentVariables()
 {
  TargetEnviroment target = TargetEnviroment.Test;
  string environment = target.ToString();

  string fullPath = Application.streamingAssetsPath + "/Environment.txt";
  if (File.Exists(fullPath))
  {
   environment = File.ReadAllText(fullPath);
  }

  try
  {
   target = (TargetEnviroment) Enum.Parse(typeof (TargetEnviroment), environment, true);
  }
  catch
  {
   // Must do this was as Enum.TryParse not available in .NET 2.0.
  }

  switch (target)
  {
   case TargetEnviroment.Test:
    _patchServerRootData = "http://TestPatchServer/"; break;

   case TargetEnviroment.Beta:
    _patchServerRootData = "http://BetaPatchServer/"; break;

   case TargetEnviroment.Prod:
    _patchServerRootData = "http://ProdPatchServer/"; break;
  }
 }

 private static string _patchServerRootData;

 enum TargetEnviroment
 {
  Test, Beta, Prod
 }
}
From here, any reference to the Patch Server from the Unity3D client code can be made simply like this:
  string patchServerRoot = Utilities.PATCH_SERVER_ROOT;
Note: ensure Optimization Api Compatibility Level is updated to accommodate Enum.Parse();

Navigate to File menu | Build Settings | Platform | Switch Platform | Other Settings
Change Optimization Api Compatibility Level from .NET 2.0 Subset to .NET 2.0.

Build Job
Make one small change to the existing AngryBotsIOS build job: pass arguments to the executeMethod.
 Unity3d installation name Unity3D
 Editor command line arguments:
 -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinIOS  -CustomArgs:Environment=Beta;Version=2.0;BundleID=com.studiosstevepro.angrybotsbeta

Click Save button | Click Build Now. Deploy Angry Bots to iOS device as per previous instructions.
Congratulations! Angry Bots should be installed as before but now target Beta build environment.

TestFlight
TestFlight Beta Testing makes it easy to invite users to test pre-release versions of iOS apps before they are released to the App Store. Up to 1,000 beta testers can be invited using just their email addresses.

Here is a good tutorial on how to setup TestFlight and how to use TestFlight to distribute the beta app:
1. Create an account on TestFlight with "Developer" option checked.
2. Log in to the TestFlight web site and "Add a team" for your app.
3. Add teammates, either testers or developers or available public.
4. Ask teammates to register iOS devices following the instructions.
5. Once accepted / registered, device information will be available.
6. Add devices / UDIDs to Devices section of Apple web site portal.
7. Update provisioning profile and install onto Jenkins build server.

Upload API
Extend the Automated Build process to upload latest revision to TestFlight via Upload API.

API Token
API Token gives access to Upload API and any endpoints for future TestFlight releases.
In TestFlight, click User icon (top right) | Account Settings | Upload API | API Token.

Team Token
Team Token is the token used for the Upload API "Team" parameter (in Jenkins).
In TestFlight, click Team name (top right) | Edit Info | Team Info | Team Token.

Distribution List
Create distribution list to notify testers once latest revision upload to TestFlight:
In TestFlight, click People tab | click "+" icon next to "Distribution Lists" to add.

Apple
Add same device information to the Apple portal and include in Distribution provisioning profile.
Navigate to Apple portal | iOS Apps | Log In | Certificates, Identifiers & Profiles | Devices.
Click the "+" icon (top right) to Register new device(s): Enter a valid name and the UDID.

Once all the device(s) have been registered, include in the Distribution provisioning profile:
Provisioning Profiles | Distribution | Click profile | Edit | Select All devices | Click Generate.

Download the updated Distribution provisioning profile and install onto Jenkins build server:
Copy /Users/username/Downloads/*.mobileprovision file to here:

1. /Users/username/Library/MobileDevice/Provisioning\ Profiles/
2. /Users/Shared/Jenkins/MobileDevice/Provisioning\ Profiles/
3. /System/Library/MobileDevice/Provisioning\ Profiles/

Upgrade
Mac OS/X El Capitan 10.11 upgrade magically removes the MobileDevice folder under /System/Library/
Thus, without provisioning profiles stored there, Jenkins system process will fail to sign the iOS builds;

Therefore provisioning profiles must be copied back to /System/Library/MobileDevice folder as before.
However, when you attempt to copy them back you may receive the "Operation not permitted" error.

Solution: Reboot Mac | Press cmd + R on boot | OS X Utilities prompt | choose Utilities menu | Terminal
Type csrutil disable | Type reboot. Now when back in to you can manually copy the MobileDevice folder
Launch Terminal window. Type cp /Users/Shared/Jenkins/MobileDevice /System/Library/MobileDevice

Jenkins
Finally, update Jenkins automation: install the TestFlight plugin and add new nightly build job.

Manage Plugins
Main dashboard | Click Manage Jenkins | Click Manage Plugins | Click "Available" tab.
Ensure the following plugin is installed: Testflight plugin (upload to testflightapp.com)
Restart Plugins
Manually stop daemon: sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
Manually start daemon: sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

Configure System
Main dashboard | Click Manage Jenkins | Click Configure System. Enter the following: Test Flight | Test Flight Tokens
 Token Pair Name  AngryBotsIOSTestFlight
 API Token  Enter API Token from Test Flight
 Team Token  Enter Team Token from Test Flight

New Item
Main dashboard | Click New Item | Enter Item name: AngryBotsIOSTestFlight.
Click radio button next to "Copy existing Item". Enter "AngryBotsIOS". Build Triggers
For example, build every night at 8pm Sunday-Thursday, ready for next working day.
 Build periodically  Checked
 Schedule  H 20 * * 0,1,2,3,4

Build | Add build step | Invoke Unity3d Editor
 Unity3d installation name Unity3D
 Editor command line arguments:
 -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinIOS  -CustomArgs:Environment=Beta;Version=2.0;BundleID=com.studiosstevepro.angrybotsbeta

Xcode | General build settings
 Target Unity-iPhone
 Clean before build?  Checked
 Configuration Release
 Pack application and build .ipa?  Checked
 .ipa filename pattern  AngryBotsIOSTestFlight

Build | Add build step | Execute shell. Enter the following shell script:
# This is the output IPA file.
filename="AngryBotsIOSTestFlight.ipa"

echo "Copying $filename to /mnt/builds now STARTING..."
echo "Running as :"
whoami

# Navigate to destination.
cd /mnt/builds

# Remove current file if exists.
if [ -f "$filename" ]; then rm "$filename"; fi

# Copy latest version of file.
cp ${WORKSPACE}/output/"$filename" .

echo "Copying $filename to /mnt/builds now COMPLETE..."
echo Finished
Post build Actions | Upload to TestFlight
 Token Pair  AngryBotsIOSTestFlight
 IPA/APK Files (optional)  /mnt/builds/AngryBotsIOSTestFlight.ipa
 Build Notes  Angry Bots IOS no. ${SVN_REVISION}
 Append changelog to build notes  Checked

Click Advanced button | Update the following:
 Distribution List  Enter Distribution List from Test Flight
 Notify Team  Checked

Click Save button. Build job will now upload latest revision to TestFlight automatically!
Note: Beta testers will receive notification once latest revision available for download.

Conclusion
In conclusion, open source continuous integration tools like Jenkins make it easy to successfully build Test / Beta / Prod ready binaries. Additional tools like TestFlight make them even easier to distribute!

Friday, July 4, 2014

Unity3D and Automated Builds

In the previous post, we discussed Unity Remote to lessen the time taken to view constant changes on target platforms. Now, we would like to package game code once to be deployed to multiple devices.

In this post, we use Jenkins: an open source continuous integration tool to perform automated builds.
At the time of this writing, Jenkins is available for Android on PC and both Android / iOS on the Mac.

Let's check it out!

Windows PC
Ensure you have the following pre-requisites installed and setup as per previous post.
 The Java SDK  Environment Variables
 Android SDK  USB Drivers
 Unity 3D
Also, ensure you have an SVN client installed on localhost, for example, Tortoise SVN.

Example
As an example, let's upload Angry Bots into source control (SVN) to demonstrate automated builds.

Add the following "PerformBuild.cs" file to the "Editor" folder to help automate the Android build:
Note: upgrade to Unity Pro licence if you would like to programmatically Build the Player Pipeline.
using System;
using System.Collections.Generic;
using System.IO;
using UnityEditor;
using UnityEngine;
using Object = UnityEngine.Object;

public class PerformBuild
{ 
  [MenuItem("Automated/Automated Android Build")]
  static void CommandLineBuildOnCheckinAndroid()
  {
    const BuildTarget target = BuildTarget.Android;

    string[] levels = GetBuildScenes();
    const string locationPathName = "AngryBotsAndroid.apk";
    const BuildOptions options = BuildOptions.None;

    DeleteStreamingAssets();
    BuildPipelineBuildAssetBundle(target);
    BuildPipelineBuildPlayer(levels, locationPathName, target, options);
  }
 
  private static string[] GetBuildScenes()
  {
    List<string> names = new List<string>();
    foreach (EditorBuildSettingsScene e in EditorBuildSettings.scenes)
    {
      if (e == null) { continue; }
      if (e.enabled) { names.Add(e.path); }
    }
    return names.ToArray();
  }
 
  private static void DeleteStreamingAssets()
  {
    // Delete streaming assets (downloaded from source control).
    string[] filesToDelete = Directory.GetFiles(Application.streamingAssetsPath, "*.unity3d*");
    foreach (string file in filesToDelete) 
    {
      File.Delete(file);
    }
  }

  private static void BuildPipelineBuildAssetBundle(BuildTarget buildTarget)
  {
    string[] assetPaths = AssetDatabase.GetAllAssetPaths();
  
    string pathName = Application.streamingAssetsPath;
    foreach (string f in assetPaths)
    {
      if (!f.Contains("Master Assets")) { continue; }
      Object a = Resources.LoadAssetAtPath(f, typeof(Object));
      if (a == null) { continue; }

      Object[] asset = new Object[1];
      asset[0] = a;
      string assetType = a.GetType().Name;
      if (assetType.Equals("Object")) { continue; }

      string assetName = assetType + "_" + asset[0].name + ".unity3d";
      string fullName = pathName + "/" + assetName;

      const BuildAssetBundleOptions options = BuildAssetBundleOptions.CollectDependencies | BuildAssetBundleOptions.CompleteAssets | BuildAssetBundleOptions.UncompressedAssetBundle;
 
      BuildPipeline.BuildAssetBundle(a, asset, fullName, options, buildTarget);
    }
  }

  private static void BuildPipelineBuildPlayer(string[] levels, string locationPathName, BuildTarget target, BuildOptions options)
  {
    PlayerSettings.productName = "Angry Bots";
    PlayerSettings.bundleIdentifier = "com.studiosstevepro.angrybots";
    PlayerSettings.bundleVersion = "1.0";

    String error = BuildPipeline.BuildPlayer(levels, locationPathName, target, options);
    if (!String.IsNullOrEmpty(error))
    {
      throw new System.Exception("Build failed: " + error);
    }  
  }
}
Jenkins
Download Jenkins Binary Installer and run. Restart computer. Navigate to http://localhost:8080.

Manage Plugins
Main dashboard | Click Manage Jenkins | Click Manage Plugins | Click "Available" tab.
Ensure the following plugins are installed: Subversion Plugin and Unity3d plugin.
Restart Plugins
Start | run | services.msc. Scroll down to Jenkins. Right click. Restart.
Configure System
Main dashboard | Click Manage Jenkins | Click Configure System. Enter the following:
Unity3d | Add Unity3d | Unity3d installations
 Name Unity3D
 Installation directory  C:\Program Files (x86)\Unity\

Jenkins Location
 Jenkins URL  http://buildserver:8080
 System Admin e-mail address  build@mycompany.com

Subversion
 Subversion Workspace Version  1.7

E-mail Notification
 SMTP server  smtp@mycompany.com

New Item
Main dashboard | Click New Item | Enter Item name: AngryBotsAndroid.
Click radio button next to "Build a free-style software project". Click OK.
Configuration
 Project Name  AngryBotsAndroid

Source Code Management
 Subversion Modules  Checked
 Repository URL  https://mycompany.com/svn/AngryBots
 Check-out Strategy  Use 'svn update', with 'svn revert'
Note: you may be prompted to enter SVN source control credentials at this point.
Credentials | Click Add button | Add SVN source control Username and Password.

Build | Add build step | Invoke Unity3d Editor
 Unity3d installation name  Unity3D
 Editor command line arguments:
 -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinAndroid

Build | Add build step | Execute shell
 Command  echo ${WORKSPACE}

Post-build Actions | E-mail Notification
 Recipients Enter email distribution list

Click Apply button | Click Save button | Click Build Now.

Note: during build job, Unity may prompt you to select Android SDK path. Choose D:\Android\sdk

Deploy
Main dashboard | Click AngryBotsAndroid | Build History | Click DateTime link | Console Output.

Scroll to bottom: note the absolute path to AngryBotsAndroid build job workspace entered there.
For example, it may be C:\Program Files (x86)\Jenkins\jobs\AngryBotsAndroid\workspace

Connect Android device to PC. Launch command prompt, navigate to Android APK file and install:
Start | run | cmd | cd "C:\Program Files (x86)\Jenkins\jobs\AngryBotsAndroid\workspace"
adb install -r AngryBotsAndroid.apk

Congratulations! Angry Bots should be installed on your Android device ready to play.

Apple Mac
Ensure you have the following pre-requisites installed and setup as per previous post.
 The Java SDK Environment Variables
 Android SDK  USB Drivers
 Unity 3D  Xcode
Also, ensure you have an SVN client installed on localhost, for example, svnx for Mac.

In order to deploy code to an iOS device, this post assumes you have an Apple Developer ID.
This post also assumes you have a Provisioning Profile setup via the Apple Developer portal.

Archive
Automated builds may actually benefit from archiving final packaged products to a network file share.
That way, packages can be deployed from mounted network file share on to multiple devices directly.

Mount Drive
For example: there is a Shared/builds folder on ARCHIVE-SERVER for automated builds.

Finder | Go menu | Connect to Server... Enter the following Server Address: smb://username:password@ARCHIVE-SERVER/Shared. Click Connect button.

Launch Terminal window to manually mount drive. Type the following (as root):
sudo su | cd /mnt | mkdir builds | cd builds
mount_smbfs //ARCHIVE-SERVER/Shared/builds /mnt/builds

Root User
Important: Jenkins must run as root user in order to have copy permission to mount drive.
Launch Terminal window to elevate Jenkins privileges as root user. Modify configuration: cd /Library/LaunchDaemons | vi org.jenkins-ci.plist | Find <key>UserName</key>
Change <string>Jenkins</string> TO <string>root</string> | Type !wq to Save.

Example
Update the "PerformBuild.cs" file that exists in the "Editor" folder to help automate the iOS build:
Note: upgrade to Unity Pro licence if you would like to programmatically Build the Player Pipeline.
public class PerformBuild
{
  [MenuItem("Automated/Automated iOS Build")]
  static void CommandLineBuildOnCheckinIOS()
  {
    const BuildTarget target = BuildTarget.iPhone;

    string[] levels = GetBuildScenes();
    const string locationPathName = "iOSbuild";
    const BuildOptions options = BuildOptions.None;

    DeleteStreamingAssets();
    BuildPipelineBuildAssetBundle(target);
    BuildPipelineBuildPlayer(levels, locationPathName, target, options);
  }

  // same code as before.
}
Jenkins
Download Jenkins Binary Installer and run. Restart computer. Navigate to http://localhost:8080.

Note: if you receive warning "Jenkins.pkg can't be opened because from unidentified developer"
Choose: System Preferences | Security & Privacy | Click "Open anyway" button | Install.

Manage Plugins
Main dashboard | Click Manage Jenkins | Click Manage Plugins | Click "Available" tab.
Ensure the following plugins are installed: Subversion, Unity3d and the Xcode plugin.

Restart Plugins
The Jenkins Installer sets up a launch daemon listening on port 8080 for all incoming requests.
Therefore, launch Terminal window to manually restart daemon. Type the following (as root): Manually stop daemon: sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
Manually start daemon: sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

Configure System
Main dashboard | Click Manage Jenkins | Click Configure System. Enter the following: Configuration
 Home directory  /Users/Shared/Jenkins/Home

Unity3d | Add Unity3d | Unity3d installations
 Name Unity3D
 Installation directory  /Applications/Unity/Unity.app

Xcode Builder
 xcodebuild executable path  /usr/bin/xcodebuild
 agvtool executable path  /usr/bin/agvtool
 xcrun executable path  /usr/bin/xcrun

Jenkins Location
 Jenkins URL  http://buildserver:8080
 System Admin e-mail address  build@mycompany.com

Subversion
 Subversion Workspace Version  1.7

E-mail Notification
 SMTP server  smtp@mycompany.com

New Item
Main dashboard | Click New Item | Enter Item name: AngryBotsIOS.
Click radio button next to "Build a free-style software project". OK.
Configuration
 Project Name  AngryBotsIOS

Source Code Management
 Subversion Modules  Checked
 Repository URL https://mycompany.com/svn/AngryBots
 Check-out Strategy  Use 'svn update', with 'svn revert'
Note: you may be prompted to enter SVN source control credentials at this point.
Credentials | Click Add button | Add SVN source control Username and Password.

Build Triggers
For example, poll build server every 10 minutes for any source code revisions.
 Poll SCM  Checked
 Schedule  H/10 * * * *

Build | Add build step | Invoke Unity3d Editor
 Unity3d installation name Unity3D
 Editor command line arguments:
 -quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinIOS

Xcode | General build settings
 Target Unity-iPhone
 Clean before build?  Checked
 Configuration Release
 Pack application and build .ipa?  Checked
 .ipa filename pattern  AngryBotsIOS

Xcode | Code signing & OS X keychain options
 Code signing & OS X keychain options iPhone Distribution
Note: leave blank if you do not wish to override the code signing identity in project.

Xcode | Advanced Xcode build options
 Xcode Project Directory iOSbuild
 Build output directory ${WORKSPACE}/output

Build | Add build step | Execute shell. Enter the following shell script:
# This is the output IPA file.
filename="AngryBotsIOS.ipa"

echo "Copying $filename to /mnt/builds now STARTING..."
echo "Running as :"
whoami

# Navigate to destination.
cd /mnt/builds

# Remove current file if exists.
if [ -f "$filename" ]; then rm "$filename"; fi

# Copy latest version of file.
cp ${WORKSPACE}/output/"$filename" .

# Create archive directory if necessary.
testdate=${BUILD_ID}
filedate=`echo "$testdate" | cut -d '_' -f1`

if [ -d "$filedate" ]; then echo "$filedate" exists; else mkdir "$filedate"; fi
cd "$filedate"

# Remove archive file if exists.
archfile=${SVN_REVISION}_"$filename"
if [ -f "$archfile" ]; then rm "$archfile"; fi

# Copy archive version of file.
cp ${WORKSPACE}/output/"$filename" ./"$archfile"

echo "Copying $filename to /mnt/builds now COMPLETE..."
echo Finished
Click Apply button | Click Save Button | Click Build Now.

Errors
iOS automated builds may encounter errors if the build server is not completely setup.

Code Sign Errors
There are no valid certificate/private key pairs in the default keychain
Open "Keychain Access" application. Finder | Applications | Utilities | Keychain Access

Click login tab (top left) | Right click "iPhone Developer YOUR NAME" | Choose Copy
Click System tab | Right click in area | Choose Paste 2 items

Note: you may need to repeat with the "iPhone Distribution YOUR NAME" certificate.

Provisioning profile 'xxxxx-xxxx-xxxx-xxxxx' can't be found
Copy your username mobile device provisioning profile to the shared Jenkins folder:
Launch Terminal window to copy provisioning profile. Type the following (as root):

Copy /Users/username/Library/MobileDevice folder TO /Users/Shared/Jenkins
Copy /Users/username/Library/MobileDevice folder TO /System/Library

Upgrade
Mac OS/X El Capitan 10.11 upgrade magically removes the MobileDevice folder under /System/Library/
Thus, without provisioning profiles stored there, Jenkins system process will fail to sign the iOS builds;

Therefore provisioning profiles must be copied back to /System/Library/MobileDevice folder as before.
However, when you attempt to copy them back you may receive the "Operation not permitted" error.

Solution: Reboot Mac | Press cmd + R on boot | OS X Utilities prompt | choose Utilities menu | Terminal
Type csrutil disable | Type reboot. Now when back in to you can manually copy the MobileDevice folder
Launch Terminal window. Type cp /Users/Shared/Jenkins/MobileDevice /System/Library/MobileDevice

/usr/bin/codesign SecKey API returned: unknown error -1=ffffffffffffffff
Open "Keychain Access" application. Finder | Applications | Utilities | Keychain Access

Click login tab (top left) | Expand "iPhone Distribution" | Right click private key | Get Info
Click the "Access Control" button (top right). Ensure the following options are set here:
Click System tab (mid left) | Expand "iPhone Distribution" | Right click private key | Get Info
Click the "Access Control" button (top right). Ensure the following options are set here:

FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL
If you encounter this error then Jenkins wants to run as its own private user instead of Mac OS/X.
Create slave node on Jenkins thus the Unity process can connect as Mac OS/X user accordingly:

Main dashboard | Click Manage Jenkins | Click Manage Nodes | New Node
 Node name Unity3D
 Dumb slave Checked

Configure
 Name Unity3D
 # of executors 2
 Remote root directory /Users/username/JenkinsNode
 Usage Leave this machine for tied Jobs only
 Launch method Launch slave agents via Java Web Start
 Availability Keep this slave on-line as much as possible

UnityException: Launching iOS project via Xcode4 failed. Check editor log for details
If you are using Unity 4.3.* and Xcode 5.1.* then you may possibly receive this error.

Manually edit Xcode plugin metadata Info.plist file for Unity and add the missing key:
Launch Terminal window. Type the following (as root):

cd /Applications/Unity/Unity.app/Contents/BuildTargetTools
cd iPhonePlayer/Unity4XC.xcplugin/Contents

vi Info.plist | Search for DVTPlugInCompatibilityUUIDs | Add key:
<string>A2E4D43F-41F4-4FB9-BB94-7177011C9AED</string>

UnityException: Launching iOS project via Xcode4 failed. Check editor log for details
If you are using Unity 4.5.* and Xcode 6.0.* then you may possibly receive this error.

Manually edit Xcode plugin metadata Info.plist file for Unity and add the missing key:
Launch Terminal window. Type the following (as root):

cd /Applications/Unity/Unity.app/Contents/PlaybackEngines
cd iOSSupport/Tools/OSX/Unity4XC.xcplugin/Contents

vi Info.plist | Search for DVTPlugInCompatibilityUUIDs | Add key:
<string>C4A681B0-4A26-480E-93EC-1218098B9AA0</string>

UnityException: Launching iOS project via Xcode4 failed. Check editor log for details
If you are using Unity 4.6.* and Xcode 6.2.* then you may possibly receive this error.

Manually edit Xcode plugin metadata Info.plist file for Unity and add the missing key:
Launch Terminal window. Type the following (as root):

cd /Applications/Unity/Unity.app/Contents/PlaybackEngines
cd iOSSupport/Tools/OSX/Unity4XC.xcplugin/Contents

vi Info.plist | Search for DVTPlugInCompatibilityUUIDs | Add key:
<string>A16FF353-8441-459E-A50C-B071F53F51B7</string>

Method
In order to determine which GUID must be added follow steps as outlined here:
Finder | Applications | Right click Xcode | Show Package Contents | Contents
Double click Info.plist. Choose DVT PluginCompatibility UUID | Copy GUID

Next
Finder | Applications | Unity | Right click Unity.app | Show Package Contents
Contents | PlaybackEngines | iOSsupport | Tools | OSX
Right click Unity4XC.xcplugin | Contents | Double click Info.plist
Choose DVT PluginCompatibility UUID | Click + | Paste GUID

Deploy
After the iOS build is successful, deploy packaged game code once to multiple devices:

Connect an iOS device to your PC | Launch iTunes | Click the top left drop down arrow
Add File to Library... | Navigate to \\ARCHIVE-SERVER\Shared | Double click AngryBotsIOS.ipa Click iPhone button | Click Apps button | Click Install button (next to AngryBots) | Click Apply button. Congratulations! Angry Bots should be installed on your iOS device ready to play.

Summary
The only automated build option outstanding here is an Android build deployed on the Mac.
Fortunately, this is simple: clone iOS build job, remove Xcode build step and modify script.

Clone iOS build
Main dashboard | Click New Item | Enter Item name: AngryBotsAndroid.
Click radio button next to "Copy existing Item". Enter "AngryBotsIOS".

Build | Add build step | Invoke Unity3d Editor
 Unity3d installation name Unity3D
 Editor command line arguments:
-quit -batchmode -executeMethod PerformBuild.CommandLineBuildOnCheckinAndroid

Remove Xcode build step
Scroll down to Xcode | General build settings | Click Delete button.

Modify script.
Make the following changes to shell script to accommodate Android packaged game code:
# This is the output APK file.
filename="AngryBotsAndroid.APK"

echo "Copying $filename to /mnt/builds now STARTING..."
echo "Running as :"
whoami

# Navigate to destination.
cd /mnt/builds

# Remove current file if exists.
if [ -f "$filename" ]; then rm "$filename"; fi

# Copy latest version of file.
cp ${WORKSPACE}/"$filename" .

# Create archive directory if necessary.
testdate=${BUILD_ID}
filedate=`echo "$testdate" | cut -d '_' -f1`

if [ -d "$filedate" ]; then echo "$filedate" exists; else mkdir "$filedate"; fi
cd "$filedate"

# Remove archive file if exists.
archfile=${SVN_REVISION}_"$filename"
if [ -f "$archfile" ]; then rm "$archfile"; fi

# Copy archive version of file.
cp ${WORKSPACE}/"$filename" ./"$archfile"

echo "Copying $filename to /mnt/builds now COMPLETE..."
echo Finished
Conclusion
In conclusion, Continuous Integration has many advantages; one of which of course is stability: an automated build process constantly checks state of the code and immediately reports build errors.

The timeliness of this feedback is critical for maintaining stable code. Especially for software being regularly deployed to multiple devices; most of which avail to frequent online mobile store updates!

Sunday, June 1, 2014

Unity3D and Unity Remote

Unity3D is a cross platform game engine to develop video games for web, desktop, consoles and mobile.

During development, game code written in Visual Studio can be tested quickly; directly in the Editor. However, game code will eventually need to be deployed, for example, onto an actual mobile device.

Thus, typical programming sequence, especially for mobile, may be something like this:
 1. Build (or update) scene in Unity3D.
 2. Write game code in Visual Studio.
 3. Test game code in Unity3D Editor.
 4. Switch platform to mobile device.
 5. Build and Run game on device.
 6. Rinse and repeat.

The method to switch platform to build and run game code on mobile device can be a lengthy process;
Fortunately, there is a more efficient way to test game code on mobile devices directly: Unity Remote.

Let's check it out!

Unity Remote
Unity Remote: an application that targets mobile device as remote control for your project in Unity editor. Development is quicker to test through remote control rather than deploy to the device after each change.

At the time of this writing, Unity Remote is available for Android on PC and both Android / iOS on the Mac.

Windows PC
Reference: Unity Remote setup tutorial on You Tube. Outline of steps also available on Unity Answers.

Java SDK
Download Java SDK and install. Unity3D may also require 32-bit JDK installed on 64-bit Windows.

Android Device
Download Unity Remote onto Android device (Samsung Galaxy 4.2.2).
Choose Settings | More (top right) | Developer options and configure.

Ensure the following Developer options are checked:
 1. Stay awake.
 2. USB debugging.
 3. Allow mock locations.

Note: Developer options may be hidden by default! If so then perform the following:
Choose Settings | More (top right) | About device. Tap Build number multiple times.

Android SDK
Download the Android SDK and extract to root folder, for example, D:\Android.
Note: apparently the Unity3D application does not agree with root C:\Android.

Update any additional Android packages required for all target devices as necessary.
For example, it is standard to install the Android SDK Tools and Google USB Drivers:
Launch Eclipse for Android Developer Tools (ADT) from D:\Android\eclipse\eclipse.exe
Window menu | Android SDK Manager | Select Tools, Android APIs, Extras | Install

Finally, set the Android SDK Location if this has not already been set:
Window menu | Preferences | Android | SDK Location: D:\Android\sdk

Environment Variables
Export environment variables for Android SDK and Android Debug Bridge (ADB) tools:
Right click Computer | Properties | Advanced system settings | Environment Variables.

Add new System variable for name ANDROID_HOME as D:\Android\sdk
Extend to Path System variable: ;%ANDROID_HOME%\platform-tools

Launch command prompt: Windows | Start | Run | cmd. Type "adb devices".
List of devices attached should be empty when no Android device attached.

USB Drivers
Connect Android device to PC; Windows should download and install correct USB drivers automatically.
However, on certain Android devices, you may need to download and install the USB drivers manually.

Navigate to the device manufacturer website and download USB drivers as necessary (e.g. Samsung).

After USB drivers installed, launch command prompt: Windows | Start | Run | cmd. Type "adb devices".
List of devices attached should now include hex code for currently connected Android device (as above).

Finally, launch the Unity Remote application on the device before executing the next step: Unity3D.

Unity3D
Launch Unity3D. Choose to create new project or simply open an existing project.
Edit menu | Preferences | External Tools | Android SDK Location | D:\Android\sdk

In order to run the currently opened scene on the Android device, simply perform the following:
Edit menu | Project Settings | Editor | Unity Remote | Device | Choose Any Android Device

Click the Play button. The game will play directly on Android device as well as in Unity Editor!

Apple Mac
Below are instructions to setup Unity Remote on both Android / iOS devices but this time on the Mac.

Java SDK
Download Java SDK and install. Here, there is typically only one option for the Mac OS X.

iOS Device
Download Unity Remote onto iOS device. No additional configuration should be required.

Android SDK
Download the Android SDK and extract to root folder, for example, to root /Android.

Update any additional Android packages required for all target devices as necessary.
For example, it is standard to install the Android SDK Tools and any relevant Extras.

Launch Eclipse for Android Developer Tools (ADT) via Terminal window:
cd /Android/eclipse/Eclipse.app/Contents/MacOS then execute ./eclipse

Window menu | Android SDK Manager | Select Tools, Android APIs, Extras | Install

Finally, set the Android SDK Location if this has not already been set:
ADT menu | Preferences | Android | SDK Location: /Android/sdk

Environment Variables
Export environment variables for Android SDK and Android Debug Bridge (ADB) tools:
Launch Terminal window. Add new ANDROID_HOME system variable and extend Path.

export ANDROID_HOME=/Android/sdk
export PATH=$PATH:$ANDROID_HOME/platform-tools
While Terminal window is still open, type in the following: "adb devices".
List of devices attached should be empty when no devices are attached.

USB Drivers
Connect Android / iOS device to Mac. Apple should detect device without any additional configuration.

Launch Terminal window. Type "adb devices". Hex code for connected Android device should be listed.
Finally, launch the Unity Remote application on the device before executing the next step: Unity3D.

Unity3D
Launch Unity3D. Choose to create new project or simply open an existing project.
Unity menu | Preferences | External Tools | Android SDK Location | /Android/sdk

In order to run the currently opened scene on the iOS device, simply perform the following:
Edit menu | Project Settings | Editor | Unity Remote | Device | Choose Any iOS Device

Click the Play button. The game will play directly on iOS device as well as in Unity Editor!
Note: choose Any Android Device for equivalent experience on Android as outlined above.

Conditional Compilation
Unity3D includes preprocessor directives to compile and execute sections of platform dependent code:
#if UNITY_IPHONE
  //Execute script code when Build Settings Platform: iOS.
#endif

#if UNITY_ANDROID
  //Execute script code when Build Settings Platform: Android.
#endif
Note: targeting single code base to multiple devices may benefit from Device Factory implementation.

Therefore, here is full list of all current Unity Remote scenarios:
 1. Connect target device(s) to machine
 2. Launch Unity Remote on device(s).
 3. Launch Unity3D game project.
 4. Choose platform + device.
 5. Click Play button.

Choose PLATFORM:  File menu | Build Settings | Platform | Switch Platform
Choose DEVICE:      Edit menu | Project Settings | Editor | Unity Remote | Device

 Unity3D on PC Choose Any iOS Device Choose Any Android Device
 PLATFORM: iOS Scene NOT render on iOS
 Execute UNITY_IPHONE code
 Scene renders on Android
 Execute UNITY_IPHONE code
 PLATFORM: Android Scene NOT render on iOS
 Execute UNITY_ANDROID code
 Scene renders on Android
 Execute UNITY_ANDROID code

 Unity3D on MAC Choose Any iOS Device Choose Any Android Device
 PLATFORM: iOS Scene will render on iOS
 Execute UNITY_IPHONE code
 Scene renders on Android
 Execute UNITY_IPHONE code
 PLATFORM: Android Scene will render on iOS
 Execute UNITY_ANDROID code
 Scene renders on Android
 Execute UNITY_ANDROID code

At the time of this writing, Unity Remote will not work when iOS device attached to PC unfortunately.
Important: pressing play will always execute UNITY_EDITOR code regardless of platform or device!

Summary
Unity Remote does lessen time taken to view constant (minor) changes on target mobile device(s).
However, you will eventually you will need to deploy game code to the device and test directly!

Again, this can be channelled through Unity3D: choose target mobile platform then Build and Run.
On the other hand, the process to build and run once per device can also be a lengthy process.

Therefore, it would be better to package game code once to be deployed to multiple devices.
That is, the process of Automated Builds. This will be the topic in the next post.