Saturday, July 4, 2020

Python Setup Cheat Sheet

Python is an interpreted high-level and general purpose programming language. Python 2.0 was released in 2000 but officially discontinued in 2020. Python 3.x is now the preferred version for most projects e.g. v3.7.

Python is commonly used in artificial intelligence and machine learning projects with the help of libraries like TensorFlow, Keras, Pytorch and Scikit-learn. There is also open source Python distribution Anaconda used for data science and machine learning applications that aims to simplify package management and deployment.

Let's check it out!

Install Python
Install Python on Windows, Mac OS/X and Linux. Install pip as the de facto standard package-management system. Install Anaconda for future AI projects. Finally, install + configure an IDE for Python programming.

Follow instructions here how to install Python 3 on Windows. Download latest version of Python for Windows including IDLE pip + documentation. Add Python to PATH variable making it easier to configure your system.

Mac OS/X
Python is installed by default on most Mac OS/X systems. Launch terminal + type python and pip to confirm.

Python is installed by default on most Linux systems. Launch terminal + type python. However you may also need to install pip. Update your ~/.bashrc file as necessary to use Python 3.7 by default instead of Python 2.
 Install + Update + Aliases Python3  Ubuntu Linux unable to locate package python-pip
 sudo apt install python3-pip
 python -m pip install --upgrade pip
 echo "alias python=python3" >> ~/.bashrc
 echo "alias pip=pip3" >> ~/.bashrc
 sudo apt-get install software-properties-common
 sudo apt-add-repository universe
 sudo apt-get update
 sudo apt-get install python3-pip

Verify which python version and which pip version are installed and location where these are both installed:
 python --version  which python
 pip --version  which pip

Install Anaconda
Follow instructions to install open source Python distribution Anaconda for Windows and Mac OS/X and Linux
 Windows  %USERPROFILE%\Anaconda3
 Mac OS/X  /anaconda3
 Linux  /anaconda3

Add the following four environment variables: System | Advanced system settings | Environment Variables:
 %USERPROFILE%\Anaconda3  %USERPROFILE%\Anaconda3\libs
 %USERPROFILE%\Anaconda3\Scripts  %USERPROFILE%\Anaconda3\Library\bin

Mac OS/X
Follow all prompts from the install wizard. Anaconda should install at /anaconda3 by default with aliases set.

Launch terminal as root user. Type bash ~/Downloads/ For consistency with the Mac set install location to /anaconda3. After install launch Anaconda navigator. Enter the following:
 source ~/anaconda*/bin/activate root

Update your ~/.bashrc file as necessary to prefer Anaconda Python especially for data science AI + ML work.
 alias python=/anaconda3/bin/python
 alias pip=/anaconda3/bin/pip

Finally, create Anaconda desktop shortcut: create the following Anaconda.desktop at /usr/share/applications/ Add the text below. Enter sudo echo "PATH=$PATH:/anaconda3/bin" >> /etc/environment at the terminal.

[Desktop Entry]
Restart Linux. From settings type "Anaconda" to prompt Anaconda Navigator + resume projects from there.

Verify which Anaconda version is installed using conda --version command and where installed where conda.

Install PyCharm
PyCharm is an integrated development environment used specifically for Python. Install PyCharm during the Anaconda install or from navigator. Otherwise install PyCharm for Windows and Mac OS/X and Linux directly.

Launch PyCharm. Click "Configure" cog drop down | Settings. Set the base interpreter to match the Python location for Anaconda. Click cog | Show All | Click "+" | Virtualenv Environment. Enter the following details:

 Windows  %USERPROFILE%\Anaconda3\python.exe
 Mac OS/X  /anaconda3/bin/python
 Linux  /anaconda3/bin/python

Configure cog | Plugins. Install pylint as a code analysis tool that follows the recommended PEP8 style guide. Restart IDE. Pylint is handy tool to check Python code especially for developers used to static code compiler!

Track active item in solution explorer similar to Visual Studio: Click cog to right of Project and Always Select Opened File. Hit shift key twice for quick search. Remember Rename files and folder is in the Refactor menu!
 Ctrl + F12  Prompt popup to list all the methods in the class
 Ctrl + Shift + N  Prompt popup to search for text found in all files
 Ctrl + Shift + I  View definition of function or method in the class
 Ctrl + Click function  Navigate to function definition or method in class
 Ctrl + Alt + Left arrow  Navigate backwards to previous location
 Ctrl + Alt + Right arrow  Navigate forwards to the next location

On Linux navigate may actually be Shift + Alt + arrow keys instead of Ctrl due to keymap shortcut conflicts!

Hello PyCharm
Launch PyCharm. Create New Project | "HelloPyCharm". Enter the following details as Anaconda interpreter:

PyCharm uses virtualenv tool to create an isolated Python environment. Virtualenv creates venv folder in the current directory with Python executable files and a copy of pip to install other modules and packages.

Create simple "". Enter Python code print('Hello World'). Right click file | Run ""HelloWorld".

Module Not Found
Update script | Import module not currently installed e.g. import numpy. Run script: ModuleNotFoundError

Launch PyCharm terminal. Enter python -m pip install numpy to install module. Re-run script with success.

Alternatively, execute pip freeze to dump all required modules for project into requirements.txt and install:
 pip freeze > requirements.txt
 pip install -r requirements.txt

Code Sample
Let's test drive develop a simple code sample as a Python module that could be deployed as Python package.

A module is a single Python script file whereas a package is a collection of modules. A package is a directory of Python modules containing an additional file to distinguish from a directory of Python scripts.

Launch PyCharm. Create New Project | "PackagePyCharm". Configure Python Anaconda interpreter above. Right click PackagePyCharm project | New | Python Package | "MyPackage". Create other top level folders.

Create sub folders src and tests beneath "MyPackage". Create requirements.txt file + beneath "MyPackage" also. Finally, create and under src and under tests.
 import unittest
 from MyPackage.src.module import add_one

 class TestSimple(unittest.TestCase):

    def test_add_one(self):
        result = add_one(5)
        self.assertEqual(result, 6)

 if __name__ == '__main__':
 def add_one(number):
    return number + 1

Right click inside | Debug 'Unittests for'. Alternatively, Terminal command:

python -m unittest discover MyPackage

If the Terminal reveals Run 0 tests then ensure package has setup in every relevant sub folder. Finally, enter dependencies for requirements.txt file and e.g. numpy. Install package at terminal:  requirements.txt
 import setuptools

 with open("", "r") as fh:

    description="My test package.",

pip install MyPackage/.

Finally, we could also replicate the unit test code directly on the REPL. Select Terminal tab. Enter commands:

 >>> from MyPackage.src.module import add_one
 >>> result = add_one(5)
 >>> print(result)

Code Linting
Linting highlights syntactical and stylistic problems in Python source code which helps identify and correct subtle programming errors. In PyCharm, choose Pylint tab and click Play button to list any errors in code.

Alternatively, select Terminal and type specific linter like flake8 to check Python source code against PEP8 coding style programming errors. If flake8 is not installed then simply type pip install flake8 at Terminal.

To summarize, we have a simple setup for Python programming on Windows, Mac OS/X and Linux. There is much to explore e.g. Python 3.8. However, we'd like to setup Python more for AI machine learning projects. This will be topic of the next post.

Tuesday, June 30, 2020

Candy Kid Code Complete

Candy Kid is a simple maze chase video game originally programmed by Grandstand Leisure from New Zealand in September 1984. The game was written in BASIC programming language on Sega SC-3000.

In 2015, Candy Kid was re-written in C#/.NET and XNA for Windows PC and ported to iOS / Android using MonoGame. After finding the original source code on SMS Power!, Candy Kid was re-written again in 8-bit.

Inspired from previous posts on Sega Console Programming and devkitSMS Programming Setup + Sample, Candy Kid is my fifth 8-bit video game written in C / Z80 assembler to target Sega Master System (SMS).

Let's check it out!

Note: previous titles published for the Master System include 3D City, Simpsons Trivia + Platform Explorer.
Download source code here.

Sega Retro Gaming post documents how to play video games like Candy Kid SMS using the Fusion emulator.
devkitSMS Programming Setup post documents development environment required to build C / Z80 source.
Eat all the candy to pass each level. Eat all bonuses to maximize your score! The 3x Candy "Mama" enemies Pro / Adi / Suz have different passive and aggressive personalities and alternate between scatter and attack.

Here are a list of Tools and frameworks that were used in the development of this project:
 Programming  devkitSMS
 Compiler  sdcc 3.6
 Assembler  WLA-DX
 IDE  Visual Studio 2015
 Languages  C / Z80
 Graphics  BMP2Tile 0.43 / GIMP 2 /
 Music  Mod2PSG2 / vgm2psg
 Emulators  Emulicious / Fusion / Meka

ROM Hacking
You can hack this ROM! Download + dump CandyKid into Hex Editor, e.g. HxD, and modify bytes:
 0x004F  Debugger  Used to show debugging info for game.
 0x0050  Invincible  Non-zero value enables invincibility.
 0x0051  FullBoost  Non-zero value enables maximum boost.
 0x0052  Trees Type  Set value to 1=Show otherwise 2=Kill.
 0x0053  Exits Type  Set value to 1=Open otherwise 2=Shut.
 0x0054  Difficulty  Set value to 1=Easy otherwise 2=Hard.
 0x0055  Game Speed  Set value to 1=Slow otherwise 2=Fast.
 0x0056  World No.  Set start World no currently 1 to 10.
 0x0057  Round No.  Set start Round no currently 1 to 10.
 0x0058  Music Off  Set 0=Music to play otherwise silent.
 0x0059  Sound Off  Set 0=Sound to play otherwise silent.

  • There are 4x different bonuses: 100 / 200 / 400 / 800 pts. Double bonus points after level 70.
  • Player will receive extra 2,000 pts after eating all the candy and collecting all bonuses in level.

  • Press button 2 five times during Title screen and you'll be invincible each game this is actioned.
  • Press and hold button 2 during game play to action Game Over and force quit out of the game.
  • Press and hold button 2 on Splash screen to reset High score and all options previously saved.

Extra special thanks goes to sverx for the devkitSMS. Plus StevePro Studios would like to thank eruiz00 and haroldoop for sharing source code from SMS Homebrew projects. Many ideas here were used in this project!

Thanks to SMS Power! for making this video. Here is the official Candy Kid Facebook page and Blogger label.

To summarize, Candy Kid has had a very interesting journey after first appearing on the Sega SC-3000 in 1984 to being ported to Windows PC, iOS + Android in 2015 and finally the Sega Master System in 2020.

In fact, after porting the original game in 2015 the ultimate goal was always to eventually back port Candy Kid to the Master System. Thanks to the awesome Indie game dev scene this goal has now been achieved!

Sunday, May 31, 2020

devkitSMS Programming Sample IV

In 2017, we checked out devkitsms to setup productive development environment to build 8-bit retro video games for the Sega Master System (SMS). In 2019, we improved build process to develop Platform Explorer.

Though after five years of SMS development we would like to connect more the C language with underlying Z80 assembly code. Look for opportunities to consolidate 8-bit retro game development to build Candy Kid!
Let's check it out!

Candy Kid video game was originally programmed by Grandstand Leisure from New Zealand in September 1984. The game written in BASIC on the Sega SC-3000. Thanks to the awesome SMS Power! community Candy Kid is available on Sega Master System in 2020 Coding Competition. Download source code here.

Follow all instructions from the previous post: this documents how to setup the pre-requisite software.
Note: ensure you have downloaded and installed the devkitSMS and Small Device C Compiler [SDCC].

Here is a checklist of tasks to complete in order to try and improve the existing development environment:
  • Connect more the C language and Z80 assembly code via emulation
  • Implement various code architectural updates from the devkitSMS
  • Apply techniques to resolve vulnerabilities debugging source code
  • Explore strategies to confront the 32KB ROM game code capacity

In 2013, we checked out Sega Console Programming with goal to build retro game in Z80 assembler for the Sega Master System. However, the process to write Z80 assembly language from scratch seemed daunting.

In 2019, since collaborating with Calindro, we now have the opportunity to understand relationship between C source code and the generated Z80 assembly: Emulicious will decorate Z80 code in the debugger with its corresponding C function!

Launch Emulicious. Tools menu | ensure "Reopen Tools" option is checked. Launch the debugger. File menu | ensure "Load Sources automatically" is checked + Run menu | ensure "Suspend On Open" is checked. Close
Ensure the file is not deleted in build script. E.g. both output.sms and are built and loaded into Emulicious. The maps C source code functions to the underlying Z80 assembly code.

Search for main function. Ctrl + F | Find "_main" Ctrl+click "main" to navigate there. Ctrl+click any function name to navigate + inspect Z80 assembly code built for that function. Click Alt+Left to navigate back. Nice!

The following coding changes and enhancements have been made throughout Candy Kid development cycle:

Leverage the built-in SMS_VDPFlags for fast sprite collision detection. If value equals 0x20 then at least two sprites have collided then implement slow(er) collision detection by checking if sprite rectangle(s) intersect:

gamer_collision = devkit_isCollisionDetected();
if ( 0 != gamer_collision )             // Fast.
{                                       // Slow.
    for( enemy = 0; enemy < MAX_ENEMIES; enemy++ )
        eo = &global_enemy_objects[ enemy ];
        if( go->posnX > eo->posnX )     { dx = go->posnX - eo->posnX; }
        else                            { dx = eo->posnX - go->posnX; }

        if( go->posnY > eo->posnY )     { dy = go->posnY - eo->posnY; }
        else                            { dy = eo->posnY - go->posnY; }

        if( dx <= distance && dy <= distance ) { gamer_collision = enemy; break; }

In 2018, Duckslayer Adventures by haroldoop gave a great example of how levels could be stored externally in text files then converted into byte arrays using folder2c. The byte arrays are then interpreted for drawing:
 0  Blank
 1  Trees
 2  Candy
 3  BonusA
 4  BonusB
 5  BonusC
 6  BonusD
  // Level0101

Persisting important data like Hi Scores and player settings across game sessions makes for better gaming experience. Here is the Candy Kid implementation with SRAM support currently included in the devkitSMS:

#define MAGIC   0xACE0B004
#define FINAL   0xFF

// Global variable.
struct_storage_object global_savegame_object;

unsigned char engine_storage_manager_available()
    struct_storage_object *savegame = ( struct_storage_object* ) ( devkit_SMS_SRAM() );
    unsigned char foundMagic;

    foundMagic = MAGIC == savegame->Magic;
    return foundMagic;
void engine_storage_manager_read()
    struct_storage_object *savegame = ( struct_storage_object* ) ( devkit_SMS_SRAM() );
    struct_state_object *st = &global_state_object;

    st->state_object_high_score = savegame->save_high_score;
//  ...
void engine_storage_manager_write()
    struct_storage_object *savegame = ( struct_storage_object* ) ( devkit_SMS_SRAM() );
    struct_state_object *st = &global_state_object;

    savegame->Magic = MAGIC;
    savegame->save_high_score = st->state_object_high_score;
//  ...
    savegame->terminal = FINAL;
void engine_storage_manager_erase()
    struct_storage_object *savegame = ( struct_storage_object* ) ( devkit_SMS_SRAM() );

    savegame->Magic = 0x00000000;
    savegame->save_high_score = DEF_HI_SCORE;
//  ...
    savegame->terminal = FINAL;

Buffer overflow [or overrun] is an anomaly where writing data to buffer overruns the buffer's boundary and overwrites adjacent memory. In Candy Kid, the player can be killed by any one of the three Mama enemies; Actors 0-2. But Kid can also be killed by death tree ie "actor" 4. When Kid is killed we initially reset the "kill" enemy unconditionally. Thus if killed by death tree enemy "4" would be reset hence a buffer overrun occurs! Big thanks to Calindro + sverx for helping debug this issue. The solution is to check enemy kill before reset.

The following challenges were encountered with 32KB ROM game code capacity during development cycle:

The previous input manager had clean interface that's "Easy to use correctly" and "Hard to use incorrectly" But this suffered from code bloat + additional costly bytes. Therefore, inject key test for simpler interface:

#include "input_manager.h"
#include "..\devkit\_sms_manager.h"

static unsigned int curr_joypad1 = 0;
static unsigned int prev_joypad1 = 0;

// Public methods.
void engine_input_manager_update()
    prev_joypad1 = curr_joypad1;
    curr_joypad1 = devkit_SMS_getKeysStatus();
unsigned char engine_input_manager_hold( unsigned char data )
    return curr_joypad1 & data && !( prev_joypad1 & data );
unsigned char engine_input_manager_move( unsigned char data )
    return curr_joypad1 & data;

In 2017, Astro Force by eruiz00 gave great example on how to store static data in fixedbank.c to scale out more game content and bank switch on-the-fly without consuming the precious 32KB ROM for game code.

Candy Kid attempts to replicate this strategy, unfortunately still hitting 32KB capacity for game code. Next step could be store function pointer arrays in fixedbank.c like Astro Force to conserve more precious bytes!

const unsigned char bossX_object_speed[] = { 1, 1, 2, 1, 1, 1, 3, 3, 1, 1, 1, 1 };
const unsigned char bossX_object_delay[] = { 2, 2, 3, 1, 1, 1, 2, 2, 3, 3, 2, 2 };
const unsigned char enemy_object_image[] = { 0, 2, 4, 12, 14, 16, 24, 26, 28, 36, 38, 40 };
const unsigned char enemy_object_speed[] = { 2, 3, 1, 3, 2, 1, 3, 2, 1, 1, 2, 3 };
const unsigned char enemy_object_delay[] = { 3, 3, 1, 2, 3, 1, 2, 1, 2, 2, 3, 3 };
const unsigned char enemy_object_hands[] = { 90, 80, 70, 60, 95, 85, 75, 65, 99, 90, 80, 70 };
const unsigned char enemy_object_waits[] = { 80, 60, 40, 20, 85, 65, 45, 25, 90, 70, 50, 30 };
const unsigned char enemy_object_tours[] = { 64, 72, 80, 88, 68, 76, 84, 92, 72, 80, 88, 96 };
const unsigned char enemy_object_kills[] = { 65, 73, 81, 89, 69, 77, 85, 93, 73, 81, 89, 97 };
const unsigned char gamer_object_image[] = { 0, 2, 4, 12, 14, 16 };
const unsigned char gamer_object_speed[] = { 1, 2, 3, 3 };
const unsigned char gamer_object_delay[] = { 1, 1, 2, 1 };
// TODO potentially include function pointer arrays here also!

In previous projects our build.bat file had always used SDCC compiler switch to favor code speed. However, as Candy Kid was struggling to stay within 32KB ROM limit, unfortunately was necessary to favor code size:

sdcc -c -mz80 --opt-code-size --peep-file peep-rules.txt --std-c99 main.c

In 2009, we implemented the Command Design pattern to record sprite movement in Henway to playback for debugging and demo modes. Therefore, we would also like to implement the Command Design pattern for Candy Kid. The motivation here is record enemy movement for Reinforcement Learning in video games.

void engine_command_manager_add( unsigned int frame, unsigned char command_type, unsigned char args );
void engine_command_manager_execute( unsigned int frame );
void screen_play_screen_update( unsigned char *screen_type )
    // Implement logic to boost and move Candy Kid gamer.
    engine_command_manager_add( frame, command_type_gamer_speed, gamer_boost );
    engine_command_manager_add( frame, command_type_gamer_mover, gamer_direction );

    // Implement logic to boost and move Candy Mama enemies.
    engine_command_manager_add( frame, command_type_enemy_speed, ( enemy | ( enemy_boost << 4 ) ) );
    engine_command_manager_add( frame, command_type_enemy_mover, ( enemy | ( enemy_direction << 4 ) ) );

    // Execute all commands for this frame.
    engine_command_manager_execute( frame );
Rather than boost and move actors directly from game events we add these as commands to an array and execute them accordingly. However, even when storing 255 commands this quickly added to 1KB memory!

#define MAX_FRAMES 255
static unsigned int command_object_frames[ MAX_FRAMES ];
static unsigned char command_object_cmds[ MAX_FRAMES ];
static unsigned char command_object_args[ MAX_FRAMES ];
Unfortunately, due to competition time constraints and hitting 32KB capacity for game code the Command Design pattern was not fully implemented. Next steps: code refactor and perhaps storing less commands.

In 2016, 3D City was built using an old version of the z88dk which worked on emulators but real Master System hardware experienced graphics glitches. Subsequently, the updated version of z88dk integrates devkitSMS which also works on real hardware. In fact, this is how Galactic Revenge was built by eruiz00. This was preferred because the libraries are optimized in assembly language and produce smaller object code.

Therefore, this seemed a promising option for Candy Kid especially late in the development cycle when the game hit 32KB capacity for game code. Unfortunately, z88dk does not support the following two APIs used:
 SMSlib.h  extern volatile unsigned char SMS_VDPFlags;
 PSGlib.h  void PSGResume (void);

To summarize, the development process to build Candy Kid, like all previous SMS projects, has been very educational + has provided numerous opportunities to collaborate more with the SMS Power! community.

However, the biggest challenge in completing Candy Kid was to stay within the 32KB ROM game code limit. Here is a short list of current + future options:
  • Build game code in Z80 assembly directly for complete control of the machine code byte output
  • Refactor duplicate code to store generic code in function pointer arrays accessible by fixed bank
  • Integrate z88dk and devkitSMS that produces smaller object code by using only supported APIs
  • Upgrade from 8-bit Master System to 16-bit Mega Drive and bypass 32KB hard limit altogether

In fact, the final option is the most unique: much like how the Sega Master System has the devkitSMS to support homebrew game development using C language, the Sega Mega Drive has the SGDK. Therefore, would be interesting to transition many Sega Retro Gaming ideas developed from 8-bit onward to 16-bit!

Wednesday, January 1, 2020

Retrospective XI

Last year, I conducted a simple retrospective for 2018. Therefore, here is a retrospective for 2019.

2019 Achievements
  • Streamline Sega Master System retro game development using the devkitSMS
  • Win the Sega Master System annual coding competition with Platform Explorer
  • Apply devkitSMS programming skills to Mega Drive prototypes using the SGDK
  • Upgrade MonoGame version cross platform for future 3D graphics and shaders
  • Complete Linux installation process using virtual machine on Windows and Mac
  • Integrate new scripting languages including Python, Powershell and Terrarform
  • Expand cloud computing experience: Amazon Web services and Azure DevOps
  • Learn from Data Scientists and Machine Learning Engineers strategies at work
Note: winning the Sega Master System annual coding competition is a unique achievement!

2020 Objectives
  • Setup Sega dev + debugging environment for MegaDrive similar to Master System
  • Build MonoGame project cross platform containing 3D models and custom shaders
  • Integrate new software deployment techniques for containerization + orchestration
  • Continue AI and Machine Learning application to personal and professional projects

Earlier in 2019, I gave presentation on Game Development. I believe this provides excellent experience for Software Engineering in general because algorithms must be written efficiently to achieve consistent frame rates. Therefore, after working with Sega Master System would now like to experiment with the Mega Drive!

20 Predictions about Software Development trends in 2020 include: Infrastructure as all roads lead to Cloud, Containerization [Docker] + Orchestration [Kubernetes] and Microservices as REST APIs will be mainstream.

However, the big buzz continues to be centered around: Artificial Intelligence, Machine Learning and how can Big Data be leveraged by companies to improve their customer experiences and gain competitive advantage.

After giving the presentation earlier in 2019, I was advised afterwards about a new interesting development connecting Artificial Intelligence and Video Games now described as Reinforcement Learning in Video Games.

Therefore, this seems excellent motivation to investigate Machine Learning Software such as TensorFlow for various applications of ML such as Reinforcement Learning and see if this can be applied to Video Games J

Friday, November 15, 2019

Mobile Video Game Updates

In 2015, we built Candy Kid as a simple maze chase video game on Windows PC using an older version of MonoGame published on Android and iOS. In 2018, we built 3D City "Shoot 'em up" using similar process.

Now both Google and Apple require apps + games be 64-bit compliant. Apple will stop support for 32-bit on iOS 11.0. Plus, Google require updates on Google Play to target minimum Android 9 (API level 28) or higher.

All these factors mandate upgrades to our mobile games to be rebuilt using latest version of MonoGame 3.7.
Let's check it out!

Too Long; Didn't Read
  1. Install all pre-requisite software on Windows + Mac. Check for updates to get latest dependencies
  2. Launch Android + iOS projects. Update all settings to increment build version numbers and codes
  3. Rebuild + upload the updated binaries to relevant online stores then tag all archived source code

Reference previous instructions for Android and iOS ports. This post assumes software already published. Install the following software on Windows and Mac. Install "Mobile development with .NET" for Xamarin.
  Windows   [Android]   Mac Os/X   [iOS]
Note: Check for updates on all Visual Studio IDEs to get latest the frameworks tools and dependencies.

If, for any reason, there are issues updating any IDEs mentioned here then simply Run as Administrator.

Previous posts used older versions of MonoGame which meant the Content Pipeline was arguably not built correctly. Upgrade to MonoGame 3.7 and build the Content Pipeline correctly using the Content.mgcb file.

Launch the MonoGame Pipeline tool. Navigate to project Content folder. Open Content.mgcb. Click Content root node. Under Properties select relevant Platform. Create following folders and add content accordingly:

Import any ASCII text files or XML configuration files. Set the Build Action in the Properties section to Copy.

Typically import pre-built XNB binary files from previous games e.g. Emulogic.xnb. Set Build Action to Copy. However, if you create custom spritefont files copy the TTF file also otherwise "Could not find font file" error.

Import all music and sound effects as MP3. Some WAV files sounded strange as built sound effects. Convert WAV files to MP3. Set Build Action to Build. Set Processor to Song for music otherwise set to Sound Effect.

Import all 2D images as PNG. Some JPG files rendered strange as built textures. Convert JPG files to PNG. Also, convert BMP files to PNG. Set Build Action to Build. Set Processor to Texture for all imported images.

On November 1, 2019 apps + games on Google Play required to target Android 9 (API level 28) or higher. After this date, Google Play Console will prevent any APK updates submitted with targetSdkVersion < 28.

Therefore, install Android 9 (API level 28) or higher. Launch Android Studio. File | Settings... | Appearance and Behavior | System Settings | Android SDK. Note default location may be different from Visual Studio:
 Android Studio  %USERPROFILE%\AppData\Local\Android\Sdk
 Visual Studio [VS]  C:\Program Files (x86)\Android\android-sdk

Actually, it may be easier to align Android SDK location in Android Studio to that of Visual Studio because all Android game code will be built using MonoGame from Visual Studio. Therefore, update Android SDK path:
Ensure SDK Tools are also installed. Verify SDK Platforms installed in "platforms" sub-folder and SDK Tools installed in "build-tools". Finally, add "platform-tools" folder to System %PATH% to use adb at cmd prompt.

Currently VS2019 does not include MonoGame project templates. Therefore, launch VS2017 and create new MonoGame Android project e.g. 3D City. Close and re-open in VS2019. The benefits to be explained shortly.

Assume previous version 3D City 1.0.0 already exists on Google Play. Import existing content per previous build under the Content folder as explained above. Import existing code. Right click project and Properties:

Compile using Android version (Target Framework). Choose Android 9.0 (Pie) as minimum Android version.

Application Manifest
 Key  Value  Element  Attribute
 Application name  3D City  application  android:label
 Package name  com.steveproxna.x3dcity  manifest  package
 Application icon  @drawable/Icon  application  android:icon
 Version number  2  manifest  android:versionCode
 Version name  1.1.0  manifest  android:versionName
 Minimum Android version  Android 9.0 (API Level 28 - Pie)  uses-sdk  android:minSdkVersion
 Target Android version  Android 9.0 (API Level 28 - Pie)  uses-sdk  android:targetSdkVersion

<?xml version="1.0" encoding="utf-8"?>
<manifest package="com.steveproxna.x3dcity" android:versionCode="2" android:versionName="1.1.0">
  <uses-sdk android:minSdkVersion="28" android:targetSdkVersion="28" /">
  <application android:label="3D City" android:icon="@drawable/Icon">
    <meta-data android:name="android.max_aspect" android:value="2.1" />

Application Options
Click the Advanced button. Target the following 2x Supported architectures: armeabi-v7a and arm64-v8a.

Currenlty, Android 10.0 (Q) is available although this API level 29 seems to be missing from Visual Studio Tools menu | Android | Android SDK Manager. However, here is a workaround to circumvent the problem:

Launch Android Studio. Install Android 10 (API level 29) as above. Ensure android-29 exists in Android SDK "platforms" folder and 29.0.2 in "build-tools". Launch Visual Studio 2019. Update Android version in project:

Launch VS2019. Open MonoGame Android project. Attach Android device. Select Release build configuration. Clean and rebuild solution. Right click project | Archive... After the archive is generated choose Distribute...

Ad Hoc
Select distribution channel: Ad Hoc. Select Signing Identity or Create Android Keystore. Save APK. Enter the password. Open Folder. Navigate to "signed-apks" subfolder. Here is the signed APK that can be installed:
# Start | run | cmd
# cd signed-apks
adb devices -f install *.apk

Google Play
Select distribution channel: Google Play. Select Signing Identity. Continue. Click "+". Login to Google Play | Settings | API access. Select existing OAuth client. Download JSON and extract the Client ID + Secret here:

If you receive any of the following 403 errors while attempting to upload latest Android build then read on:

This release is not compliant with the Google Play 64-bit requirement
This can occur if you have not selected arm64-v8a in the Android project properties Application Options. If after selecting arm64-v8a you receive NoAudioHardwareException then must upgrade MonoGame to 3.7.1.

The apk must be signed with the same certificates as the previous version
This can occur when you have signed previous Android build using keystore typically on different computer. Export Android keystore as needed and ensure critical information like keystore alias and password are safe.

Here is a good overall guide for finding the Android keystore. Below is quick summary for all configurations:
 Windows  %USERPROFILE%\AppData\Local\Xamarin\Mono for Android\Keystore\alias\alias.keystore
 Mac OS/X   $HOME/Library/Developer/Xamarin/Keystore/alias/alias.keystore

Don’t forget to backup your keystore file! Here is how to retrieve the alias and do not forget the password:
# Search for Alias name in the output from the following command
cd %USERPROFILE%\AppData\Local\Xamarin\Mono for Android\Keystore\alias
keytool -list -v -keystore alias.keystore

Apple will stop support for 32-bit apps on iOS 11.0. Therefore, any 32-bit apps will need to be upgraded to 64-bit. Also, Apple may require paid apps be upgraded within every two years to remain on the App Store.

Sign in Ensure all developer provisioning profiles use certificates for Xcode 11 or later:

Currently VS2019 does not include MonoGame project templates. Therefore, launch VS2017 and create new MonoGame iOS project e.g. 3D City. However, do NOT click "Pair to Mac" here because you may be prompted to install Mono version incompatible with this version of Visual Studio. Instead, close and re-open in VS2019.

Assume previous version 3D City 1.0.0 already exists on the App Store. Import existing content per previous build under the Content folder as explained above. Import existing code. Right click project and Properties:

 CFBundleName  3D City
 CFBundleDisplayName  3D City
 CFBundleIdentifier  com.steveproxna.3dcity
 CFBundleVersion  1.1.0
 CFBundleShortVersionString  1.1
 MinimumOSVersion  7.0
 UISupportedInterfaceOrientations  UIInterfaceOrientationLandscapeLeft
 UIStatusBarHidden  true
 UIRequiresFullScreen  true
 UIDeviceFamily  1, 2
 XSAppIconAssets  Assets.xcassets/AppIcon.appiconset

Currently there seems to be inconsistency with AppIcon assets creation on Windows and Mac. Therefore, launch MonoGame iOS project in Visual Studio for Mac. Open Info.plist | Click "Use Asset Catalog" here.

This action creates the AppIcon.appiconset under Assets.xcassets folder whereas Windows creates under Media.xcassets which means App icon may not appear on iOS device! Also, ensure Content.json is correct.

Launch Visual Studio for Mac. Open MonoGame iOS project. Attach iOS device. Select Release configuration. Clean and rebuild solution. Right click project | Archive for Publishing. After the archive is generated choose Sign and Distribute...

Ad Hoc
Select distribution channel: Ad Hoc. Choose provisioning profile | Publish | Save IPA file | Reveal in Finder. Launch Xcode | Window | Devices + Simulators. Select iOS device. Click "+". Navigate and open IPA file.

App Store
Launch browser. Sign in to iTunes Connect. Select published app e.g. 3D City. Click "+" Version or Platform. Choose iOS. Enter new Store Version Number | Create. Enter "What's New in this Version" text. Save data.

Sign in Apple ID web site. In Security section under "APP-SPECIFIC PASSWORDS" click "Generate Password". In the popup window enter, for example "3DCity" | Create. Apple will generate a new app specific password.

Select distribution channel: App Store | Upload. Choose provisioning profile. Enter Apple ID and App Specific Password. Next | Publish. Choose file name | Save IPA. This process should upload build to iTunes Connect.

Finally once build has completed processing select it in iTunes Connect. Save. Complete following questions:
 Export Compliance
 Have you added or made changes to encryption features since your last submission of this app?

 Advertising Identifier
 Does this app use the Advertising Identifier (IDFA)?


If you receive Publishing Failed error "Invalid Toolchain: You app was built with unsupported version of Xcode or SDK" then you must upgrade to Xcode 11.2.1 because Apple deprecated Xcode 11.2 after Nov 5th 2019.

Right click MonoGame iOS project | Archive... | App Store | Save IPA file. Upload on Mac Application Loader:

Also, if you receive Publishing Failed error "Failed to parse PList data type" then simply increment the build version number CFBundleVersion in Info.plist. Align project properties. Rebuild and upload to iTunes again.

If you receive any of the following crashes while attempting to run latest iOS build on device then read on:

Got a SIGABRT while executing native code. This usually indicates a fatal error in the mono runtime used
For some strange reason the following code with IDictionary<Byte, T> worked previously but now crashes. Fortunately, there is a workaround to this issue: simply use the Int16 type as the key instead of type Byte.
// This code will crash on iOS device!
private IDictionary<Byte, Vector2> BasePositions { get; private set; }
private IDictionary<Byte, Rectangle> BasePositions { get; private set; }

// This code will work without crash.
private IDictionary<Int16, Vector2> BasePositions { get; private set; }
private IDictionary<Int16, Rectangle> BasePositions { get; private set; }

Microsoft.Xna.Framework.Content.ContentLoadException: 'The content file was not found.'
This general exception can occur when MonoGame project has not built the Content as described above i.e. New MonoGame project using version 3.7. Open Content.mgcb. Add all content relative to this content file.

Also, I believe it helps to fully qualify Content path and remove default code to set Content RootDirectory:
// Delete or comment out default code.
// Content.RootDirectory = "Content";
// Fully qualify path to load content e.g. Image.png
Texture2D image = Content.Load("Content/Textures/Image");

To summarize, all games built in the past using MonoGame were 2D. However, now that all frameworks tools and dependencies are upgraded with latest MonoGame installed using correct build pipeline then we are in a good position to assemble 3D graphics and games on Windows deployed to Android and iOS cross platform!

Thursday, October 31, 2019

Linux Setup Cheat Sheet II

In the previous post, we discussed various options to set up Ubuntu Linux. Now let's install some software.
Let's check it out!

First, let's list some general purpose commands that can be applied + executed anywhere. Launch Terminal:
 Command  Description
 hostname  Confirm host computer name
 whoami  Confirm user executing command
 sudo su  Run as administrator
 sudo apt-get update  General all-purpose update
 sudo apt-get -f install  Install outstanding packages
 sudo dpkg -i *.deb  Install any Debian package
 sudo tar -xvzf *.tar  Extract tar ball [tape archive] file
 sudo add-apt-repository *  Add package to repository
 sudo add-apt-repository --remove *  Remove package repository
 shutdown -r now  Shuts down Ubuntu Linux

By default, Java may not be installed but is a good idea to install because a lot software may be dependent. Launch Terminal and enter the following commands:
sudo apt install openjdk-11-jre-headless  # version 11.0.3+7-1ubuntu2~19.04.1, or
sudo apt install default-jre              # version 2:1.11-71
sudo apt install openjdk-8-jre-headless   # version 8u212-b03-0ubuntu1.19.04.2
sudo apt install openjdk-12-jre-headless  # version 12.0.1+12-1
sudo apt install openjdk-13-jre-headless  # version 13~13-0ubunt1

Repeat if you would like to install javac but install JDK this time instead of JRE. Enter these new commands:
sudo apt install openjdk-11-jdk-headless  # version 11.0.3+7-1ubuntu2~19.04.1, or
sudo apt install default-jdk              # version 2:1.11-71
sudo apt install openjdk-8-jdk-headless   # version 8u212-b03-0ubuntu1.19.04.2
sudo apt install ecj                      # version 3.16.0-1
sudo apt install openjdk-12-jdk-headless  # version 12.0.1+12-1
sudo apt install openjdk-13-jdk-headless  # version 13~13-0ubunt1

Here is a list of general software applications that I believe are handy to have installed. Launch Terminal:
sudo apt-get install git
sudo apt-get install subversion
sudo apt-get install curl
sudo apt-get install python
sudo apt-get install python3

Download Google Chrome as default browser. Save *.deb file in Downloads folder and install from Terminal:
cd ~/Downloads
sudo dpkg -i google-chrome-stable_*.deb
By default, certain packages may not be installed for this software. If this is true then enter the commands:
sudo apt-get update
sudo apt-get install libgconf2-4
sudo apt-get install libnss3-1d
sudo apt-get install libxss1
sudo apt-get -f install

Source Control
In the previous post, we setup a folder share and made it possible to copy + paste files across from host to virtual machine. However, it may be easier to install source control client to simply pull + push files directly.

Download GitEye is available completely free but only includes very limited features. Good for starting point:
cd ~/Downloads
mkdir GitEye
mv GitEye*.zip GitEye
cd GitEye
unzip GitEye*.zip
rm GitEye*.zip

Download SmartGit as full-featured and similar product to SourceTree but it requires a commercial licence:
cd ~/Downloads
tar -xzvf smartgit*.tar.gz
rm smartgit*.tar.gz
cd smartgit/bin

Download GitKraken as a great compromise. This software is full-featured like SourceTree available for free: Note: GitKraken can launch Terminal [a handy feature] but seems only from using Alt+T keyboard shortcut.
cd ~/Downloads
sudo dpkg -i gitkraken*.deb

Now we'd like to build our own software so let's install various Integrated Development Environments [IDEs].

VS Code
Visual Studio Code is a popular lightweight IDE available cross platform on Windows, Mac OS/X and Linux. Install directly from Ubuntu Software. Otherwise, launch the Terminal and enter the following commands:
curl | gpg --dearmor > microsoft.gpg
sudo mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
sudo sh -c 'echo "deb [arch=amd64] stable main" > /etc/apt/sources.list.d/vscode.list'

sudo apt-get update
sudo apt-get install code # or code-insiders
Note: if there's an issue with Visual Studio Code install then try sudo apt-get --fix-broken install + try again.

Geany is a lightweight IDE using the GTK2 toolkit and may seem to be popular for C development on Linux. Launch Terminal and enter the following commands:
sudo add-apt-repository ppa:geany-dev/ppa
sudo apt-get update
sudo apt-get install geany geany-plugins-common

Code Blocks
Code Blocks seems to be popular IDE for Sega Genesis development> Install directly from Ubuntu Software.

CLion is a commercial IDE from JetBrains for C/C++ available cross-platform on Windows, Mac OS/X and Linux using CMake build system. Download CLion. Launch Terminal and enter these following commands:
cd ~/Downloads
tar -xvzf CLion-*.tar.gz
cd CLion*
cd bin

Specifically .NET Core is now open source and available cross platform on Windows, Mac OS/X and Linux! Launch Terminal and enter the following commands:
wget -q -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb

# Install .NET Core Runtime
sudo apt-get install apt-transport-https
sudo apt-get update
sudo apt-get install dotnet-hosting-2.0.6

# Install the .NET SDK
sudo apt-get install apt-transport-https
sudo apt-get update
sudo apt-get install dotnet-sdk-2.2

# Verify installation
dotnet --version
dotnet --list-sdks
dotnet --list-runtimes
If you receive an error message similar to Unable to locate package dotnet-sdk-2.2 then refer to this guide.

After installing IDEs for C/C++ let's install a static code analysis debugging and profiling tool like Valgrind:
sudo apt install valgrind
# Example "HelloWorld"
cd ~/CLionProjects/HelloWorld/cmake-build-debug
valgrind ./HelloWorld --lead-check=yes

Also, if you would like to take a screenshot then use [Alt +] PrintScr to save in output ~/Pictures directory.

Finally, let's install some emulators in order to play + test 8-bit and 16-bit SEGA retro video games on Linux.

Kega Fusion
Download Kega Fusion as emulator for general Sega retro video games. Launch Terminal + run commands:
cd ~/Downloads
sudo dpkg -i Kega-fusion_3.63-2_i386.deb
sudo apt-get -f install

Gens GS
Download Gens GS as emulator for Sega Genesis retro video games. Launch Terminal and run commands:
cd ~/Downloads
sudo apt-get update
sudo dpkg -i Gens_2.16.7_i386.deb
sudo apt-get -f install

Download Emulicious as emulator for Sega retro video games with great debugger! Launch Terminal + run:
cd ~/Downloads
mkdir Emulicious
mv Emulicious
cd Emulicious
sudo unzip
java -jar Emulicious.jar

Finally, there is an awesome YouTube video documenting how to setup Doom on Linux. All notes found here!

Now we have Ubuntu installed with a ton of Ubuntu software we are in a good spot to progress with Linux. Being open source it is also possible to modify the Linux kernel which would be a phenomenal next step!

Sunday, September 15, 2019

Linux Setup Cheat Sheet

Linux is everywhere! Linux is free open source operating system based on Unix OS. Linux powers 90% of the world's fastest servers thus is the backbone of the Internet. Linux dominates mobile platforms and the cloud. Let's check it out!

Install Ubuntu Linux on Windows using VirtualBox. Download VirtualBox and install. Follow all the prompts. Next, follow all instructions here to create the virtual machine to host Ubuntu. I have tweaked some inputs:
 Step  Key  Value
 VM Name and OS Type  Version  Ubuntu (64-bit)
 Memory  Base Memory Size  8 GB
 Virtual Hard Disk  Start up Disk  Create new hard disk
 Create New Virtual Disk  File type  VDI (VirtualBox Disk Image)
 Virtual disk storage details  Storage details  Fixed size
 Virtual disk file location and size  Size  100 GB

Complete virtual hard drive: click Create. Download Ubuntu 19.04. In VirtualBox, click Settings | Storage | Controller IDE | Empty | Optical Drive: IDE Secondary Master. Click icon | Choose Virtual Optical Disk File...

Navigate to downloaded Ubuntu ISO file | OK. In VirtualBox, double click Ubuntu [Powered Off] to Start the virtual machine. Next, follow all instructions here for the Ubuntu installation. I have tweaked some inputs:
 Step  Key  Value
 Welcome  Option  Install Ubuntu
 Keyboard layout*  English  English (US)
 Updates and other software  What apps would you like to install  Normal installation
 Installation type  Computer has no detected OS  Erase disk and install Ubuntu
 Where are you?  City  Leave default selection
* IMPORTANT do not change keyboard layout here as can corrupt buttons! You can always change it later.

The final install step "Who are you?"
 Key  Value
 Your name  Steven Boland
 Your computer's name  stevepro-VirtualBox
 Pick a username  stevepro
 Choose a password  password
 Confirm your password  password

VT-x is not available (VERR_VMX_NO_VMX) After Windows 10 Update
You may receive this error on Windows 10 especially after a Windows Update. If so then try the following;
  • Launch the Terminal command prompt Run as Administrator
  • Start | Programs | right click Command Prompt | Run as administrator
  • Type the following command: dism.exe /Online /Disable-Feature:Microsoft-Hyper-V
Restart computer when prompted. Note: Windows may perform additional updates one or two more times.

Hyper-V Manager
If you are still experiencing issues running Ubuntu Linux on Windows using VirtualBox because of the Microsoft hypervisor then try host Ubuntu using the Hyper-V Manager. Start | run | Hyper-V Manager.

Next, follow all instructions here to create a Linux virtual machine on Windows 10 using Hyper-V. Verify Hyper-V is supported with systeminfo.exe command and that Hyper-V is enabled via Windows features.

In Hyper-V Manager choose Action menu | Virtual Switch Manager | New virtual network switch | External. Create Virtual Switch e.g. "MyVirtualSwitch". Choose network adapter as External Network | Apply | OK.

Create virtual machine: Action menu | New | Virtual Machine | Ubuntu. Use default location. Use default Generation 1. Choose "MyVirtualSwitch" | Finish. Follow steps as above to complete Ubuntu installation.

Windows Subsystem for Linux
Windows Subsystem for Linux [WSL] is an optional feature in Windows that provides a kernel compatibility layer based on Ubuntu. This layer allows Linux programs to run on a Windows 10 version of the Bash shell.

Determine if your Windows 10 64-bit version supports WSL: Settings | System | About. Version >= 1607. Follow YouTube instructions: Enable Developer Mode via Settings | Update & Security then For developers.

Enable Windows Subsystem for Linux: Control Panel | Programs and Features | Turn Windows features on. Finally, launch Microsoft store. Search for Ubuntu. Install and Launch. Set username and password to begin. From cmd prompt type bash to shell out. Windows files are mapped from /mnt directory thus C:\ => /mnt/c

Mac OS/X
Install Ubuntu Linux on Mac OS/X using VirtualBox. Download VirtualBox and install. Follow all the prompts. Next, follow all instructions here to create the virtual machine to host Ubuntu. Repeat install like Windows!

This section assumes Ubuntu Linux is installed irrespective of the platform. Now, let's do some basic setup: Login and verify an Internet connection: Settings (top right hand side) | System Settings | Network | Wired.

Typically, the initial display will not max out monitor resolution so let's fix that: Show Applications (bottom right) | Settings | Display. Choose resolution that makes sense for monitor size. Use Landscape orientation. In VirtualBox, View menu. Choose Scaled Mode. Drag screen borders to maximize remaining monitor space.

Guest Additions
In order to copy + paste between the host and virtual machine enable Guest Editions. In VirtualBox Devices menu | Insert Guest Additions CD image. Follow all prompts and choose "Y" accordingly. Restart Ubuntu VM.

After installation, choose Ubuntu virtual machine | Settings | General | Advanced. Ensure that both "Shared Clipboard" and "Drag n' Drop" are set to Bidirectional. If this doesn't work then launch Terminal commands:
sudo apt-get update
sudo apt-get install virtualbox-guest-x11

Settings | Region & Language. Choose Language English (US) and Format for keyboard layout that you like. Also, you may switch the Command and Control key if you are on a Mac for consistency. Launch Terminal:
cd /usr/share/X11/xkb/symbols
sudo cp pc pc_bak
sudo gedit pc
key <LCTL> {    [ Super_L       ]   };
key <LWIN> {    [ Control_L     ]   };
key <RCTL> {    [ Super_R       ]   };
key <RWIN> {    [ Control_R     ]   };
sudo rm -rf /var/lib/xkb/*

Following Keyboard, update Terminal keyboard shortcuts for copy and paste consistency. Launch Terminal menu | Preferences | Shortcuts | Edit. Choose Copy and press Ctrl+C and choose Paste and press Ctrl+V. Also, handy Files Manager shortcuts: Ctrl+D to create [bookmark] link to folder and Ctrl+L to show path.

Remove any unwanted applications from Favorites. Add any new applications to Favorites accordingly. E.g.: Show Applications | Search "Terminal". Double Terminal to launch. Right click Terminal | Add to Favorites.

Folder Share
In order to share files between the host and virtual machine enable folder share: Show Applications | Files | Other Locations. Connect to Server "smb://host-pc" | Connect. Enter Windows Username and Password and WORKGROUP for Domain. Otherwise, choose Domain as the Local Area Network [LAN] domain accordingly.

Screen Lock
Finally, you may like to disable screen lock for long periods of time. Settings | Privacy | Screen Lock | Off.

If, for any reason, changes made don't replicate then it may be necessary to simply restart virtual machine.

Now that Ubuntu Linux is installed on computer whichever configuration chosen it is time to install software! This will be the topic in the next post.