Dynamic Asset Downloading in Real Racing 2

In the beginning...

Starting with Real Racing 2 we were looking to add a special GUI for managing the players account and save games. We also intended (and did!) use this UI across other games, so it is built as HTML and JavaScript instead of C++ and OpenGL like the games. It would have been difficult to insert ourselves into the input and rendering pipeline, while also supporting other languages so it was a lot easier just to pop up a web browser and do everything there.

We also supported making server requests from this embedded JavaScript, they went through the same system as normal game requests so had no real limit on what could be done there. This allowed us to create a reusable interface for core functionality (account management, save game backup and restore) that can be themed to each game simply. And allowing for extra functionality to be added per game if needed, for example, Real Racing, Spy Mouse and Flight Control Rocket implemented their own leaderboards using this tech.

Special thanks to Touch Arcade for this screenshot. Since the servers are no longer online, the leaderboards are now lost to the ages.

All the assets for this UI (HTML, JavaScript, images) are shipped with the game and loaded off disk to reduce load times. During development of this feature, the edit and test loop was quite painful, we would have to edit the HTML, rebuild the game in XCode, redeploy the whole game to the device and launch it before we could see our changes.

So in an attempt to save time and restore my sanity, I developed a system to automatically refresh these files if the change, when launching the game or multitasking. This chopped a lot of steps from the process and saved a lot of time. I could now just edit the files, and refresh them quickly on the device.

Eventually to save the bandwidth of downloading every file every time, this was improved so that the device would be able to report what files it has, and the server can send back any updates needed.

The game itself also had a change, so that when a file is loaded, it would check the asset download folder to see if the file exists there, otherwise it will load the asset that shipped with the game. This simple change now allowed us to do a lot, by being able to substitute any asset or assets in the game, by simply serving new ones from the server.

Flipped BMW Logo

Just as we were about to publicly release the game, in fact after the final submission had been made to the App Store, we found out that the BMW badge on one of the cars in game was the wrong way around. The blue and white checker pattern was flipped, so we would not be able to get approval from BMW to release the game.

Here is the car in question, with a blue arrow pointing to the badge that was flipped. But note that the logo is actually correct here!

This was the first big success of the asset replacement system, as we were able to seamlessly fix the logo, by downloading a new asset when our players first launched the game, well before they even had a chance to see the car for the first time.

The most straightforward way to fix the issue would be to fix the texture that is drawn onto the car, but that was a large file, almost 1mb if I recall correctly. So having every player download it was going to be a lot of data. So we managed to find a more efficient fix by changing the 3D mesh of the car instead, flipping the texture coordinates over to flip the logo. This fix was about a tenth of the size.

Fixing Broken Save Games

A few months after release, some players reported bugs where the game would crash after completing a race. We eventually worked out that there was a fixed array of results that was being appended to after every race and these players had simply ran out of space.

The fix was straightforward enough, just keep the best result for every event, and remove the rest from the array. But this would require us to ship an update and wait for it to pass review, so the affected players would either have wait a week for the fix to arrive before they can continue playing, or delete their saved game and start over from the beginning.

But since we already had the tools to backup and restore a players save game, we had a third option to remotely fix the players save games.

Using asset replacement, we added a new button to the HTML UI that would do three things.

• Using the existing backup functionality, upload the save game to our servers.
• Send a new request to the server to fix the players most recent save. The game devs created a tiny command line program based off the C++ game code that would load a save, strip out the unnecessary results and save that file back to disk. So a new endpoint was written on the server to pull the players most recent save from the database, run the program to correct it, then update the database with the corrected save file.
• Again using existing functionality, the client then restores save game from the server to the client, so now the local save file is fixed!

This fix allowed us to have a fix out within the day for our players without having to rush out a new build of the game that would have been delayed up to a week with QA and App store reviews.

AB Testing

Once the game was up and running for a while we eventually extended it to support AB testing. It was somewhat rudimentary, we'd select a control and experiment group of players as they launched the game for the first time. Then for the experiment group we would send a different set of game assets that would change how the data driven game worked. For example, we could make cars cheaper or give more rewards. Then after some time, we could compare the two groups of users in our analytics database, and see what affect the changes had.