No Comply – a WebGL, Audio-API Demo: click here to watch this demo.

Over the past few months I have been working with Mozilla’s Audio API Team to create a WebGL/Audio API Demo for the release of Firefox 4.

We wanted to create something a little different for this demo so we enlisted the help of Glitch artist Kraddy. Kraddy’s new Album is called Labyrinth and has some sweet tracks. Kraddy’s concept for the track was a “Double-Dragon” style aesthetic in which Kraddy gets sucked into his own 8-bit nightmare. We spent a week in California at Mozilla, working on the proof of concept and the tools we would use to create the demo.

Once Kraddy’s help had been enlisted, we needed an artist. Omar Noory stepped up to the 8-bit challenge and did a wonderful job turning our half-baked ideas into awesome sprite sheets for the game engine. I encourage you to check out Omar’s site, “These are Things“: his graphics are simply stunning.

Less Work, More Tools


Blender 3D – click here to browse levels in WebGL (requires Firefox 4).

One thing we realized pretty quickly, was that with a little tweaking, Blender3D could be used to export Collada files that we could load into CCliffe’s very own Cubic-VR WebGL engine. CCliffe championed this effort, which greatly sped up development time and also allowed us to push the WebGL demo envelope, baking animation paths and exporting Bullet-Physics data from Blender’s Game Engine.

While CJ hacked away on Blender, Corban Brook get to work on the Audio API section of the demo. Corban built a circular audio sequencer that mapped all of the kicks and claps in the track so we could use them to fire visual events in time with the music.

With all the other bases covered, I jumped into the sprite work. I started by defining a data structure for my sprite files that I would store in JSON format. I wanted to store the image data in the actual JSON file itself, so I used Boaz Sender’s Drag & Drop Data URI tool to run a few experiments. Once I bounced the data-structure off the rest of the team and made some tweaks, I was ready to start building some play-back methods.

Storing Sprite Data in JSON Format


Sprite Viking

After defining a data format for my sprites, I created a play-back script called “Sprite Viking Player”. Sprite-Viking-Player renders a sprite’s image data onto a 2D Canvas element based on the current selected action. With a data-format and a player in working commission, the last phase of my sprite creation process was to build a sprite editor. For this I used CJ’s Cubic-VR.js WebGL engine as a base, and sprinkled some jQuery UI magic on top. Once I had the editor working, the production line from artist to game-engine was very quick. Unfortunately this sprite editor still has a few bugs and missing features, but it worked well for the task at hand.

What Did We Learn?

We learned many subtle differences between the Chrome and Firefox WebGL implementations. As usual, Mozilla squashed some very difficult bugs and performance improvements as result of the demo-work. We learned how to usefully profile code that was pushing the limits in almost every area of the application. This helped the team to develop a balanced asset pre-loading system, an important piece of the puzzle required to keep things running smoothly across the Web.

It may look all glitz-and-glam when you see these demos released, but the amount of hard work that goes in to producing them is almost immeasurable. We learned so much from doing this demo it would be impossible to record everything into one blog post. I only wish I had the time to cover some of the outstanding solutions produced by David Humphrey and Bobby Richter, but you will have to find them on irc.mozilla.org/audio and ask them about some of the awesome things they are working on.

I can’t thank Mozilla and the Audio team enough for their hospitality during the making of this demo. You guys are awesome!