Jul 18 jacob

Terrain Tech Launched

The terrain rendering tech I’ve been working on with Bruce has been integrated into an app called WingX (article from Aero News). An intro to WingX from Bruce:

WingX is a product used by pilots for flight planning and navigation while giving them access to the huge amount of data they deal with. This includes federal regulations, maps, weather, imaginary boundaries where flight is prohibited, radio navigation beacon frequencies, terminal approach procedures, and much more. Traditionally, all pilots carried a large brief case full of paper books, maps, and printouts for all but the most trivial flights. This has all now been reduced to simply carrying an iPad and running software like WingX. Even commercial pilots are using tools like this now because it is so much more convenient to carry, update, and use.


Terrain engine integrated into WingX

Bruce describes why we worked on this project:

The motivation for putting synthetic vision into the cockpit is primarily safety, particularly for general aviation pilots who typically have less flight hours and have faced fewer heavy work load situations than commercial and certainly military pilots. Synthetic vision helps improve a pilot’s situational awareness in poor weather conditions or reduced visibility. If you look at the statistics, many general aviation (small plane) pilots have lost their lives due to something called ‘Controlled Flight into Terrain’. This basically means the pilot didn’t realize he was flying into the side of a mountain. I became keenly aware of this type of accident when I got my pilot’s license and always wanted to create something affordable that any student pilot could use to reduce these kinds of risks.

Previous posts on the development of this product:

www.ba3.us
www.hiltonsoftware.com

Jul 4 jacob

Expensive Pixel Shader

Combining a smaller frame buffer and multisampling we were able to hit some of our performance goals and solve some aliasing problems at the same time.  We have been working the terrain project for three months now and many features were added without a close eye on performance.

After some experiments on the iPad 1 (which was running at 11 fps) we find our or pixel shader is the bottleneck. Reducing the size of the frame buffer dimensions by 50% increases the framerate to 32fps.  This is great but it looks horrible and doubly so in motion.

Aliasing was a problem we’ve been fighting for a long time.  So we combined the resolution tests with some multisampling tests.  Here is a chart showing a chunk of a screenshot:

Here is a great resource for how to enable multisampling:
http://db-in.com/blog/2011/05/all-about-opengl-es-2-x-part-33/

For the scale I started writing some upsampling code but it turns out UIView does this for you.  The transform property, if set before the frame is changed, will change the bounds without changing the frame.  Here is my code where the device scale is either 0.5 or 0.75:

float scale = [self getDeviceScale];
[self setTransform:CGAffineTransformMakeScale(1/scale, 1/scale)];

Some iPad 1 numbers:

  • 100% scale, no multisampling: 11 fps
  • 50% scale, no multisampling: 32 fps
  • 50% scale, 4x multisampling: 29 fps

For the iPad 2:

  • 100% scale, no multisampling: 51 fps
  • 75% scale, 4x multisampling: 51 fps

For now we are going with 50% scale, 4x multisampling for the iPad 1 and 75% scale, 4x multisampling for the iPad 2.  It’s hard to see the difference between the two 50% scale shots in the chart but the temporal aliasing is horrible without multisampling.

Jun 20 jacob

Cocos2D Prototype: Shooting

 
I made some progress on the prototype and I wanted to share the movement and shooting snippets.

When I was working in XNA most of my logic was done in an update loop. The Cocos2D actions neatly replace a lot of that code with an event sequence that you construct and run. This event-based approach was easier for me to wrap my mind around even for simple behaviors. Here is an example of the movement code:

- (void)moveTo:(CGPoint)destination
{
    [self stopAllActions];

    float distance = ccpDistance(center, destination);
    float duration = distance / speed;
    [self runAction: [CCMoveTo actionWithDuration:duration position:destination]];

    [self pointAtLocation:location];
}

And here is a slightly more complicated action for shooting:

- (void)shootAtTarget:(Vehicle*)target
{
    [self stopAllActions];

    id fire = [CCCallFuncND actionWithTarget:self selector:@selector(fire:data:) data:target];
    id wait = [CCDelayTime actionWithDuration:0.3];
    id sequence = [CCSequence actions:fire, wait, nil];
    [self runAction:[CCRepeatForever actionWithAction:sequence]];

    CGPoint location = [target position];
    [self pointAtLocation:location];
}

The shoot function builds up a sequence: a call to a fire function, which spawns a projectile, and then a delay. The sequence is set to repeat until another action comes along and stops it.

I’m looking forward to trying some more complex behavior trees.

Jun 6 jacob

Prototype in Cocos2D vs XNA, Part 1

Jonathon Rippy gave a presentation at the last Triangle CocoaHeads meeting on Cocos2D that made getting something up and running look so easy that I had to try it.  I began porting a simple movement test that I did with XNA a few months back.

Up until this point, I hadn’t tried porting this test because Objective C was (and to some degree still is) very foreign to me.  Thus far, I’ve been doing iOS development over the last two months mostly in C++ using OpenGL ES.  My projects so far have a very small interface to the parent Objective C based UIView container.

Needless to say, I feel like I’ve been missing out. However, coming late to the Cocos2D party has its advantages. There are tons of Cocos2D resources in the wild.  The one I see most often when browsing idevblogaday is the Rod Strougo’s blog about his Cocos2D book.

Here is something I wrote in XNA a couple months ago:

And here is the start of my Cocos2D version:

So far I’ve added a background sprite and a character sprite to a single layer.  When you click and drag from the character to a spot on the map the character starts to the move there from the current position.  I’ve spent about an hour on it mainly reading documentation and I’m impressed with how succinct this little example is so far.

After watching Rippy’s presentation I was impressed by two interesting aspects of Cocos2D: 1) Scene composition and 2) Behavior composition.  In Cocos2D, it’s easy to build up a scene hierarchy as well as a complex behavior.  It’s a little unfair to do a full comparison of my Cocos2D and XNA prototypes just yet.  In the next couple of days I’ll bring them to parity and show the code differences.

May 30 jacob

CocoaHeads Demo

Bruce showing a demo at the local CocoaHeads meeting:

Made some improvements to the terrain like lighting and fog.  Here is the fly-through shown in the video above:

May 23 jacob

Terrain LOD on iPad (part 2)

I’ve made some progress on the terrain LOD in the last 2 weeks on:

  • moving terrain LODs
  • loading in new data as the mesh moves around
  • blending between the different LODs

Moving LODs

First thing that was working was the movement of the different levels in the clipmap.   There is one mesh with size of 64 units (n from the paper so n = 64, 64x64x2 = 8192 triangles) and there are a configurable number of levels visible (m from the paper).  As there camera moves up, the higher detail levels are dropped and lower ones are added.

The video shows how I cheated a bit and just overlapped the different levels by one cell.  This diverges from the paper because I thought it would hide artifacts until blending was implemented and be a little easier.

Loading New Data

This is how the geometry is laid out:

  • 1 static vertex buffer with a vec2 Position
  • m dynamic vertex buffers with a short Height
  • 16 static index buffers that form a 4 x 4 grid over the vertex data

The entire dynamic vertex buffer for a level is updated when that level moves.  This keeps things simple for now and seems to be fast I guess because the shared video and main memory.  Here is a video of new data coming in (ignore the blending for now):

One of the big advantages of the paper is they limit a small number of dynamic buffer updates every frame.  I don’t do that yet… so I have most frames with no updates, few frames with small updates, and rare frames with large updates.  I’m hoping to resolve this at some point…

Blending

To quickly add blending I made a couple of simple changes:  add another height parameter to my dynamic vertex structure and blend between these in the vertex shader based on the distance from the camera.   The visualization for the blend regions is in the video above.  Here is a comparison of before and after blending:

This to the original geometry clipmap paper’s implementation and I think it works really well.  The alternative is the method in the CDLOD paper.  I don’t know which one would look better but I would have to fix my overlap problem before the other method would work.

Next Steps

The next big features are computing normal maps and working on a similar scheme for dynamically updating the texture data (color and normals).  Also there are several performance related tasks like culling and reducing redundant state calls.  I hope to have a video of this working by the end of next week.

May 9 jacob

Terrain LOD on iPad

For this project with Bruce we need to render a large amount of terrain data.  There is a ton of research into this but the modern approaches branch from 2 papers:

Geometry Clipmaps

Clipmaps use an almost static set of geometry centered around the viewpoint.  Here is part of the intro from Frank Losasso and Hugues Hoppe:

We introduce the geometry clipmap, which caches the terrain in a set of nested  regular grids centered about the viewer.  The grids are stored as vertex buffers in fast video memory, and are incrementally refilled as the viewpoint moves.

With a flat terrain it would look something like this:

If you locked the mesh and zoomed out it would look something like this:

The big advantage is the incremental update as the camera moves through the data.  Any one level would only need to update one row and column.

More recent approaches branch from the GPU Clipmapping technique described in GPU Gems.  This technique stores the height data as texture data and keeps the mesh data static.  In the vertex shader the mesh can be transformed after sampling the height data.

All of these techniques use a complicated system for stitching the levels together and transitioning between levels.

Chunked Level of Detail

This method breaks down the terrain into a quad tree and precomputes simplified meshes for each node in the tree.  At run-time, meshes are selected based on the projected world space error represented by each node while trying to stay under the triangles per frame budget.  Here is a picture of the mesh data in a quad tree from Thatcher Ulrich’s Siggraph course notes:

There are more modern extensions of this approach that also combine some of the features of GPU geometry clipmaps like Continuous Distance-Dependent Level of Detail for Rendering Heightmaps.   I like this technique for its really simple transitioning scheme.

Conclusions

These later methods like CDLOD and Spherical Clipmaps seem perfect but all of these either try to fit all the height data in video memory (which we can’t do) or some virtual texturing solution to stream data in.  Unfortunately the PowerVR GPU doesn’t support access to texture data in the vertex shader.

This leaves the original Geometry Clipmapping as the best option.  The up-side is the iPad has shared video and main memory.  This means updating dynamic vertex data is much faster than you would expect on a desktop video card.

I’ve been working on it for a few days and so far it looks like it will work out.  Here is a screenshot:

I made some pretty big simplifications to the LOD stitching scheme that may or may not result in some serious artifacts.  I’ll describe in more detail what did and didn’t work in the next post.  Also, I’ll show a video of the dynamic updating.

May 7 jacob

Thank You Hobo

Check out Thank You Hobo.  Chris recently finished it as part of Ludam Dare 20.  He is shunning his 3d graphics roots and making some awesome stuff.

Thank You Hobo screenshot

Apr 30 jacob

Getting more into iOS development

To jump into iPad development I’ve been working with a friend, Bruce, on an app to be announced. I’ve been working on it since Monday and the progress has been great.

The app currently has instruments at the bottom and a 3d view at the top but it will soon be configurable in full or split screen on portrait or landscape.  Here is a current screenshot:

 

Hurdles

I’m mostly new to the tech (spent one weekend on iOS development detailed in a previous post) so here are some of the micro hurdles I had:

  • Loading real terrain data from resources

All the digital elevations file formats are well documented and pretty easy to parse.  My main problem for a few hours was just finding a packaged resource!  You can query it with NSBundle:

[[NSBundle mainBundle] resourcePath]

  • Setting up basic 3d environment (device state, render targets, vertex buffers)

It seems no matter what 3D API you use the standard setup process can be somewhat tedious and all before you get a triangle rendered!  Plus, I’ve been spoiled with XNA for too long…  There are lots of examples on simple Hello World style Opengl ES apps to reference.  I’ve been using mostly the iPhone 3D Programming book.  Also the OpenGL ES 2.0 Quick Reference is great if your OpenGL is rusty.

  • Getting a OpenGL view mixed in with the other views

Bruce solved this one and I don’t really really know enough about the iOS UI libraries to describe it.  If anyone has a quick answer please post it!  Otherwise I’ll get Bruce to try to boil it down next week.

Ongoing

I’ll continue to post about this project as we make progress.  Expect it to look a lot better in a week!  Perhaps I’ll show some google earth comparison shots.

Apr 25 jacob

Feedback from GDC

We took our game-in-progress to GDC and we got some great feedback.  It came in many forms ranging from hands-on testing to nuggets of wisdom from sessions.  I would like share just a few points and then describe them in more detail:

  • controls - dual analog doesn’t work well on touch screens
  • visuals – need more character; bright with high contrast
  • form factor – tablets present new opportunities; can be financially viable as a primary platform

GDC is amazing to many people for many reasons, for example, Owen’s recent post Why I Attend GDC.  For me it was a great learning opportunity, however these points may seem obvious to some.

Below green comments = positive and red = negative

Controls

The control scheme was like Smash TV; the left stick moves the player while the right stick shoots.  Once you press down on a side of the screen that position is the center of an analog stick until you release.

Main character looks fluid while observing someone else play
Feels clunky while playing

The action looked fluid until the crucial time of trying to avoid an enemy or an obstacle; the lack of precision in the controls was a problem.  One way I tried to address this was to help the player automatically move around small obstacles.

If game would be great with a controller, you are making the wrong game

This quote from Infinity Blade: What We Learned was echoed by many developers. One alternative that was mentioned repeatedly: interact, watch, interact, watch.  I had fun with this type of control scheme like in Battleheart so I started to experiment with it:

Visuals

Overall looks great!

We got some good first reactions.  People simply liked that it was 3d.  Also, we got a few comments on how the world felt cool when moving around it.  We use a cylindrical style like in Deathspank or Final Fantasy: The 4 Heroes of Light.

Too hard to see

There were two main complaints about visibility: the objects were too small and the player and enemies are hard to distinguish from the background.

I work probably too much with the emulator at 200% zoom.  In this mode, everything is easy to see.  The fix seems easy but zooming in on the character creates a problem with how many units we planned to be on-screen in each encounter.  Also see the form factor discussion below.

The contrast can be improved by rethinking the textures and lighting.  Not much focus has been put into this but I would like to get more concept art made to help explore this space.

Lacks recognizable character

I’m working on this part… After showing some of Marco’s newer assets that aren’t yet integrated into the game we got better feedback.  Here is a lineup of some of the characters including the ones not yet in-game (helicopter, spider, drill):

Form Factor

There was general optimism about the larger form factor of a tablet especially from this talk: Building Strategy Games for Mobile Devices.  I highly recommend watching it especially for the ton of real data from Highborn.

The tablet allows for a richer experience even for games ported directly from the iPhone.  At the same time, iPad app sales alone are very strong now and the space is rapidly expanding.

While incorporating this GDC feedback I’m moving to the tablet as the primary release platform.  Many very promising ideas come back to the table now that the small phone screen is not the main concern.  To me this is very exciting!