Z-Sorting in HaxeFlixel

Even if you’re just coding a 2D game, eventually you’ll run into the issue of the draw order, z-sorting, depth-sorting… There are many names, but they all describe the same problem: Drawing sprites in the right order.

This is especially important in top-down games like Zelda, where tiles and sprites drawn at different depths create the illusion of a coherent, three-dimensional world. But, of course, this is important in other genres as well, such as 2D platformers, where you have objects in the background and foreground.

In HaxeFlixel, the z-order of objects is determined by the order in which they’re added to the current state using add(). Objects that are added first are also drawn first, meaning that later objects will be drawn on top of previous ones, thereby covering them up. There are two ways to leverage this behavior for more fine-tuned control!

1. Groups as Layers

The first and simplest approach is to create groups for each “layer” of objects that you have. For example, if you’re making a platformer, you might want a background group to store background objects such as trees and mountains, as well as a foreground group for the player, enemies and pickups. Then you simply have to add the background group to the state first, and the foreground group second. The objects from the foreground group will now always be drawn on top of the objects in the background group.

This simple setup is enough in many situations, but it doesn’t always work. In a Zelda game, for example, grouping things into “layers” doesn’t make a lot of sense, because the z-order of the objects changes way too much during gameplay. Plus, separating your foreground and background objects into groups is fine, but what if you need to change the z-order of the objects within one group? That’s where sorting functions come in.

2. Sorting Functions

In theory, it’s actually quite easy to reorder the objects in a HaxeFlixel state. This is because a FlxState is nothing more than a fancy FlxGroup, and a FlxGroup is nothing more than a fancy array. So, all we need to do is to reorder the elements in this array to change their drawing order. Sounds simple, doesn’t it?

But: While we could sort all members of the current state directly (it’s just a group after all), it is usually advisable to create a separate FlxGroup for all objects that should be sorted. This way, you get more control over which objects will be affected. You probably don’t want to throw HUD-elements or on-screen text into the mix, after all!

But anyway, let’s start sorting. Every FlxGroup provides a sort function we can use to sort its members. Its signature is as follows:

sort(Function:(Int, T, T) ‑> Int, Order:Int = FlxSort.ASCENDING):Void

It takes a sorting function, as well as the order, which can be set to FlxSort.ASCENDING or FlxSort.DESCENDING. The sorting function does the heavy lifting, of course. Let’s make one!

Say we have our own class MySprite that extends FlxSprite and implements a custom property zDepth. We’ll stick these sprites into a group grpSprites, then call sort on them with a custom function:

var grpSprites = new FlxTypedGroup<MySprite>();

grpSprites.add(a);
grpSprites.add(b);
//...

var sortByZ = function(Order:Int, Obj1:MySprite, Obj2:MySprite):Int
{
    return FlxSort.byValues(Order, Obj1.zDepth, Obj2.zDepth);
}

grpSprites.sort(sortByZ, FlxSort.ASCENDING);

Now, objects with the lowest zDepth-value will be drawn first, meaning they’ll end up being in the background!

As you can see, we’re using FlxSort.byValues as a basis for our own function. This function simply compares the values that we pass in (in this case, the zDepth of our two objects) and sorts the sprites according to the order we specified (FlxSort.ASCENDING). It’s simple!

But what if it’s too simple? What if you need more control? Then you can simply write a completely custom sort function. For example, a custom z-Sort function could look like this:

var customSort = function(Order:Int, Obj1:MySprite, Obj2:MySprite)
{
    if (Obj1.zDepth < Obj2.zDepth)
        return -1;
    else if (Obj1.zDepth > Obj2.zDepth)
        return 1;
    else
        return 0;
}

The three return values mean the following:

-1: Places Obj1 before Obj2 in the list
0: Leaves both Obj1 and Obj2 where they are
1: Places Obj1 after Obj2 in the list

And that’s it! Using these two techniques, you can control the draw order of your objects at any point in the game. Just call sort whenever you need it.

Oh, and a final tip: HaxeFlixel has a built-in function for Zelda-style sorting. Just call group.sort(FlxSort.byY, FlxSort.ASCENDING); and you’re done! Unless your game needs more complex behavior on top of simple y-sorting, that is.

If you have any questions, comments or criticism, post them in the comments below or reach out to me on Twitter @ohsat_games!