Several months ago we switched from sprite sheets to vector data for all our artwork. Image quality went up, network transfer went way down, and we got to keep Flash as the tool for creating the raw assets. Read on for how we hackily convert Flash animations into a custom JSON vector format EaselJS can use and manipulate in all sorts of useful ways, along with the pitfalls and obstacles that came along the way.
The Problems with Raster
The main issue is that sprite sheets are enormous. File size needs and image quality needs are diametrically opposed in a raster world.
The fangrider's impressive wingspan begets nearly 3MB in sprite sheet data, if it's good quality
Sprite sheets are also very inflexible. We wanted sprites to be able to change their colors dynamically, for team colors and customizing wizard avatars. We tried a couple different techniques for this with mixed success. First we broke down the sprite into several different parts we would color with CreateJS filters then recombine. This was about as complicated as it sounds, and it made sprite sheets even larger.
No fun putting this wizard back together again
The next attempt was green-screening the sprite sheets by using custom filters. For each pixel on the sprite sheet, if it was within certain HSL ranges, the filter would shift the values. Much simpler and easier, and just about as effective as the sprite break down technique.
And you could apply patterns or images with a green screen.
A Vector-Only Pipeline Between Flash and EaselJS
Finally, late last year I found a Flash plugin that exports to EaselJS. However, there was only one problem: it was in the form of a rather verbose JavaScript file that had the constituent pieces in code and only exposed the final, non-serializable product.
Luckily, at this point Nick was quite the expert at parsing JavaScript, having done all the work on parsing, transpiling and sandboxing needed for CodeCombat gameplay (see Aether). So in the middle of his 120 hour work week we hacked on it together until we had a serviceable (read: ugly) parser that would take the JavaScript file and spit out a nice, compact JSON object we could store in MongoDB and use to reconstruct the vector sprite from the ground up on the game side. Perfect.
Well, not entirely. The new system has had its upsides and its downsides. Here’s a rundown:
Total Benefit: Tinting
EaselJS vector sprites exported by Flash are mainly composed of three things:
- Shapes. A list of instructions about a path and a fill that generate an atomic vector image.
- Containers. Collections of Shapes and other Containers, with information on static placement only.
- MovieClips. Collections of Shapes, Containers, other MovieClips, and ‘Tween’ information that define where those various elements go and how they move over the course of an animation.
The shapes are where the colors are defined; each shape has one fill color and one stroke color. With the raw data in our own format, it’s trivial to grab those colors and change them as needed. Here’s our formula: Given a target color and a set of shape colors, find the average hue, saturation, and lightness values of the given shape colors. Adjust those so that the averages are now the target color’s hue, saturation and lightness, creating a color map ( implementation). This makes it so that we can target a specific HSL value we want, while also preserving the slight differences between them.
Selecting and testing color groups in the thang editor
Partial Benefit: File Size
Post-gzip, vector data is the clear winner compared with a sprite sheet of decent resolution. Animated sprites have the greatest improvements; they tend to be less than 1/10th the size.
Sizes in parentheses are gzipped sizes.
PNGs are already compressed, so they don't really improve.
Unfortunately, vector sometimes just fails to work. For example, the specially-made Brawlwood level map is completely unusable because Flash can't export it to an EaselJS file; it just crashes the program. Even if it did work, it'd probably be a monster of a JSON file. Would it be smaller than a high-quality PNG? We're not sure.
Full Brawlwood map we have yet to be able to use
We ended up exporting the road by itself and incorporating other assets separately, and even then it's been problematic. Canvas regularly chokes when we try to render it at a high resolution. There's special logic in the code just to handle this case.
We are not above doing this (source)
Some sprites can also end up being surprisingly large. Our artist gave us an energy ball (to be used as a missile for spellcasters) that ended up being over 600KB of vector data. One blue sphere! Even gzipped, it ended up being much larger than a raster sprite sheet. We had him remake it into a much more compact, though admittedly less cool version.
Vector sizes are 113KB (28KB) on the left compared to 620KB (126KB) on the right.
A raster sprite sheet of the right one is only 53KB.
And finally, adding more animations to a unit has not been as cheap as we had hoped. Most animations should use the same shape and container assets; after all Tharin walking and Tharin falling down to the ground all have the same arms, legs, head, sword and shield. But adding animations is still fairly expensive in bytes.
Sharing vector assets between animations only gets you about a 20% savings.
It's too bad that we can't just continue adding more and more specialized animations to Tharin without dramatically increasing his file size for all levels. Some sort of compartmentalization is in order if we want to give sprites one-off animations, and each generally available animation needs to pull its own weight.
Smaller Benefit: Ongoing Maintenance
Seeing as this is a rather hacky way to get vector data, the parser has a tendency to break down. At regular intervals George will report to Nick and I that sprite importing doesn't work, and it's because the Flash exporter has changed things slightly or our artist has used some new feature we previously did not know about, or some other edge case crops up. It's a great example of how hacked solutions are unstable. It usually doesn't take much time for us to make the necessary adjustments, but it is a distracting nuisance, and it slows down George's ability to get work done when he's waiting on fixes.
This method is still a net improvement over the old raster method. Exporting consistent sprite sheets comes with its own set of headaches, and trying to hack together ways to tint units was a much larger time-consuming mess (though the green-screen method would have been cool).
It would be ideal if, one day, EaselJS has an official JSON format for MovieClips and all their constituent data. That would make the process of going from Flash to EaselJS much smoother.
Un-Benefit: SpriteSheet Creation Time
Though the whole artwork pipeline is vector, we still need to render the artwork as sprite sheets at the end for performance reasons, and this is a surprisingly CPU intensive process. It can take ten seconds or more to render everything for an average level even with a good computer, and that's time players have to wait patiently through. Unfortunately, rendering at smaller sizes does not make things go faster, or else we would render low-res versions first then replace them with high-res versions during gameplay. However, we are currently building the sprites asynchronously, and it turns out building sprites synchronously with EaselJS goes about twice as fast, with the downside of course that it completely blocks the main thread. This trade-off may be worth it.
What To Try Next
- JIT loading. I've started experimenting with dynamically loading marks (icons that float above a unit's head, indicating they have a status effect like haste or poison). So rather than fetch all marks for each level on the off chance they're needed, Marks load and render themselves only when they appear in game. We could do this for all sprites, not just marks, and get the game started faster this way. On the other hand, it wouldn't look good to see a level before all sprites are rendered, popping in as they finish, or possibly lowering frame rate by rending during gameplay.
- Centralized, shared data. Except when going directly from one level to the next, all the vector data is thrown out when you go somewhere else on the site, and that's a waste. Even if the first level takes a while to load, we can at least make subsequent levels load much faster. This would also be good for Systems and Components, the shared logic between levels.
- Exporting to other formats. Our artwork is creative commons, but it's not very accessible in this custom format. Ideally there would be a way to customize these sprites (with tinting, or setting a resolution size) and export them in some more helpful format, like a series of images.
- Support both vector and raster. It's probable that exporting large vector files like the Brawlwood map is simply an intractable problem. We could have some ThangTypes use raster data where it makes sense, like with static assets, as long as it's worth giving up the flexibility of sizing these images any way we want easily.
- Experiment with new file formats. The custom JSON format could possibly be made more efficient, maybe with binary, maybe with specialized compression rather than relying on generic gzip.
That's it, the past, present and future have all been covered. Check out our GitHub repository to stay abreast of the ongoing vector artwork saga. Thanks for reading!