Rendering:
Don’t try this at Chrome!
If, like me, you’re writing a graphics intensive web application that renders on Canvas, you may, with some justification, have thought that the era of cross-browser compatibility issues (and the inevitable accompanying hacks) was over. Well, here at Prezi, we’re working on a player to use Javascript and Canvas2D, and from deep in the trenches, we can report that you’re wrong. The most interesting thing about this wrongness is that the browser that’s kept us camped in the mud is not IE, but the world’s most popular browser, Chrome. Here are the gory details behind the battles we’ve been having with its graphics capabilities.
Battle #1 — Lost Canvases
If your application needs to do high performance 2D rendering, you’ll probably end up caching a lot of content on off-screen Canvases, things like image mipmaps, spritesheets, or pre-rendered vector graphics. If used well, this can save you rendering time, as you don’t have to recalculate & rerender them in each frame. The problem was that with Chrome, it turned out that on OSX, it became easily overloaded and 2D Canvas content was silently lost. The only known way to detect this is to read back pixels from the graphics card, which, being the slowest operation of all time, anywhere, ever, kind of kills the whole point of caching in the first place.
Well, losing precious pixels is actually a normal occurrence, Canvases of a meaningful size are hardware accelerated, and the GPU is a shared resource. Other tabs and applications also use hardware accelerated graphics (even the browser itself for rendering both its windows and page content), so you can expect that at some point it will be overloaded and resources will be released, maybe on a least recently used or over-quota basis. Other non-overload situations like the machine going to sleep and then waking up will also cause GPU memory to be released. In such situations the typical thing for the browser to do is to save Canvas2D surfaces to persistent stores and restore them when needed. This works fine in Firefox and Safari: when over-allocating 2D Canvases, they gracefully degrade in performance, but they keep going like a dog running after a tennis ball for the 4000th time (at some point you will run out of RAM as well, but that is a different story). Another approach is to provide an API for the application to deal with the hard facts of life, as done by the WebGL specification, which introduces both an async “webglcontextlost” event and a method “isContextLost” on the 3D context that can be used to detect the loss events in the application.
None of this happens on Chrome on OSX though — quite a bit of trench time was spent narrowing down the issue to this platform — our cache Canvases simply disappeared, no exception, no event, no goodbye, just broken hearts (I don’t want to overdo the war analogy, but this is surely the geeky equivalent of trenchfoot). The only way to detect that content was gone, was to actually draw them, and then realize that there was nothing drawn (only a black bitmap with 0 alpha).
All right, if it’s not detectable, then let’s prevent it in the first place. No, this is also practically impossible, since we can’t control other applications. In an overloaded environment maybe even a single Canvas is already too much, so no matter how clever a limit we invent on cache size, it can still just happen. This is when the trenches really started to stink and we all began getting on each others nerves.
The way we ended up handling the issue was to implement a watchdog (as this photo essay shows, there were dogs used as medical assistant in the trenches of WWI, making my metaphor surprisingly accurate). In idle times of non-rendering — for example when our presenter doesn’t move around in her prezi — it (the dog, not the user) walks around and checks our cache Canvases one-by-one. Each check involves drawing the Canvas to a detector image (similar to a monoscope test image on a television) and then reading back the pixels of the result. If the detector image is unaffected, then a lost Canvas is detected, thrown out, and regenerated later on demand.
This is very slow and a waste of CPU processing. Even worse, our users will still see some temporary flickering when caches disappear, so we’re not really satisfied with our “solution”. In fact, just like Martin Sheen in Apocalypse Now, we still spend a large portion of our day lying on our beds staring up at fans as they spin around, hallucinating nightmares. Perhaps we should ask the office team to remove the fans.
Battle #2 — No Anti-alias on drawImage
We very quickly realized that anti-aliasing is not available on Chrome for rotated image draws. This is one of the longest outstanding issues on Chromium, first reported back in 2009. The debate is lengthy, and involves the counter-argument that, in image tiling use-cases, anti-aliasing introduces artifacts (which is true). Instead of perfect pixel alignment between neighbor image tiles, you get overlapped anti-aliased pixels. I don’t think the argument holds though, as anti-alias should simply be optional.
Implementing anti-aliasing ourselves is not really an option: we would need to do the image-to-screen transformation on the CPU side, or read back the transformed image form GPU, which would make image drawing — already a performance bottleneck in many cases — even slower. CSS transformed images have anti-aliasing, but that would require the image to be on a separate DOM element. This could be a solution, but we didn’t want to take this path, because of the complexity and unexplored performance implications of multiple overlapping Canvases. Not to mention opening up a second frontier to fight with possible CSS and DOM cross-browser issues.
Fortunately a workaround can be crafted: Chrome does anti-alias when clipping, and this can be exploited. If we draw a rectangle clip on an image using “destination-in” globalCompositeOperation mode, then the resulting clipped image will be anti-aliased on the clip edges. Unfortunately this cannot be part of main draw pipeline, because it will destroy the very frame that we’re building (all the objects that were rendered on the Canvas earlier). But we can do the dirty work on a separate Canvas (although according to my earlier trench comparison, all our work is dirty). Put it together, and it works like this: when drawing an image with rotated transform, we render it on a buffer Canvas, clip a half pixel off on the circumference using the above trick, then copy the anti-aliased image on the main frame and continue rendering with the next object.
Besides adding additional complexity, this ugly hack has a performance toll as well — we’re doing an additional image draw for each rotated image. And of course let’s not forget to check the buffer Canvas now and then to make sure it is not lost.
The latest news (and totally correct at time of going to press) is that the Canvas loss issue seems to be fixed in the latest Chrome Canary (39+), and also an API similar to that of WebGL context loss is in the works (proposed here, and in feature request here). It would be awesome to see this on Chrome mainstream before the New Year. Because then maybe I can get home for Christmas.
