M$ Site
Original Paper
Two researchers -- Johannes Kopf from Microsoft, and Dani Lischinski from The Hebrew University -- have successfully created an algorithm that depixelizes and upscales low-resolution 8-bit "pixel art" into lush vector graphics. The algorithm identifies pixel-level details to accurately shade the new image -- but more importantly, the algorithm can create smooth, curved contour lines from only-connected-on-the-diagonal single pixels. Look at the Super Mario World dolphin below, and compare it to the original source sprite below that: the results speak for themselves.
Check out the video!To achieve such beautiful images, the researchers use a complex blend of pixel analysis and spline curves. These approaches in specific, and vectorization of bitmaps in general, are nothing new -- Adobe Illustrator does it quite well -- but in this case, because the researchers were only working with 8-bit pixel art, they could create a very specialized algorithm. First and foremost, they can assume that every pixel is important: early 8-bit sprites were all masterfully hand-crafted by artists, so it's safe to assume that every pixel is significant. If a pixel is all alone in a sea of another color, it's an important feature, not an anomaly. Then the algorithm works out which pixels are connected to each other -- in a 2x2 checkerboard pattern, should the diagonals form a line, or not? Finally, these pixel cells (groups) are re-shaped, smoothed with spline curves, and then rendered as edges or shaded areas.
For a better idea of just how accurate their algorithm is, you should look at the other (nearly all Nintendo-related) samples in the Depixelizing Pixel Art research paper. The algorithm isn't always successful, though -- and the paper includes a hilarious version of the "Doom face" to illustrate its shortcomings. The other problem is that the Depixelizing Pixel Art approach always smooths images, even when an object shouldn't necessarily be smooth. For example, are Space Invaders really meant to be cute and round? Maybe, in the creator's eye, they had long, angular, razor-sharp mandibles and straight-out antennae!
While the researchers' conclusion admits that the algorithm is computationally complex, some optimization could eventually lead to emulator that can upscale our our favorite retro games in real time. It's unlikely -- and purists are no doubt nervously tugging at their beards at the thought -- but Nintendo could also license the technology for its next-generation console! At long last, we might be able to play Super Mario Bros. on a big screen without stretching our beloved plumber's pixels to breaking point.
Pretty cool! If they can get some of the intense calculations down, this will be awesome for old games!
