Adam Maciaszek, the mind behind ‘The Skeleton Dance’ 4K remaster, discusses his AI-assisted process as well as upcoming classic cartoon restoration projects.
It’s beginning to look a lot like 4K this holiday season! Video editor Adam Maciaszek, the genius behind the 4K remastered Silly Symphony short The Skeleton Dance, talked to AWN about his AI-assisted remastering process, as well as upcoming classic cartoon restoration projects.
For Maciaszek, every new step towards innovation begins with passion that buds into unparalleled inspiration. “I have always had a passion for older cartoons and film inspired by my father,” he says. “When I tried to share these cartoons and shorts that I loved with my friends, I realized no one would continue to watch them on their own, [because they] couldn’t get past the scratchy grainy quality it was shown in. Modern cartoons made in flash have spoiled us with their crisp lines and I want the current generation to see how amazingly fluid and unique animation of the 20s and 30s was as well as silent films.”
While Maciaszek’s passion for sharing these beloved cartoons lasted for years, he only recently started remastering them as a computer engineering student at Umass Amherst. According to the editor, “In early 2019 there was a symposium on AI and machine learning with images and video. I was amazed by how black and white images could become colorized and remove any and all scratches on them. So, I decided to take more classes in machine learning as well as learn about film. The COVID-19 outbreak really inspired me to work a lot more on this and gave me lots of time to take online film restoration classes and improve the efficiency of different methods.”
Extrapolating on the AI-assisted process of restoration, Maciaszek explains, “I use AI in three different ways. One way is in upscaling, in which the neural network tries to assume details that are not present in the original and sharpen up lines that are jagged. This also allows me to work in the same 8K resolution as regular film restoration experts who have access to the original film. This upscaling is done with Topaz AI Video.”
“The second use for AI is time interpolation, but I don’t use this in the way it is intended,” he continues. “First, I make a list of all the unsalvageable frames. What makes them unsalvageable varies and can result from heavy film degradation, partially missing or cut off frames, or massive scratches during motion. With this, I take the frame before and after, and compose them all into one clip which I feed into DAIN (Depth-Aware Video Frame Interpolation). Then I pull out the AI generated interpolated frames and replace the damaged frame.”
The third method is colorization, which he used in more recent projects.
Most of his work is done with regular editing. “I use Premiere Pro with the plugin Neat Video, which is great for grain and temporal noise such as fluctuations in brightness,” Maciaszek reveals. “For scenes with low motion I sample from 5 frames ahead and behind. In contrast, for scenes with lots of motion I sample only the previous and next frame. I can also use this to remove different types of artifacts left behind from upscaling, as well increase the sharpness.”
“Any and all stabilization is done in Mocha Pro, which is much more reliable than the built-in adobe stabilization, as it allows you to select where in the background you want it to be stabilized,” he notes. “The last step I do is most helpful when there are large scratches or dust that is visible in more than one frame. It involves going into Phoenix Digital Vision Touch and using the Dust+Fix to manually select each area frame by frame, as well as which frames to fill in the area with, or whether to clone it from another area in the same frame.”
That’s a lot of work! A typical project takes “roughly 12-20 hours” of active editing, and with rendering time, can run “36+ hours.” His favorite labor of love is Betty Boop Snow White (1933), as, “Each frame is a work of art, and you notice all the small things in it so it doesn’t become tedious.” Watch the fascinating side-by-side comparison below:
If you’re looking to get into the holiday spirit, check out Maciaszek’s remasters of Ub Iwerks’ Jack Frost (1934) and Wladyslaw Starewicz’s glorious stop-motion short The Insects Christmas (1913). He is also working on Somewhere in Dreamland (1936) and Christmas Comes but Once a Year (1936). Watch an exclusive sneak peak of the upcoming 4K remaster of Christmas below:
As for the future, Maciaszek is looking into both the 3D world for virtual reality, and more Starewicz animation, “The first ever appearance of Mickey Mouse in color was when Disney featured him in green and yellow rather than the iconic red. I also plan on doing more Max Fleischer cartoons of Bimbo and Betty.”
Check out all of Maciaszek’s impressive work on his YouTube Channel, and keep an eye out for more videos coming soon.
Cybersecurity specialist by day, investigative journalist by night.
