Do you have home movies or home videos that are, shall we say, less than perfect? Do these have shaky video or unstable camerawork? If so, read on:

Filmmakers may soon be able to stabilize shaky video, change viewpoints and create freeze-frame, zoom and slow-motion effects – without shooting any new footage – thanks to an algorithm developed by researchers at Cornell University and Google Research.

The software, called DynIBar, synthesizes new views using pixel information from the original video, and even works with moving objects and unstable camerawork. The work is a major advance over previous efforts, which yielded only a few seconds of video, and often rendered moving subjects as blurry or glitchy.

The code for this research effort is freely available, though the project is at an early stage and not yet integrated into commercial video editing tools.

“While this research is still in its early days, I’m really excited about potential future applications for both personal and professional use,” said Noah Snavely, a research scientist at Google Research and associate professor of computer science at Cornell Tech and in the Cornell Ann S. Bowers College of Computing and Information Science.

Snavely presented this work, “DynIBaR: Neural Dynamic Image-Based Rendering,” at the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, on June 20, where it received an honorable mention for the best paper award. Zhengqi Li, Ph.D. ’21, of Google Research was the lead author on the study. 

“Over the last few years, we’ve seen major progress in view synthesis methods – algorithms that can take a collection of images capturing a scene from a discrete set of viewpoints, and can render new views of that scene,” said Snavely. “However, most of these methods fail on scenes with moving people or pets, swaying trees and so on. This is a big problem because many interesting things in the world are things that move.”

You can read more in an article published in the Cornell University web site at: https://news.cornell.edu/stories/2023/07/software-creates-entirely-new-views-existing-video.