U.S. Markets open in 5 hrs 29 mins

Nvidia’s new AI software can turn a crude sketch into a stunning work of art

Jacob Siegal

It’s somewhat difficult to square the hand-drawn animation that dominated the box office for decades with the almost hyper-realistic CGI that we see today, but as technology continues to advance, both art and animation will continue to change. To that point, Nvidia appears to be on the verge of changing the game once again with a new deep learning model capable of transforming the most basic of sketches into photo-realistic images.

The AI leverages generative adversarial networks (GANs), which you can read about here, to convert the maps you see in the video below into beautiful landscapes. In a nod to French post-impressionist painter Paul Gauguin, Nvidia decided to name the interactive app which uses the model “GauGAN.”

Related stories

CES 2019 kicks off with 5 sleek laptops powered by brand new Nvidia GeForce RTX graphics

Nvidia's AI can generate fake human faces that look 100% real

Leaked Nvidia RTX 2080 benchmarks shows it narrowly beating a GTX 1080 Ti


The app works by letting users draw “segmentation maps” (the left side of the image at the top of this article), labeling each segment as mountains, snow, water, or grass, and then filling in the detail automatically. Bryan Catanzaro, vice president of applied deep learning research at NVIDIA, compares the technology to a “smart paintbrush,” which the company says has been trained on a million images to know what to fill in where.

[youtube https://www.youtube.com/watch?v=p5U4NgVGAwg?version=3&rel=1&fs=1&autohide=2&showsearch=0&showinfo=1&iv_load_policy=1&wmode=transparent&w=782&h=440]

“It’s like a coloring book picture that describes where a tree is, where the sun is, where the sky is,” Catanzaro said. “And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows and colors, based on what it has learned about real images.”

The key to the realism of these computer-generated scenes is something called a discriminator, which gives another important element — the generator — “pixel-by-pixel feedback on how to improve the realism of its synthetic images.” For example, after seeing enough photos of lakes, the generator understands that objects cast reflections on water, and going forward, it will do its best to imitate those reflections when generating landscapes.

GauGAN is on display at the GPU Technology Conference this week, but is not yet available to the public.

Sign up for BGR's Newsletter. For the latest news, follow us on Facebook, Twitter, and Instagram.

Trending Right Now:

  1. Nvidia’s new AI software can turn a crude sketch into a stunning work of art

See the original version of this article on BGR.com