3D VisualizationAIVirtual Reality

New AI tool by Nvidia can automatically generate realistic virtual worlds

The GAN-based AI technology was presented at the NeurIPS conference in Montreal this week

As one might expect, creating a virtual 3D world is a laborious process that requires not only time and effort but also a high level of skill. Look at any video game, and the graphics will have required thousands and thousands of hours of work on the part of developers specialized in 3D rendering, lighting, textures and more.

That level of work to create virtual environments may not be necessary for much longer, however, as graphics processing unit company Nvidia has developed a powerful AI algorithm capable of generating photorealistic details for a virtual world. The AI tool could not only impact the video game development industry, but could also have important applications in generating environments for virtual reality as well as for training self-driving cars and robots.

Bryan Catanzaro, the vice president of applied deep learning at Nvidia, explained: “We can create new sketches that have never been seen before and render those. We’re actually teaching the model how to draw based on real video.”

If the AI tool sounds impressive, that’s because it is. Nvidia’s team has utilized a conventional machine-learning approach to identify various objects appearing in a video scene, such as cars, trees, homes, etc. From there, it has leveraged a more sophisticated generative adversarial network (GAN) to train a computer to generate photorealistic 3D imagery to fill the outlines of objects.

In other words, users can input the outline of a scene (indicating where different objects should be), and the algorithm will fill in the outlines with realistic details. At this stage, the graphics generated by the algorithm are not perfect—as some objects can appear lightly warped, almost dreamlike—but the output is already highly advanced.

“Classical computer graphics render by building up the way light interacts with objects,” added Catanzaro. “We wondered what we could do with artificial intelligence to change the rendering process.”

The innovative approach was presented at NeurIPS, an AI conference hosted in Montreal this week and has gained significant attention already, largely from the gaming sector, which could drastically improve (in terms of ease) game design.

NVIDIA AI

“NVIDIA has been inventing new ways to generate interactive graphics for 25 years, and this is the first time we can do so with a neural network,” said Catanzaro. “Neural networks — specifically generative models — will change how graphics are created. This will enable developers to create new scenes at a fraction of the traditional cost.”

Located in Santa Clara, California, Nvidia is the latest company to unveil a GAN-based algorithm. Invented by Google, GANs have only been around for a few years but are already demonstrating their ability to produce realistic imagery and sounds. You might recall a viral video of Trump from May 2018 calling on the Belgian people to withdraw from the Paris climate agreement. The video—though clearly fake—was created using GANs and demonstrated the potentially dangerous ramifications of the technology on creating realistic and (actually fake) fake news.

Still, from a technology point of view, Nvidia’s algorithm is mighty impressive and we’re eager to see what its first real world (virtual world?) applications are.

Tags

Tess Boissonneault

Tess Boissonneault moved from her home of Montreal, Canada to the Netherlands in 2014 to pursue a master’s degree in Media Studies at the University of Amsterdam. It was during her time in Amsterdam that she became acquainted with 3D printing technology and began writing for a local additive manufacturing news platform. Now based in France, Tess has over two and a half years experience writing, editing and publishing additive manufacturing content with a particular interest in women working within the industry. She is an avid follower of the ever-evolving AM industry.

Related Articles

Leave a Reply

Your email address will not be published.

We use cookies to give you the best online experience. By agreeing you accept the use of cookies in accordance with our cookie policy.

Privacy Settings saved!
Privacy Settings

When you visit any web site, it may store or retrieve information on your browser, mostly in the form of cookies. Control your personal Cookie Services here.

These cookies are necessary for the website to function and cannot be switched off in our systems.

In order to use this website we use the following technically required cookies
  • PHPSESSID
  • wordpress_test_cookie
  • wordpress_logged_in_
  • wordpress_sec

Decline all Services
Accept all Services
Close
Close