As creative as they are innovative, MIT researchers are developing a technology that can faithfully reproduce paintings in 3D based on a 2D image. The technique, called RePaint, uses a combination of deep learning and 3D printing to reproduce canvas artworks.
The technique is being developed by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), which has captivated us with a number of other 3D printing-related projects, including 3D printed movement sculptures, interactive “Robogami,” and 3D printed “Roboats,” to name only a few!
And though we’ve come across other 3D printed painting projects—even actress Portia de Rossi founded a startup dedicated to making art more accessible through 3D printing—the MIT researchers say their method is one of the most cutting edge out there.
That is, RePaint is reportedly capable of authentically recreating paintings based off of a 2D image, no matter what the lighting conditions or placement of the painting are when the photo is taken. This ability could make it possible to recreate famous artworks to be hung in homes, to replace stolen or damaged art in museum settings or to create 3D prints or souvenirs of famous, historical pieces.
“If you just reproduce the color of a painting as it looks in the gallery, it might look different in your home,” commented Changil Kim, one of the authors on a new paper about the system. “Our system works under any lighting condition, which shows a far greater color reproduction capability than almost any other previous work.”
This authenticity is due to a novel printing method the researchers call “color-contoning.” Unlike 2D printers which use four inks (cyan, magenta, yellow and black) to create various hues, color-contoning utilizes 10 different transparent inks which are applied in very thin layers using a 3D printer and a half-toning technique, which creates the image from many tiny dots of colour.
To make the process more automated, the researchers have developed a deep-learning model that is capable of predicting the optimal stack of the different inks to copy a given artwork. Presently, the CSAIL team has only made small reproductions (about the size of a business card) because of cost and time constraints. Even with these small reproductions, however, the team says the RePaint method is four times more accurate than state-of-the-art physical models when it comes to creating exact colour shades.
Moving ahead with the project, the researchers hope to 3D print larger artworks (which is slightly reliant on the development of more efficient commercial 3D printers) and to expand the colour scope of the process. For example, it is currently difficult to reproduce some colours, such as cobalt blue. To resolve this, the team will develop more inks.
The team also plans to create a painting-specific algorithm for selecting inks, and improve the RePaint technique so that it can reproduce surface textures and finishes such as matte or glossy.
“The value of fine art has rapidly increased in recent years, so there’s an increased tendency for it to be locked up in warehouses away from the public eye,” stated mechanical engineer Mike Foshey. “We’re building the technology to reverse this trend, and to create inexpensive and accurate reproductions that can be enjoyed by all.”