Art analysis by algorithms

Hey there, time traveller!
This article was published 15/11/2014 (4209 days ago), so information in it may no longer be current.

At Rutgers University in New Jersey, scientists are training a computer to do instantly what might take art historians years: analyze thousands of paintings to understand which artists influenced others.

The software scans digital images of paintings, looking for common features — composition, colour, line and objects shown in the piece, among others. It identifies paintings that share visual elements, suggesting that the earlier painting’s artist influenced the later one’s.

The project is part of a broader effort at Rutgers to apply computer science techniques to the humanities. This year, the university established a Digital Humanities Lab, based in its Computational Biomedicine Imaging and Modeling Center. The art application is among its first projects.

The field is growing. But some art historians — including Lisa Strong, director of Georgetown University’s art and museum studies program — are skeptical about visual algorithms such as the one in development at Rutgers.

“You can’t really impose a scientific framework so profitably on an exercise like painting analysis,” she said. “It’s not something where raw data tells you something. It’s all subjective.”

Still, the software has revealed some connections that art historians had not — at least, according to the team’s survey of existing art history literature, said Ahmed Elgammal, an associate professor of computer science, who has been working on the project for about three years. “The advantage is it can easily mine thousands and millions of art works in a very (efficient) way.”

For instance, after churning through a database of 1,700 pieces created between the 15th and 20th centuries, its visual algorithm zeroed in on American artist Norman Rockwell’s Shuffleton’s Barbershop, completed in 1950, and French impressionist painter Frederic Bazille’s 1870 Bazille’s Studio; 9 rue de la Condamine.

“The painting might not look similar at the first glance, however, a closer look reveals striking similarity in composition and subject matter, that is detected by our automated methodology,” Elgammal’s team wrote in a paper outlining their research, called Toward Automated Discovery of Artistic Influence.

Both Rockwell’s depiction of a barbershop, seen through a window, and Bazille’s painting of his studio have heating stoves on the right side; roughly where Bazille’s has a window, Rockwell has placed a door; and the composition of objects in each painting creates a triangular space in the lower left corner. (The software was trained to identify certain objects in photographs by analyzing Google images.)

The software also picked up similarities in composition between Vincent van Gogh’s 1890 painting Old Vineyard with Peasant Woman and Joan Miro’s 1922 The Farm.

Although detecting similarities between paintings can help art historians discover possible influences, the software cannot definitively establish a connection between two artists, according to the Rutgers team.

“Our final goal is not to get a final answer,” Elgammal said. Rather, it is “to be a tool to art historians, so it can help them do their job.”

Even if the visual software calls attention to two paintings that had never been compared before, only an art historian can analyze that comparison within the social context of two painters, Strong said.

 

— The Washington Post