By Kelsey Cruz
iPhones and iPads and iPods, oh my! Technology runs our lives from every phone call we make to every television program we stream. We order food from a computer screen and propose marriage on a baseball screen. We download an album before the artist releases it and listen to it on the couch we purchased online. No person, place, or thing is safe from technology – not even art.
And until recently, understanding and analyzing art has been considered a task for humans. It’s something you have to feel to understand and appreciate, not categorize and file. But according to computer scientists Dr. Lior Shamir and Jane Tarakhovsky of Lawrence Technological University in Michigan, machines understand art by using an algorithm.
“The algorithm was actually developed for analyzing biomedical images for automatic diagnostics, early diagnostics, and for studying diseases and clinical conditions in a quantitative fashion,” Dr. Shamir explains. “After the first simple experiment that was based on just nine painters, I started working with Jane (who has a background in art) and designed a whole new and more complex experiment that took it to a new level.”
iPhones and iPads and iPods, oh my! Technology runs our lives from every phone call we make to every television program we stream. We order food from a computer screen and propose marriage on a baseball screen. We download an album before the artist releases it and listen to it on the couch we purchased online. No person, place, or thing is safe from technology – not even art.
And until recently, understanding and analyzing art has been considered a task for humans. It’s something you have to feel to understand and appreciate, not categorize and file. But according to computer scientists Dr. Lior Shamir and Jane Tarakhovsky of Lawrence Technological University in Michigan, machines understand art by using an algorithm.
“The algorithm was actually developed for analyzing biomedical images for automatic diagnostics, early diagnostics, and for studying diseases and clinical conditions in a quantitative fashion,” Dr. Shamir explains. “After the first simple experiment that was based on just nine painters, I started working with Jane (who has a background in art) and designed a whole new and more complex experiment that took it to a new level.”
"And until recently, understanding and analyzing art has been considered a task for humans. It’s something you have to feel to understand and appreciate, not categorize and file. But according to computer scientists Dr. Lior Shamir and Jane Tarakhovsky of Lawrence Technological University in Michigan, machines understand art by using an algorithm."
Yes, scientists have reached their code-creating hands into the heart and soul of art galleries and exhibits and found something new with which to play, coddle, and experiment. Because of this study, is art still in the eye of the beholder? Or has it proven to be understood and classified by even the smallest microchip?
In the experiment, Dr. Shamir and Tarakhovsky used approximately 1,000 paintings of 34 well-known artists and let the computer analyze the similarities of the works sans human guidance.
“An image is compound of many discrete regions (pixels),” Dr. Shamir says. “However, machines do not excel in processing such multi-dimensional data. Therefore, each image is first converted into around 4,000 numerical image content descriptors – numbers that describe the image in a quantitative fashion. These numbers can describe the color, the texture, luminosity, polynomial decomposition of the pixel intensities, statistical distribution of the pixel intensities, frequencies, and many more.”
The computer was able to identify the differences between classical realism and modern artistic styles and automatically separated the painters into two groups – 18 classical painters and 16 modern painters. What’s more, the computer was able to break down the groups even more by identifying subgroups of painters that were part of the same artistic movements, whether they’re High Renaissance artists or Baroque painters. For example, the computer recognized Gauguin and Cezanne as similar, as well as Salvador Dali, Max Ernst, and Giorgio de Chirico.
“In the study, each artist had 19 different paintings,” Tarakhovsky says. “Two random paintings per artist were selected for training, and the last 17 paintings were randomly selected for testing the concept. To prove the concept, this random process was repeated 100 times.”
And although the computer fares better than an art novice in associating related artistic styles, it is not smarter than an art expert. In fact, according to Dr. Shamir, the experiment was designed based on the perception of art historians as the ground truth so, by design, it cannot outperform art historians. However, a previous experiment (also done by Dr. Shamir) showed that the low-level image descriptors show similarities between the artistic styles of Van Gogh and Pollock, something that is very difficult to identify and quantify by the untrained eye – even the eye of an expert.
“Computers are machines, and by the deterministic nature of the way they work, they are widely not considered art,” Dr. Shamir says. “However, the complexity of computers and the creativity of those who invented them have a lot in common with a work of art. It’s true that computers can be explained by equations, but in a way, Pollock’s drip paintings can also be explained by equations. It’s the complexity and creativity that makes computers what they are.”
If that’s the case, is technology exchanging creativity for efficiency? Would we rather a computer take our art appreciation class, believing it would do as well (if not better) during finals?
“I hope not,” Dr. Shamir says. “Creating new technologies clearly requires creativity. Most people, however, do not create new technologies but consume existing ones, and that might bind our perspective to the limits of the technologies that we use. That might not be a step in the right direction. In my opinion, the key practical thing to avoid exchanging creativity for efficiency is by expanding STEM education…I think that if everyone understands technology from the inside, it will help avoid a situation where technology becomes a sort of new god that dictates our culture and lives. Understanding how technology works and is designed will help to identify the limits of technology and avoid taking it as part of nature. It will still be useful, but not magical.”
Tarakhovsky agrees, believing the algorithm could benefit art museums and galleries, not thwart their existence.
“Nobody can substitute intuition, taste, and deep knowledge of art historians and artists,” Tarakvosky says. “This is not the goal of this research. The goal is to offer new ideas, outlooks, and new concepts to the art community to propel ingenuity and profound insight into the nature of art.”
So what does this all mean? Although many argue that art and science are entirely different disciplines, they are both analytical, creative, and complex. Is art autonomous of science or dichotomous with it?
“The two different disciplines may speak different languages, but the way these disciplines are organized and the forces driving them are much more similar than what most artists and scientists would be willing to admit,” Dr. Shamir says. “It’s like two different cultures in two distant continents: The elemental rules and mechanisms that drive the two cultures can basically be very similar, but the different languages and the different customs and traditions make the people of each culture think that their culture is actually different from the other.”
More specifically, because of this experiment and evolving applications, computations, and algorithms, what relationship exists and will exist between computer science and art?
“Basically, computer science is in a stage where it should become driven by discovery and become open to all other disciplines by developing and applying computational methods that solve actual problems in science, engineering, and art,” Dr. Shamir says. “Art is a good example because it might seem something that is fundamentally different from what computers can do, but in fact the quantitative analysis of art done by computers can introduce a totally new approach to the studying of art, an approach that can take art further by providing it with research tools that did not exist before.”
In the experiment, Dr. Shamir and Tarakhovsky used approximately 1,000 paintings of 34 well-known artists and let the computer analyze the similarities of the works sans human guidance.
“An image is compound of many discrete regions (pixels),” Dr. Shamir says. “However, machines do not excel in processing such multi-dimensional data. Therefore, each image is first converted into around 4,000 numerical image content descriptors – numbers that describe the image in a quantitative fashion. These numbers can describe the color, the texture, luminosity, polynomial decomposition of the pixel intensities, statistical distribution of the pixel intensities, frequencies, and many more.”
The computer was able to identify the differences between classical realism and modern artistic styles and automatically separated the painters into two groups – 18 classical painters and 16 modern painters. What’s more, the computer was able to break down the groups even more by identifying subgroups of painters that were part of the same artistic movements, whether they’re High Renaissance artists or Baroque painters. For example, the computer recognized Gauguin and Cezanne as similar, as well as Salvador Dali, Max Ernst, and Giorgio de Chirico.
“In the study, each artist had 19 different paintings,” Tarakhovsky says. “Two random paintings per artist were selected for training, and the last 17 paintings were randomly selected for testing the concept. To prove the concept, this random process was repeated 100 times.”
And although the computer fares better than an art novice in associating related artistic styles, it is not smarter than an art expert. In fact, according to Dr. Shamir, the experiment was designed based on the perception of art historians as the ground truth so, by design, it cannot outperform art historians. However, a previous experiment (also done by Dr. Shamir) showed that the low-level image descriptors show similarities between the artistic styles of Van Gogh and Pollock, something that is very difficult to identify and quantify by the untrained eye – even the eye of an expert.
“Computers are machines, and by the deterministic nature of the way they work, they are widely not considered art,” Dr. Shamir says. “However, the complexity of computers and the creativity of those who invented them have a lot in common with a work of art. It’s true that computers can be explained by equations, but in a way, Pollock’s drip paintings can also be explained by equations. It’s the complexity and creativity that makes computers what they are.”
If that’s the case, is technology exchanging creativity for efficiency? Would we rather a computer take our art appreciation class, believing it would do as well (if not better) during finals?
“I hope not,” Dr. Shamir says. “Creating new technologies clearly requires creativity. Most people, however, do not create new technologies but consume existing ones, and that might bind our perspective to the limits of the technologies that we use. That might not be a step in the right direction. In my opinion, the key practical thing to avoid exchanging creativity for efficiency is by expanding STEM education…I think that if everyone understands technology from the inside, it will help avoid a situation where technology becomes a sort of new god that dictates our culture and lives. Understanding how technology works and is designed will help to identify the limits of technology and avoid taking it as part of nature. It will still be useful, but not magical.”
Tarakhovsky agrees, believing the algorithm could benefit art museums and galleries, not thwart their existence.
“Nobody can substitute intuition, taste, and deep knowledge of art historians and artists,” Tarakvosky says. “This is not the goal of this research. The goal is to offer new ideas, outlooks, and new concepts to the art community to propel ingenuity and profound insight into the nature of art.”
So what does this all mean? Although many argue that art and science are entirely different disciplines, they are both analytical, creative, and complex. Is art autonomous of science or dichotomous with it?
“The two different disciplines may speak different languages, but the way these disciplines are organized and the forces driving them are much more similar than what most artists and scientists would be willing to admit,” Dr. Shamir says. “It’s like two different cultures in two distant continents: The elemental rules and mechanisms that drive the two cultures can basically be very similar, but the different languages and the different customs and traditions make the people of each culture think that their culture is actually different from the other.”
More specifically, because of this experiment and evolving applications, computations, and algorithms, what relationship exists and will exist between computer science and art?
“Basically, computer science is in a stage where it should become driven by discovery and become open to all other disciplines by developing and applying computational methods that solve actual problems in science, engineering, and art,” Dr. Shamir says. “Art is a good example because it might seem something that is fundamentally different from what computers can do, but in fact the quantitative analysis of art done by computers can introduce a totally new approach to the studying of art, an approach that can take art further by providing it with research tools that did not exist before.”
Kelsey Cruz is a journalist in Philadelphia. Although she always struggled with anything science-related in school, she is fascinated by the intelligence and determination of scientists. She strongly believes in The Scientista Foundation and its vision and hopes its message can influence scientists and women everywhere.
RSS Feed
