February 12, 2021

A movie still from “Skyfall” used in Maurer’s original research of color palettes in films.

Filmmakers rely heavily on color palettes to invoke a mood or set a tone in cinema. But do human brains actually pick up on those strategic color schemes? That’s what one Mizzou Engineer hopes to discover over the course of the coming year. Gillian Maurer, director of the Information Technology (IT) Program, is conducting research to mathematically quantify color palettes in some of the most visually stunning films of the past 25 years.

“The question is whether humans prefer artistically motivated color palettes or mathematical ones generated by empirical data,” she said.

A Big Data Approach

To get the answer, Maurer and her research team plan to capture the Red, Green, Blue (RGB) color value of every single pixel of every frame of 50 movies. That’s a lot of data—about 13.4 trillion pixels to be exact. Specifically, the team is analyzing movies that have won an Academy Award or been runner-up for best cinematography over the past 25 years (think Blade Runner, Inception, Avatar and Slumdog Millionaire).

Maurer

“The interesting element of this research is that it marries a traditionally artistic part of filmmaking to the engineering approach of big data analytics,” Maurer said.

That’s where technology comes in. The team will write code that accelerates graphics processing and train a computer to collect the information. Once they’ve set up the process, they will be able to mathematically quantify colors from each film.

When they have the data, researchers plan to test the findings on viewers. That will determine whether the quantifiable colors match what viewers think they’re seeing.

“If we have the three palettes – mathematical average, mathematical maximums, and arbitrary or artistic – we can use that data to assess which palettes participants prefer based on images from the films. This helps us to understand if there are connections between the statistical data collected and the more artistic preferences employed by their creators,” Maurer said. “It’s interesting because art and mathematics have been linked for centuries. Hopefully we discover more meaningful connections.”

Process is Paramount

Maurer is a director and producer and former director of film production in the Film Studies Department at Mizzou. She brought her love for filmmaking to engineering, where she teaches media post-production and project management.

In 2019, she challenged students in an IT capstone class to map RGB values in three-dimensional space. The students graduated before the work was completed, but it laid the foundation for her current study.

This year, Maurer formalized the project with students from the IT undergraduate research course.

“Engineering research tends to lean towards traditional fields and industries unrelated to the arts. This is an exceptional way to apply competencies we teach in our program to research that excites and connects on an artistic level,” she said. “Big data is an ever-growing field of interest, as is graphic processing unit, or GPU, acceleration and software engineering. We’re giving our students an opportunity to engage in some of the most exciting fields in IT while applying important skills to research.”

And while her research is focused on film, the deep learning process itself could have other applications such as object identification.

“Say you have a ship lost at sea and autonomous drones are searching thousands of square miles of surface area. Those photographs can be analyzed for deviations in pixel data that can then later be reviewed by humans,” Maurer said. “It’s an enormous job made easier by machine learning, autonomous systems, and GPU cluster processing. This can speed up the process of identifying objects from satellite imagery or alter the way in which we collect geographic data.

“From photographs to live feeds from autonomous robots, the important thing to remember is that the process is paramount. We’re researching art, using science as its backbone, and technology as its conduit.”

Want a unique IT experience? Apply now!