August 16, 2021

Imagine unexpectedly seeing yourself in a publicly shared photo on social media. Maybe you don’t want anyone to know you’re on vacation for security reasons but get captured on the beach in a stranger’s family photo. Or in a crowd at a sporting event when you called in sick that day. Or maybe it’s just not a flattering picture of you.

Right now, there’s no way to control your image if it inadvertently ends up in someone else’s selfie. But a Mizzou Engineering team hopes to change that. They’re devising a tool that social media platforms could implement to help you dictate who gets to share your face.

Associate Professor Dan Lin and Professor Kannappan Palaniappan, both in the Department of Electrical Engineering and Computer Science, have received a more than $700,000 grant from the National Science Foundation for the work.

“Right now, social media only looks into privacy for owners or co-owners of images, and people in the background are ignored,” said Lin, who is the Principal Investigator. “You could be taking a photo while sightseeing or with friends at a restaurant and don’t think about the people behind you.”

Blurring out strangers in the background isn’t an optimal solution, she said. Not only does it ruin the aesthetics of an image, but it also draws more attention to background dwellers. And with new techniques to deblur photos, other users could simply restore the original image.

Lin’s program would instead replace the faces of those captured behind the subject of a photo. This so-called face-swapping method doesn’t change the focal point of an image and is harder to detect by those with malicious intent. The program generates synthetic faces that do not exist in the real world. If you selected the security feature, you would have the ability to dictate the gender and skin tone of the facial replacement.

Lin is also hoping to add a location factor to the system. While you might not care if you’re seen in the background of a football game, you may not want to end up in a photo at a certain location, such as a public swimming pool or gym.

The research team, which includes collaborators from Purdue University, will develop a prototype using their own facial recognition tools and platform. They’ll test the scalability of the system and its impact on original users. For instance, if the program scans millions of photos to recognize and swap out faces, Lin doesn’t want it to negatively impact the time it takes a picture to upload.

Ultimately, it will be up to each social media platform to determine whether to implement such models, she said.

“We’ll need help from the big companies if they want to provide such services and put these privacy policies in place,” she said. “We’ll test out our prototype, then it’s up to them.”

Lin, a Robert H. Buescher Faculty Fellow, is also director of the I-Privacy Lab.