Grading health of open-source projects on Goggins’ radar

Sean Goggins stands in a road in front of lawns and trees while wearing a grey shirt.

Mizzou Engineering’s Sean Goggins is a founding board member on a project that aims to answer key questions about the health of open-source projects.

Today, more than 80 percent of the software in any technology product or service is open source. And this trend is growing. According to a recent study by Sonatype, every day the supply of open source across all ecosystems increases by about 1,100 new projects and 10,000 new versions.

This raises important questions about which open-source projects matter. What code should I bet my product, my company or career on? Will those projects grow or shrink? Is the code base stable or changing? Does the project depend on one organization or many? Is the community healthy or hopelessly ill?

Mizzou Engineering’s Sean Goggins is a founding board member on a project that aims to answer these questions. Goggins, the director of the MU Data Science and Analytics Master’s Program and associate professor of computer science, is part of the team behind the Community Health Analytics Open Source Software (CHAOSS) project, supported by the Linux Foundation.  Initial members contributing to the project include Bitergia, Eclipse Foundation, Jono Bacon Consulting, Laval University (Canada), Linaro, Mozilla, OpenStack, Polytechnique Montreal (Canada) Red Hat, Sauce Labs, Software Sustainability Institute, Symphony Software Foundation, University of Missouri, University of Mons (Belgium), University of Nebraska at Omaha, and University of Victoria.

The project aims to:

  • Establish standard implementation-agnostic metrics for measuring community activity, contributions, and health, which are objective and repeatable.
  • Produce integrated open source software for analyzing software community development.

Goggins has spent the last 12 years studying open-source software and the last seven looking at the sociotechnical challenges of software development, with support from the NSF, Sloan Foundation and more. In this latest test, he’ll be working to determine a method of measuring project health along a variety of factors, as well as guidelines as to what determines the health of an open-source community.

The trick is determining health among an array of factors that include all key stakeholders in a project — technology foundations, corporations, contributors and consumers, among others.

“In open-source projects, because the main currency exchanged isn’t money — it’s goodwill, it’s prestige, it’s things that are harder to measure — it’s harder to compare projects,” he explained. “Everyone involved with a project is going to assess it through their own lens. What we’re trying to do is provide a set of collective measures that people can use and assemble in their own ways to determine value in their own ways according to their own value systems.”

The even trickier part comes into play when you consider that not every stakeholder’s value system values the kind of diversity in computing and tech that many, Goggins included, are hoping to grow. The field can be particularly harsh for women, and the line Goggins wants to walk in this project is creating methods of measurement that don’t provide incentive to disregard underrepresented groups in computing.

“In particular towards women,” he said. “It’s a big deal. It’s a huge problem. Right now, we’ve tapped a very large set of communities and corporations that are interested in metrics, and we’ve put together a great coalition of people from academia and industry, but we want to make sure that what we do doesn’t cause harm.”

Diverse input into open-source projects can only serve to make them better by broadening the range of ideas. And projects with more activity, more technological issues solved more quickly, more contributors are also potential indicators of a healthy project and community. But different users, different organizations, different contributors are in search of different things before they decide to sink time, effort and/or money into a project. Which is why creating a road map of sorts rather than a one-size-fits-all set of metrics is key for a lot of key players in the open-source landscape.

“The Linux Foundation is working with us, Github is working with us. The Mozilla Foundation is supporting our work,” Goggins said. “We have the Sloan Foundation supporting our work, all in the interest of trying to understand this.”

Enter your keyword