Signals of Quality
NEW ORLEANS — Any effort to measure the value of education and training is sure to face serious blowback. So it’s not surprising that the core message from groups behind a project to gauge the quality and economic payoff of nondegree credentials is that such a thing could be done at all.
“We think that measuring quality is possible,” said Stuart Andreason, managing director at the Burning Glass Institute. Yet he added that, “it’s never been done before.”
Andreason was speaking here this week at the annual meeting of Jobs for the Future, which acquired the Educational Quality Outcomes Standards (EQOS) framework last year. After receiving a $2.9M grant from the new GitLab Foundation in April, JFF has teamed up with the Burning Glass Institute to try to make the standards a reality.
Part of the urgency behind the work is a “real moment of time” where employers are willing to think differently about hiring, said Ellie Bertani, the GitLab Foundation’s president and CEO.
The two groups gave a room full of experts, many of whom work on credential and outcomes data in postsecondary education, an early look at what they’ve developed during the last two months. The result is an attempt to identify “signals of quality” about short-term credentials and how they translate to employment and career advancement.
“It’s built on a lot of public data,” said Andreason, who previously led the Center for Workforce and Economic Opportunity at the Federal Reserve Bank of Atlanta. The framework is designed to be flexible and to be able to add data sources as it evolves. The first draft of metrics unveiled this week stretch across five categories:
-
Employment Outcomes: Where do workers end up and how much do they make?
-
Access: Do workers in different demographic groups earn the credential? How much does the program cost?
-
Demand: Are occupations that require the credential in demand and is the credential recognized in the market?
-
Learning: What skills do workers learn when they earn a credential?
-
Opportunity: Do credentials lead to good career pathways?
The two groups said they wanted to kick off the discussion about EQOS with a real test run. The opportunity category included 17 metrics, for example, ranging from completer access to paid leave and health insurance to their years to a first promotion and the number of promotions earned.
The employment outcomes also included 17 metrics, including wage levels and gains over the years. The wage data used were estimations based on occupations, or in some cases on data from employers about specific titles. However, the groups said more data could and should be added, with contributions sought from states, employers, and education providers. They also plan to begin adding weight to the various metrics, with inevitable tweaking to follow.
“Some of this is not going to be welcome information,” Andreason said. “It’s much less carrot-based and more stick-based.”
The initial test run featured five providers each across seven types of credentials, including those in IT, healthcare, commercial driver’s licenses, and OSHA certifications. It featured letter grades for the providers, while not naming them. However, by the next day the EQOS team was already considering whether to drop the grades based on feedback, perhaps swapping in another way of rating providers and programs.
The concerns raised by experts here largely were along expected lines, and sometimes colored by self interest. Even so, they signaled the uphill struggle EQOS may face in coming months.
“It’s about trust that the data is going to be used responsibly,” said one observer.
Another pointed to the inherent complexity of deciphering which problems a credential should solve and for whom. And one expert asked whether it’s possible to fairly compare the quality of OSHA certification programs, for example, given regional differences in economic opportunity.
While acknowledging skepticism about the effort, Matt Zieger, a senior program officer at the GitLab Foundation, called for partners, data contributions, and suggestions for improving the metrics.
The Kicker: “We’re building roads not cars. These are your roads, too,” he said. “We are in the early days of this.”
Microcredentials in Louisiana
Whether you’re a believer in short-term credentials or not, a growing number of states are backing the potential on-ramps to careers and degrees (more on this next week).
For example, Louisiana last year tapped federal pandemic funding to begin offering professional certificates on Coursera’s platform to jobseekers across the state. These nearly 30 online and self-paced credential programs, most of them created by Big Tech companies, are available for free to Louisiana residents, who can receive support services as well.
“This is a way to start small,” Ava Cates, secretary of the Louisiana Workforce Commission, says of the professional certificates. “It makes you more marketable and it gives you job skills.”
Roughly 3K Louisianans have registered through the program with Coursera so far, the company says, with 8,600 enrollments and 2,200 course completions.
The state’s collaboration with the online course platform followed its work to expand broadband access to bridge the digital divide for its many rural and low-income residents. Cates says that project, which also benefited from federal funding, elevated the need to add online and flexible educational opportunities for newly-connected people.
Coursera worked with the state to bring the credential programs into job centers, supplementing some courses so learners can meet eligibility requirements for training benefits.
“Building it into the existing systems” and the incentives they create is a winning strategy, says Jeff Maggioncalda, Coursera’s CEO. Beyond the flexibility offered by professional certificates, he says learners are drawn by the promise of being “broadly educated and specifically skilled.” The strong brands of Big Tech companies also are a bonus.
The U.S. has the highest demand for industry microcredentials of any country globally, Coursera found in its newly released global skills report. Learners amassed 1.3M enrollments in entry-level professional certificates in the last year (a 22% increase).
Louisiana wants to expand the uptake of professional certificates on Coursera, and will continue to market the programs to residents. The commission also plans to collaborate with the platform to offer learners new credentials for high-growth fields like EV battery production.
“We’re not just focusing on jobs that are currently available,” says Andrea Morrison, assistant director of the commission’s office of workforce development. “We’re not waiting on the colleges to develop that curriculum.”
Equity and Credential Data
An intentional focus on the publishing of transparent, linked, open data about credentials is needed to drive systemic change in postsecondary education and training.
That premise underlies newly released recommendations from a coalition of experts convened by Credential Engine, a nonprofit group that’s seeking to map the credential landscape.
The recommendations are organized into three tiers, beginning with data points the coalition says all providers should be able to analyze and publish now (costs, transfer, and earnings), moving into information they should be working on (stackability and support services), and then into a third level where the field is headed (career outcomes and job placement).
“You have to be constantly using data to see, ‘Are we having the intended consequences?’” says Cristen Moore, associate principal and director of equity and change management at HCM Strategists, who contributed to the project. “There has to be a clear message to policymakers on both the economic and the moral imperative.”
Likewise, the report says the power of data rests in analysis that is disaggregated by race and ethnicity, income, gender, and other key characteristics.
“Don’t we want to make sure that students know what they need to know to have successful outcomes?” says Scott Cheney, Credential Engine’s CEO. “That’s pretty noncontroversial.”
Notable and Quotable
JFF last week announced a new “North Star goal”—that 75M people who face systemic barriers to advancement will work in quality jobs within 10 years. This group of people includes those without four-year degrees. To move the needle for them will require real progress on skills-based hiring amid the flurry of employers dropping degree requirements.
“Skills-based hiring as a culture shift is very much underway and growing,” Brett Parton, acting assistant secretary at the U.S. Department of Labor, said at JFF’s conference this week. “What’s the infrastructure and systems change that are going to make that real?”