The Workforce Training Swamp

When southwestern Pennsylvania’s steel mills began falling like dominoes in the 1980s, some 3,400 workers turned to publicly funded workforce training programs for help shifting careers. One was a robotics training course offered by the Community College of Allegheny County (CCAC) and its partner, Westinghouse Electric Corp. Federal dollars flowed to CCAC to pay for the course, which Westinghouse helped design, teach and equip. It was considered a standout.

But its acclaim was not based on rigorous study and data that could confirm whether the course resulted in long-term wage growth for those who took it or whether it set them on a new, sustainable career path. It was based on a survey of those who underwent the training. “Respondents who went through this program were the most laudatory in the assessment of the value of retraining,” according to an analysis of the work history of displaced Pittsburgh steelworkers published by The Institute of Labor Economics.

Decades later, the impact of the nation’s workforce training remains poorly understood, keeping policymakers and funders in the dark about whether the money they’re spending is doing any good, what works, what doesn’t and how programs can be improved.

In Allegheny County, about $20 million a year is spent on workforce training — which averages out to $29,416 for every person who is placed in a job after completing a program. Yet, it’s unknown whether the training improves their long-term earnings and career prospects.

The nation’s workforce training system has become a crowded, uncoordinated patchwork of programs. The federal government alone runs more than 40 workforce development programs across 14 agencies, spending about $17 billion a year.

Approaches have evolved over time and promising practices have emerged, including having employers develop training specific to their industries.

But untangling the sweeping system and answering key questions ranging from which programs work to where programs overlap and efforts are duplicated has proven difficult. One ambitious national study found federally supported training made little difference in the income graduates earn. But with so few rigorous evaluations being done, it’s not clear whether that finding is an anomaly or a red flag.

And in Pennsylvania — unlike Ohio and Texas — restrictions on state-collected data pose an obstacle for researchers looking into the effectiveness of training programs.

“Even though billions of dollars are spent on workforce training programs every year across the country, we really don’t have the data or the high-quality studies to understand what the effects are in the long run in the majority of cases,” said Lee Branstetter, professor of economics and public policy at Carnegie Mellon University.

A COMPLICATED SYSTEM
The largest pot of federal workforce training money comes from the U.S. Department of Labor’s flagship program: the Workforce Innovation and Opportunity Act (WIOA). Other significant federal sources include the Department of Agriculture’s Supplemental Nutrition Assistance Program Employment & Training, the Department of Health and Human Services’s Temporary Assistance for Needy Families and the Department of Education’s Perkins Career and Technical Education.

In addition to the federal government programs, state and local governments and private sector groups, such as the region’s philanthropic foundations and trade unions, provide support for workforce training programs, pilots and apprenticeships.

Even the U.S. Department of Labor acknowledges the byzantine nature of the workforce training system. In a 2019 budget brief, the department bluntly stated that the “system remains fragmented at the Federal level, perpetuating unnecessary bureaucracy and complicating State and local efforts to meet the comprehensive needs of Americans seeking workforce-related services.”

“One of the problems with the system is that it’s very complicated,” said Burt Barnow, professor of public service and economics at George Washington University. “There are lots of programs. They are not always well-connected and it’s not always clear what’s a workforce program and what’s education. The biggest single complication is that we have something in the U.S. called career and technical education (CTE). It used to be called vocational education. It’s virtually identical to workforce training, except that it’s handled by a different congressional committee and it’s mainly handled by states and not the federal government. Because of those features it’s hard to find data and not double count. That makes it very difficult to understand.”

The U.S. Department of Labor’s largest workforce training program, WIOA, had a $4.2 billion budget in 2022. But there is additional federal funding for other programs that target specific demographic groups, including programs for senior citizen job training, programs for veterans, and a program that uses the fees from H1B visas to provide job training for U.S. workers in order to reduce the need for foreign workers.

Federal dollars for workforce training in the City of Pittsburgh and Allegheny County travel through the U.S. Department of Labor to the Pennsylvania Department of Labor and Industry, then down to Partner4Work, the local workforce development steward, where they’re distributed to a network of nonprofits and training providers.

From July 1, 2020 to June 30, 2021, a total of $19,708,476 in federal, state, foundation and private contributions flowed through Partner4Work to support local training programs, according to data provided by Partner4Work. It supported training for 814 people in fields ranging from tech support and web development to brick and block masonry and other building trades. That training resulted in 670 job placements.

But not every organization that provides a specific type of training, such as medical assistant training or programs in the culinary arts, are supported by federal funds. Some, such as CCAC, are part of the local WIOA provider system. But others that provide similar training, such as Bidwell Training Center, are not WIOA contractors. High schools have CTE tracks for students and many local trade unions have apprenticeships that are supported through varied funding sources. “It’s a very crowded landscape,” said Seth Chizeck, a workforce development researcher at Carnegie Mellon University.

MESSY MEASUREMENTS
Although workforce training has been around for decades and billions of dollars are spent on it every year, “it’s hard to know what’s a good program and what’s not,” said Barnow.

Researchers face a host of complications when investigating the effectiveness of the workforce training system. For example, each funding agency involved has its own reporting requirements and measurements for evaluating a program to which they contribute.

The largest federal program, WIOA, uses six performance measures: number of participants served, employment rate in the second quarter after leaving the program, employment rate in the fourth quarter after leaving the program, median earnings in the second quarter after leaving, credential attainment rate and measurable skills gains. Results are published on the Department of Labor website by state across the WIOA program categories, which include several that target specific populations, such as dislocated workers, adult workers and youths.

By those measures, Pennsylvania’s workforce training programs all meet federal performance standards. In 2020, for example, there were more than 6,200 Pennsylvanians in the dislocated workers program statewide. They earned a median income of $8,804 in the second quarter after leaving the program, which was above the national average. That translates into more than $35,000 a year.

“By all standards at the federal level, we are meeting performance,” said Sheila Ireland, deputy secretary of workforce development at the Pennsylvania Department of Labor and Industry. “When you drill down to the specifics, is there room for improvement? Absolutely. At the community [nonprofit]-level there’s just not enough money to be as evidence-based, comprehensive as we would perhaps like.”

Local agencies are able to see real-time reporting from training providers as each class graduates, said Robert Cherry, chief executive officer of Partner4Work. “We can see in real-time if we have classes that are graduating with under 50 percent of credential attainment or placement rate. Then we can go in and talk to those training providers and provide technical assistance to try to bring them up to speed and figure out what the issue is. We have that flexibility.”

But the federal performance measures provide an incomplete picture. They don’t, for example, take into account whether or not the job which people report having after leaving the program is related to the training, or whether it pays more or less than what workers earned before they enrolled in a training program.

And the method of evaluating programs lacks control groups, which prevents comparing the outcomes of program participants to those of workers who didn’t get training. It also raises concern that some programs might select participants who are likely to boost their placement numbers. “That creates perverse incentives for local programs to only offer training to people who, ahead of time, appear to meet the performance measures,” Chizeck said. “It’s necessary to have a control group to gauge the effectiveness of the programs.”

There are few national evaluations of these programs, particularly evaluations that try to determine whether or not these programs lead to long-term employment or salary gains.

The most rigorous was an evaluation of WIOA’s predecessor, the Workforce Improvement Act (WIA), done by Mathematica Policy Research in 2017. That study, which used a control group, found that Americans who received WIA training ended up seeing minimal or no improvement in earnings 30 months later compared to Americans who did not receive WIA training. It suggested that, even if job seekers complete a job training program and earn a credential, they may not earn more or fare better in the job market. One of the drawbacks of the programs noted in the study was that the training offered did not closely align with the needs of local employers. Only half of those who enrolled in the programs said they found jobs because of the training they received.

“Evaluation of these programs is messy,” Barnow said. “For largely ethical reasons, they can’t deny people the ability to get training. The control group could maybe get the same course through the CTE program in the state, so they still might have gotten training.”

APPROACHES EVOLVE
A 2019 audit of Pennsylvania’s workforce development system by Pennsylvania’s auditor general found that its system was “misaligned and not meeting the needs of employers.” Since then, there has been an increased focus on partnerships between training providers and employers.

One program that takes that approach is the University Talent Alliance, a workforce training pilot involving the local nonprofit, InnovatePGH, Partner4Work and CCAC, which tries to connect low-income residents without a four-year degree to jobs in Pittsburgh’s universities. “A job at a university can be transformative — they have great benefits, you are able to go to school if you wanted to get a higher degree,” said Lindsay Powell, director of workforce strategies at InnovatePGH.

The month-long program comes with a stipend and the possibility of getting into the career pipeline of the university system. “The unique piece of this is we work directly with the university so that the hiring manager of that position is a part of the training,” Powell said. “You’ll see exactly where you work, where your desk is. It’s meant to be incredibly hands-on. We can’t say that people are guaranteed a position, but basically if you graduate successfully there are jobs waiting for you in that career, in that profession.”

Workforce training programs at CCAC are supported through a broad range of sources from the U.S. Department of Labor dollars to a grant that pays for veterans and their families to train and obtain a Commerce Driving License. “Right now, there is a major push in collaborating with business and industry,” said Evon Walters, northwest president of CCAC. CCAC leadership consults advisory councils — comprising the business and industry leaders working closely with the faculty and staff to make sure their curriculum aligns with local jobs.

“What we’ve been finding that works are real industry-led programs,” Cherry said. He pointed to a bank teller training program Partner4Work supports called BankWorks. “The bank teller certification is portable between financial institutions. The best training programs have been the ones where we’ve been able to get the whole industry or whole sector to at least agree upon the baseline skills to see what credential or skillset is needed.”

The program has managed to place everyone in the school’s last four classes in jobs, he said. “We would like to mimic this across sectors — create trainings, create credentials for sectors, but also bring these companies together to see what is needed for the industry as opposed to what a [specific] company needs.”

Some innovations in workforce training are showing promise. Among them are approaches for low-wage workers that combine occupational and soft-skill training with services that address issues that can be obstacles to long-term success in the workplace, such as childcare and transportation. The programs resulted in “substantial and persistent” increases in the earnings of people who completed them that ranged from 11 percent to 40 percent, according to a study published last year in the Journal of Labor Economics.

But such evidence-based insights into training programs are the exception rather than the rule. In addition to the breadth and complexity of the workforce training system, restrictions on getting the data they need can stymie researchers studying it.

Personal information about people who get training, such as their income by quarter and work history, enables researchers to track their careers months and years after they complete programs. But access to that kind of information raises privacy issues. Some states, such as Ohio and Texas, have guidelines and confidentiality agreements that allow for sharing such information.

Pennsylvania, however, does not, and that’s a problem for researchers like Branstetter, who investigate how well programs work. “If you know where they were before the program, what they learned in the program and where they went after the program, you can start to evaluate the returns for everyone in a workforce development program the same way as you would look at your portfolio of personal investments,” he said. “If we allow state agencies and researchers to do this evaluation on a large scale, long term, that would be the beginning of optimizing investment across programs.”