Timely data analysis needed to create education policy

As the Trump Administration considers shifts in U.S. educational policy, Matthew Chingos, director of Education Policy Program at the Urban Institute, explains the importance of timely data in making sound decisions at the local, state and federal levels.

Julianne Hill: Education is an emotionally charged issue, especially in the current U.S. political climate. Intense debates are happening within families alongside those among policy makers. Your research is based on hard data rather than personal experience and ideology. Does your research support – or diffuse – strong emotions?

Matthew Chingos: Education decisions have long been dominated by personal experience. Everyone believes they are knowledgeable about education because everyone has been to school. And that’s what we do when we don’t have access to scientific information – we fall back on that personal experience. To prevent that, we give people data. Of course, people are not going to ignore their intuition simply because they have data. But, if we give factual information, we can elevate the role that information has in policy making.

We are building a portal of national data sets from k-12 schools, school districts, college and universities – things like school location, demographics, revenue and spending.

The data is there but disorganized, and we will clean it up and make it accessible to the public. It will help create more policy decisions at the federal, state and local level driven by relevant information and analysis.

“Everyone believes they are knowledgeable about education because everyone has been to school.”

JH: College and its costs are a hot button topic. What is the thinking behind the evidence-based policy proposals on higher education that you’re preparing for the new administration and Congress?

MC: We are looking at accreditation, simplification of the financial aid process, transparency, accountability and income-based repayment of student loans.

JH: Student loan policy is evolving in America. What does data show us about student debt and its impact on individual lives?

MC: Last year I published a book, “Game of Loans: The Rhetoric and Reality of Student Debt.” In it (with co-author Beth Akers), we bring data that shows where the bigger problems are. We all hear about people with huge debt balances, and it seems like that’s the problem. But a lot of people who borrow a lot of money get degrees in medicine or law, and they can pay that debt back with the money they will make in their careers.

Our data shows the real issue is the people who borrow but who don’t finish college. They don’t make as much – and so they struggle more under that debt. That’s important to know. We can use data to find where the most important challenges are, who is struggling the most, and which policies can be created to help them.

JH: School vouchers – state-funded scholarships that pay for students to attend private rather than public schools – are a hot topic in the U.S., part of the Trump Administration’s school choice discussion. Is there data to support school vouchers?

MC: I did a study a few years ago that looked at vouchers in New York City that had been distributed by a lottery system to see if it had made a difference in the long term regarding college attendance. Overall, it didn’t make much of a difference. But for certain subgroups – African-Americans, Latinos – it did, with a larger percentage of those students who had vouchers going to college than those who did not.

So, we at the Urban Institute are now looking at a few places around the country that use vouchers, like Washington, D.C., and Florida – and checking the impact that receiving a voucher in elementary school has on the likelihood that the student that will go on to college. We’re still collecting data on that, but it will be interesting to see.

JH: You’ve done work examining the states’ roles in educational policies. How can we analyze states’ performance in education?

MC: States certainly do have a big role in education, especially when you consider that 10 percent of educational funding comes from the federal government, while 45 percent comes from the state and 45 percent from local governments. There are a lot of federal policies that flow through states, like testing kids in subjects. Those are federal mandates implemented by the states – so states play an important role.

We looked at data from the National Assessment of Educational Progress, which is the only one given to kids nationally to assess math, reading science, and history, and we created “America’s Gradebook.” That lets you see the results of the NAEP test, and identify which states perform better or worse.

We have made provisions so that you can compare demographically similar states, so you don’t compare Massachusetts to Mississippi. But there were surprises. Florida and Texas students were doing quite well even with more disadvantaged populations. More research is needed to understand why that is so, but it is a good starting point.

“Academics and universities are always doing rigorous work, but it’s often not done in a timeline or in a format that helps those making policy decisions.”

JH: Education policy doesn’t exist in a vacuum. It’s influenced by other things like housing, criminal justice and poverty. Is it possible to tease out strictly education’s performance?

MC: At the Urban Institute, we are very lucky to have experts on those various policies. We are always collaborating with our colleagues to better understand how these different policy moves interact with one another.

For instance, we are now looking at student transportation – how far students live from schools in cities where there is school choice. What are transit policies and options in cities like D.C., New Orleans, Denver, New York and Detroit? Then, we check the implications for students’ access to different kinds of schools.

 JH: Tell me about the Urban Institute’s new Education Policy Program.

MC: You know, academics and universities are always doing rigorous work, but it’s often not done in a timeline or in a format that helps those making policy decisions. Maybe it comes out in an academic journal that no one reads, or maybe it is complete two months or two years after policy decisions need to be made.

Meanwhile, think tanks and advocacy groups have smart people who know a lot and have good insight on issues, but often they don’t have the relevant technical capabilities to handle data.

We at Urban saw a space for an education policy program that was rigorous like academics’ work but timely like policy analysts’. Our idea is giving the right information at the right time in right format for policy makers.