Peltzman uses interstate data to explain changes in school performance, concentrating on variables that affect the political system controlling the schools. There is a serious risk that such a procedure may produce false positives. It is well established that the family background of students is a major determinant, perhaps the major determinant, of performance. Variables describing family background by state are likely to correlate with other state variables, including the sort Peltzman is interested in. Results that appear to show the influence of unionization, or centralization of school financing, or the needs of employers may merely mean that those variables are serving as proxies for environmental variables.
This is a familiar problem in statistics, and one for which multiple regressions are a common solution, but it is particularly serious here for two reasons. The first is that there is some reason to suspect that environmental variables may be much more powerful determinants of the dependent variable than political variables. The second is that it is unclear what environmental variables at what dates should be controlled for. A graduating senior has spent twelve years in the school system and eighteen in his family. If, as some would argue, it is early childhood development that determines later performance, the relevant environmental variables may enter with lags of well over a decade. Peltzman attempted to control for environmental effects by adding several environmental variables to his regressions and found that they did not have much effect on the results, but that may mean only that he did not try the right ones. In order to explore this possibility, I experimented with a number of additional variables.
Table 1 shows part of a correlation matrix relating Peltzman's measure of performance growth, his union variables, and a number of variables relevant to student backgrounds.[1]
|
|
School 72-80 |
Performance 81-89 |
DBRK |
FAM80 |
FAM60 |
Inc70/60 |
Cons |
|
---|---|---|---|---|---|---|---|---|---|
1 |
DBRK |
0.141 |
-0.225 |
1.000 |
|
|
|
|
|
2 |
FAM80 |
-0.546 |
-0.144 |
-0.445 |
1.000 |
|
|
|
|
3 |
FAM60 |
-0.508 |
-0.307 |
0.171 |
0.806 |
1.000 |
|
|
|
4 |
Inc70/60 |
0.359 |
0.282 |
-0.441 |
-0.404 |
-0.736 |
1.000 |
|
|
5 |
Cons |
0.311 |
0.314 |
-0.367 |
-0.295 |
-0.567 |
0.513 |
1.000 |
|
6 |
pnea |
0.114 |
-0.263 |
0.444 |
0.142 |
0.450 |
-0.462 |
-0.467 |
|
7 |
paft |
-0.193 |
-0.260 |
0.544 |
-0.082 |
0.269 |
-0.209 |
-0.522 |
|
The most striking result appears on lines 2 and 3. Fam80 and Fam60 show the percentage of children under 18 from broken homes in 1980 and 1960 respectively. Either variable, by itself, fits changes in school performance from 1972 to 1981 with a correlation coefficient greater than .5 . The change in the percentage of children from broken homes from 1960 to 1980 (not shown above) correlates much more weakly with school performance--and with the wrong sign.
We would expect broken families to result in worse educational outcomes, but why is it the level rather than the change that is significant? One possible explanation is long time lags; the level in 1960 may be a proxy for changes prior to that date whose effect shows itself in students graduating in the seventies. A second answer is that we may be observing the combined effect of a family variable and a change in the schools. Charles Murray, in Losing Ground, argues that one of the problems with the educational system during the sixties and seventies was a shift of attention from the best to the worse students. If so, we might expect states where there were a lot of problem students, due to a high level of broken families, to have a particular sharp decline in educational performance over that period.
In addition to FAM8 and FAM6, other variables that correlate more strongly with school performance 1972-81 than do Peltzman's union variables are Inc70/60 (the percentage growth in real per capita income from 1960 to 1970) and Cons (the sum of the Republican vote percentages in the presidential elections of 1964 and 1972).
|
Coefficient |
t-ratio |
---|---|---|
Constant |
4.15424 |
0.754 |
"Real" Expenditure per pupil 61-81 |
2.60635 |
1.25 |
pnea |
2.77758 |
3.46 |
paft |
-5.03255 |
-2.99 |
FAM80 |
-14.1430 |
-2.89 |
RIn70/60 |
3.36275 |
1.62 |
Enrollment per district 78/57 |
-0.110184 |
-1.61 |
In order to see whether these variables might explain away Peltzman's results, I reran several of his regressions, including various selections of these additional variables. Table 2 shows the result of one such regression, corresponding to Peltzman's Table 5(3) with a few changes. The main effect of the added variables was to sizably increase -- from .29 in the regression 3 on Peltzman's Table 5 to .495 for the regression shown in Table 2 above.[2] Although adding additional variables weakened the significance of some existing variables and increased that of others, the general pattern remained the same, suggesting that Peltzman's results are reasonably robust--although there is always the possibility that they would be eliminated by the inclusion of some other environmental variable that neither he nor I have tried.
An alternative approach to avoiding confusion between the effects of political and environmental variableswould be to find political variables that we have good reason to believe are independent of environmental variables--ideally, political outcomes that are the result of random causes. I have one suggestion along those lines. We would expect electoral results to correlate in a continuous fashion with population characteristics such as income, education, or family structure; as certain characteristics become more common, vote totals for one party or ideology should change accordingly. Political outcomes, however, depend on electoral outcomes in a highly discontinous fashion. While any change in a party's vote total affects its political strength, the increase from 49.9% to 50.1% has a much larger effect than an increase from 50.1% to 50.3%.
This suggests that one could distinguish the political outcome from the electoral outcome in a regression by including one variable for percentage of votes by one party, and a second showing the number of years during which that party held the governorship. The first variable should pick up effects associated with non-political variables that correlate with voting patterns, making the second a reasonably good measure of the consequences of different political outcomes.
Peltzman discusses a variety of possible causes for the decline in American schooling, but his fundamental model is one in which the change is due to a change in the power or objectives of the interest groups influencing the political institutions that control the schools. While I agree with the underlying idea of looking for an explanation based on the behavior of those institutions, I would like to suggest a slightly different approach--one based not on changes in the interest groups but on changes in the level of government at which the decision is made.
Such changes are important for at least three different reasons. One is that different levels of government may be subject to different interest groups with different interests. A firm that is the main employer in a small town is a highly concentrated interest with regard to the school board of that town, but part of a rather dispersed interest with regard to the state legislature and a still more dispersed interest with regard to Congress, so its influence will be much greater if local school districts are small and independent.[3]
A second reason why the level of government matters is that the relevance of the interest group model may depend on the level of government. In a naive model of democracy, elected politicians do good things because because that is what the voters who elect them want. This is a poor description of the working of large governments because of rational ignorance; it is rarely worth the cost to an individual voter to discover whether his elected politicans are acting in his interest and do something about it. The result is a political system dominated by concentrated interest groups. But the naive model may work fairly well for a small town school board, where parent voters get information about the school's performance five days a week, at no cost, and where the number of voters is small enough so that individuals or small groups may hope to have some influence. If, as seems likely, the individual voter cares more about the educational quality of the school his children go to than the average of the interest groups affecting the schools, this effect will tend to produce a positive correlation between local control of schooling and schooling quality.
A third reason size matters is that it affects the degree to which governments compete with each other for taxpayers. In a system where schools are locally run and funded from local taxes, the resources available to the local government depend to a substantial degree on how good the schools are, since many taxpayers decide where to live in part on the basis of school quality. That provides the local government and the local school administration with a direct interest in educational quality, quite aside from political activity by parents or other groups interested in good schools. This third effect depends on both local control and local funding. If schools are funded entirely from state taxes, local control may reduce the problem of rational ignorance among the voters but it will not give local school districts any incentive to use educational quality to attract taxpayers.
All three of these arguments suggest that a movement away from local control and funding of education might well result in a decline in educational quality. This seems consistent with our experience. From 1946 to 1974, the number of school districts in the U.S. fell about six fold and the number of students roughly doubled, producing an increase of more than an order of magnitude in the number of students per school district. Since 1974 the number of students per school district has been roughly constant. That pattern fits reasonably well the pattern of educational outcomes over time. In addition, as Peltzman points out, there was a substantial shift from local to state funding of schooling over the period when school performance was declining.
To test the possibility that the increasing size of school districts was in part responsible for the decline in educational achievement, I ran a number of regressions using changes in number of school districts or enrollment per district to fit changes in educational achievement. Using the ratio of enrollment per districts in 1978 to enrollment per district in 1957 to fit the change in achievement from 1972-1981 gave an of 18.1% and a t=-3.24. The effect was reduced substantially (t=-1.67) if I included in the regression a measure of children from broken homes in 1980--which, as discussed earlier, correlates quite highly with school performance in the following two decades. The effect of this measure of changing school district size on changes in achievement from 1981 to 1989, was not significant. Adding this variable to the regression reported in Peltzman's Table 5(3) reduced the significance of his measure of school centralization (the roll of state funding), as would be expected, but did not eliminate the significance of his other variables.
These results may well underestimate the importance of increasing size of districts, due to a problem with the data. It is easy to calculate the ratio of enrollment to number of school districts in a state, which is what I did. But this gives an average over districts, not over students; it tells us how many students the average district has, but not how large a district the average student is in. A state with one district containing a million students and one containing a thousand appears, by this measure, to have an average of 500,500 students per district. But in such a state, 99.9% of the students are in a million student district; district size averaged over students is 999,002 students per district.
How serious, in practice, is the distortion due to averaging over districts rather than over students? For the state of California for 1977-8, I was able to obtain enough information on individual districts to calculate an approximate answer to that question. Averaged over districts, the figure was about four thousand students per district. Averaged over students it was over eighty thousand. If such differences are typical, and if they vary significantly from state to state, they introduce a large element of noise into my regression. This suggests that it would be worth trying to calculate, for each state, the figure averaged over students, and seeing whether that statistic predicts school performance even better than the statistic I measured.
The following summarizes an argument Peltzman makes for scepticism with regard to the evidence that "resources do not matter:"
If the marginal return of expenditure on educational achievement is independent of the student's background, and if the utility function being maximized by the government has declining marginal utility of student achievement, the government will tend to allocate more money to students with worse backgrounds, although not enough more to bring their achievement up to the level of those with better backgrounds. Unmeasured background characteristics will then correlate positively with achievement and negatively with expenditure, creating the illusion of a negative correlation between expenditure and achievement. The fictitious negative correlation may cancel out a real positive correlation, making it look as though expenditure has no effect on achievement.
The problem with this argument is that it proves too much. If we substitute the employer's utility function for the government's, it implies that employers typically invest more money training their less able employees. If we substitute the parent's utility function, it implies that poor but aspiring parents scrimp to provide additional education to their least able children. If we substitute the utility function of the political structure controlling state higher education, it implies that the state universities admit from the bottom instead of the top of the distribution of graduating seniors, and specialize in remedial rather than advanced education. Casual observation suggests, in each case, the opposite pattern. That implies either that the marginal return is higher for expenditures on abler students or that utility functions for educational achievement exhibit increasing marginal utility in the relevant range.
If this conclusion is correct, the argument Peltzman offers works in reverse. Expenditures will tend to be higher, not lower, on students with better backgrounds. Unobserved variation in background will correlate positively with both expenditure and achievement, creating the illusion of a positive correlation between expenditure and achievement. The measured correlation will then overstate the actual correlation. If, as Peltzman reports, the observed correlation is negligable, this argument strengthens the conclusion that resources do not matter-or converts it to the even stronger conclusion that increases in resources reduce achievement.
Peltzman finds that an increase in the skill level of employees of industries concentrated in a state correlates negatively with changes in educational quality. I find this result less surprising than he does, for two reasons. One of them he discusses. A local industry that hires workers directly out of high school will be more interested in the quality of local education than one that hires college graduates, since a larger fraction of its employees will be graduates of the local school system.
The second reason is more complicated. During the sixties and seventies, the percentage of college educated workers was rising, but the fraction of the population going to college was roughly constant. The explanation of this apparent paradox is that the fraction of the population graduating from college was much higher in this period than in preceeding decades. The effects Peltzman is looking for depend on changes in the skill level of new hires, but the variable he constructs measures changes in the skill level of the entire work force. A firm for which college graduates made up 8% of the workforce at the beginning of the period but 25% of new hires (roughly the average figures for the adult population at the time) would report a steadily rising skill level, even if the skill level of new hires remained unchanged throughout the period.
So industries that Peltzman interprets as having a rising share of College educated hires may simply be ones that started the period with a work force that contained few college graduates, and had a larger but constant fraction of college graduates as new hires during the period. Such industries would have been a political force in favor of good local schools at the beginning of the period. By the end of the period the shift away from local control of education would have substantially reduced their influence, quite aside from any change in the composition of the industry's work force or new hires. This provides a second possible explanation for the negative correlation between skill level of industry and quality of schooling.
This paper makes a good start on the project of using interstate variation to explain the change in school performance on the basis of changes affecting the political institutions that control American schools, and produces some suggestive results. The main improvements I would suggest are a greater effort to find and control for the relevant non-political variables, and a more careful examination of the possibility that the drastic centralization of control and financing of the schools during the post war period may have been one of the crucial political changes responsible for the decline in school performance.
David Friedman
Back to my home page.