Overall, we are able to estimate both the academic and the labor market returns of 230 teach-ers. We cannot run equation 1 for courses that have no contemporaneous nor subsequent
22For simplicity the figure only shows results for the classes of the Management program. In the pooled random effect model the coefficients on the explanatory variables are constrained to be the same across courses.
courses.23 For such courses, the set Zc is empty. Additionally, some courses in Economics and in Law&Management are taught in one single class.24 For such courses, the computation of the academic returns of teaching based on future exam grades is impossible sinceSc= 1.
When we consider contemporaneous academic returns or labor market returns we do not face the above constraints, as incomes and contemporaneous grades are available for all stu-dents and all courses. In fact, we can estimate these effects for slightly more teachers (242 in total) but, given that our main purpose is the comparison of different types of returns, we prefer to focus on the subsample for which we can estimate all three indicators.
Table 4 shows test statistics from the second-step equation 3 for the three indicators of teaching quality that we construct. Given the large set of regressors used in these models (25 in total), we have grouped them into categories: three categories for the characteristics of the classes (class size, class composition and class time and room) and four categories for the characteristics of the teachers (coordinator, demographics, citations and academic rank). The exact variables included in each category are listed in the notes to the table. Two categories only include one variable, namely class size and coordinator (a dummy equal to one when the teacher of the class is also the coordinator of the course) because we deem these variables particularly interesting. For each category the table shows the F-test and p-values for the null of joint significance of the coefficients of all the variables in the category. The table also reports the partial R-squared obtained from a partitioned regression where the full set of dummies for degree program, term and subject area are partialled out.25 All results are shown for three different specifications, one with only the class characteristics as explanatory variables, one with only the teacher characteristics and one with both.
Results indicate that all the explanatory variables included in these second-step regres-sions have very limited explanatory power. Although some individual regressors are statisti-cally significant in some specifications, none of the reported F-test allows rejecting the null.
This is consistent with both the random allocation and the common finding in the literature about the lack of correlation between student outcomes and teachers’ observables (Hanushek
& Rivkin 2006, Krueger 1999). Overall, the class characteristics alone explain slightly less than 10% of the variation in student future academic performance (panel A), slightly less than 3% of the variation in earnings (panel B) and slightly less that 6% of the variation in contemporaneous grades (panel C). Teacher observables have even less explanatory power: 3.6% for academic performance, 2.5% for earnings and about 1% for contemporaneous grades. Interestingly, pro-fessors who are more productive in research do not seem to be better at teaching.26 When conditioning on both teacher and class characteristics the (partial) R-squared remain low.27
23For example, Corporate Strategy for Management, Banking for Economics and Business Law for Law&Management (see Table A-1).
24For example Econometrics for Economics students or Statistics for Law&Management (see Table A-1).
25The full results are available upon request.
26See Marta De Philippis (2013) for a formal evaluation of research incentives on teaching performance using our same data.
27The Partial R-squared reported at the bottom of the table refer to the R-squared of a partitioned regression
Our final measure of the returns to teaching are the residuals of the regressions of the es-timatedαs on all the observable variables, i.e the regressions reported in columns 3 of Table 4. In Table 5 we present descriptive statistics of such measures. For completeness, the lower panel of the table (panel B) also reports the same results computed without conditioning on the teachers’ observable characteristics (i.e. residuals of the regressions in columns 1 of Table 4).
The average standard deviation of the academic returns is 0.048.28 As discussed in Section 3, this number can be readily interpreted in terms of standard deviations of the distribution of students’ grades. In other words, assigning students to a teacher whose academic effectiveness is one standard deviation higher than their current professor would improve grades by 4.3% of a standard deviation, corresponding to approximately 0.6% over the average.
This effect is comparable to the findings in Carrell & West (2010), who estimate an increase in GPA of approximately 0.052 of a standard deviation for a one standard deviation increase in teaching quality. To further put the magnitude of our estimates into perspective, it is useful to also consider the effect of a reduction in class size, which has been estimated by numerous papers in the literature (Joshua D. Angrist & Victor Lavy 1999, Krueger 1999, Oriana Bandiera, Valentino Larcinese & Imran Rasul 2010) and also on the same data used for this study (De Giorgi, Pellizzari & Woolston 2012). The estimates in most of these papers are in the range of 0.1 to 0.15 of a standard deviation increase in achievement for a one standard deviation reduction in class size, thus about two to three times the effect of teachers that we estimate here. Notice, however, that one of the obvious mechanisms through which reducing the size of the class affects performance is the possibility for the professors to tailor their teaching styles to their students in small classes, that is an improvement in the quality of teaching. In other words, our estimates of teaching quality are computed holding constant the size of the class whereas the usual class size effect allows the quality of teaching to vary.
In Table 5 we also report the standard deviations of the academic returns to teachers in the courses with the least and the most variation. Overall, we find that in the course with the highest variation (macroeconomics in the Economics program) the standard deviation of our measure of academic teaching quality is 0.14, approximately 3.3 times the average. This compares to a standard deviation of essentially zero in the course with the lowest variation (accounting in the Law&Management program).
The second column in Table 5 reports similar statistics for the labor market returns of pro-fessors, measured by the conditional average earnings of one’s randomly assigned students, as explained in Section 3. A one standard deviation better professor leads to an increase in earn-ings by 5.4% of a standard deviation on average. This translates in an annual increase of gross income of about 1,000 Euros, slightly more than 5.5% over the average. Also the labor market returns are vastly heterogeneous across subjects, with the variation reaching 18% of a standard deviation in earnings for mathematics in the Economics program and being close to zero for
where the dummies for the degree program, the term and the subject area are partialled out.
28The standard deviation is computed on the basis of theshrinkagemethod described in Section 3.
management III in the Management program.
Given that most of the existing literature uses contemporaneous achievement to construct value-added measures of teacher quality, the third column of Table 5 shows results based on such student outcomes. We are very much in line with previous findings: most existing papers estimate effects in the order of 10% of a standard deviation for a 1 standard deviation change in teacher quality and we report an average effect ranging between 7.1% and 11.6% (when excluding teacher attributes from the set of controls).
In the lower panel (panel B) of Table 5 we report the same descriptive statistics for our measures of professors’ quality that do not purge the effect of the observable characteristics of the teachers. Consistent with the finding that such characteristics bear little explanatory power for students’ performances (see Table 4), the results in panels A and B of Table 5 are extremely similar.
By restricting the set of students to those of high or low ability, defined as those whose performance in the attitudinal entry test is above or below the median, it is possible to replicate the procedure described in Section 3 to produce professor effects for each of these two cate-gories of students. The descriptive statistics of these indicators are reported in Table 6 and their analysis allows understanding whether it is the best or the worst students who benefit the most from good teachers and in what dimension.29
When considering academic performance the dispersion in teachers’ returns appears to be rather homogeneous across student types, with an average standard deviation of about 0.066-0.067 in both cases (panel A) and similarly for the contemporaneous returns (panel C). Larger differences emerge when teaching quality is measured with students’ earnings (panel B). In this case, the low-ability students seem to benefit from effective teaching substantially more than their high ability peers, the average standard deviations being 0.178 and 0.097, respectively.
When looking at the minimum and maximum effects it appears that the entire distribution of labor market returns is shifted to the right for the low-ability students.
These results appear consistent with the view that teaching is a multidimensional activity as in Jackson (2012). Furthermore, the comparison between students of different ability levels suggests that the degree of complementarity between teacher and students skills varies both across teaching activities and student types.
One obvious question that one can ask with these data is whether the professors who are best at improving the academic performance of their students are also the ones who boost their earnings the most. In Table 7 we estimate the correlations between our alternative measures of the returns to teaching, conditional on degree program, term and subject area effects. In these regressions we weight each observation by the inverse of the standard error of the the dependent variable and we bootstrap the covariance matrix.
Results show a positive and significant correlation between the academic and the labor
29Notice that the effects reported in Table 5 cannot be simply derived as averages of the effects for high- and low-ability students in Table 6.
market returns of professors when using data on all the students in each class, a finding that is remarkably consistent with Chetty, Friedman & Rockoff (2014a) and Chetty, Friedman &
Rockoff (2014b), despite the stark differences in the settings and the methodologies. Chetty, Friedman & Rockoff (2014b) estimate that a one standard deviation increase in teacher value-added is associated with an increase in total earnings (the long-run outcome that is more similar to our taxable income) of 353 USD per year, corresponding to approximately 0.015 of a stan-dard deviation (see Table 3 on page 2655 in Chetty, Friedman & Rockoff (2014b)). Based on our estimates we can calculate a similar effect of the order of 0.018 of a standard deviation.30
Despite being significant, the magnitude of the correlation between the academic and labor market returns of the teacher is surprisingly low: a one standard deviation increase in the first measure is associated with approximately one tenth of a standard deviation increase in the academic returns of the same teacher. When the analysis is replicated for low- and high-ability students separately, the positive association of academic and labor market returns to teaching is confirmed for high ability students but it turns negative for the low ability ones.31
Consistent with previous findings (Braga, Paccagnella & Pellizzari 2014, Carrell & West 2010), we also document a negative correlation between the academic returns computed on the basis of contemporaneous and subsequent achievement. This result is most likely driven by grade leniency induced by the system of teacher evaluations based on student opinions, a system that is common in most universities around the world and that was (and still is) applied also at Bocconi. Coherent with this interpretation, we also find that the contemporaneous effects are negatively correlated with the labor market effects.
In Table 8 we also estimate the cross-correlation between the academic and labor market returns for high- and low-ability students. In other words, we ask whether the professors who are the most effective for the good students are so also for the least able ones. As for the results of Table 7, in these regressions we condition on degree program, term and subject areas fixed effect, we weight observations by the inverse of the standard error of the estimated dependent variable and we bootstrap the covariance matrix of the estimates. Interestingly, we find a very low and insignificant correlation for academic returns and a positive and significant one for the labor market returns of professors.
The findings in Tables 7 and 8 can be rationalized under the view that the complementarity between teacher and student abilities is stronger in the production of academic achievement than earnings (Jackson 2012). A very good student learns easily and does not need to be explained things several times nor particularly clearly. Low ability students, on the other hand, really need a good teacher to understand the material. Hence, it is easier to raise both grades
30An increase of one standard deviation in our academic returns is associated to a 0.34 increase in the labor market returns (the reverse regression of the one reported in Table 7, column 1 in panel A) and a one standard deviation increase in labor market returns is associated to 0.055 standard deviation increase in taxable earnings (Table 5), hence:0.34×0.055 = 0.0187.
31Notice that there is no sense in which the estimates of the returns for the entire sample can be seen as averages of those for the low-ability and high-ability students.
and wages for the high ability students that the low ability ones.
These results suggest that teaching is better viewed as a multidimensional activity involving a variety of tasks each of which has potentially different returns on the academic and the labor market performance of the students. Hence, it is problematic to evaluate teachers on one single dimensions, as it is often done in practice. Table 9 further emphasizes this important point by showing the joint distribution (by quintiles) of the academic and labor market returns of pro-fessors. Despite the general positive correlation documented in Table 7, the two distributions overlap only very partially. Only abut one third of the professors are in the same quintile in both distributions and approximately 35% of teachers who are in the top 20% of the distribution of academic returns also appear in the top 20% of the distribution of market returns. A sizable fraction of them (approximately 13%) are in the bottom quintile. Consistent with our previous findings, the overlap is even less substantial if one compares the distribution of the contempo-raneous returns with either of the other two measures. In fact, the academic returns of teaching explain only about 16% of the variation in the labor market returns, once program, course and area fixed effects are partialled out. If one were to combine both the academic and the contem-poraneous effects to predict the labor market effects one would still be able to explain less than 17% of the (residual) variation.32