PISA Results Linked to More than Education System

HSE’s Institute of Education hosted a visit by University of Arizona Professor Emeritus David Berliner. In an interview with the Institute, Professor Berliner discussed problems of schooling in the U.S. and Russia, possible ways of evaluating the work of instructors, and also how the results of international educational research should be factored into decision-making.

The main problem is not the instructor, but the family

Most countries incorrectly interpret the results of the international educational studies PISA, TIMSS, and PIRLS, Professor Berliner believes. Overall, American schoolchildren demonstrate very low results on the Programme for International Student Assessment (PISA), which constantly pushes politicians to discuss the low quality of education, the substandard work of teachers and principals, the low quality of academic programmes, etc.

Analysis of PISA results show, however, that while most American schoolchildren really do perform poorly on the PISA, around 20% have very high results, and these are associated exclusively with socio-demographic factors. Children from middle-class families in the U.S. generally have very high results on international and national educational studies, while the scores of children from poorer families are rather abysmal. In addition, children from middle-class families are highly motivated and have all the conditions at home to do well in school. Around 24% of schoolchildren are below the poverty line in the U.S.; that is, a very substantial amount. On the other hand, in Finland, where students perform well on the PISA, only 4% of schoolchildren are below the poverty line.

 But let us assume that in the U.S., there are simply not as many jobs that must be filled by people with high results in the PISA’s key components. How many of such jobs are in the Russian economy?

 It appears that the results of international comparative studies are linked not only with the education system in a country, but also with the social situation as a whole. In the U.S. and Great Britain, stratification is very high and there are a fairly high number of poor people. This is why the results are average overall, while variance is extremely high. In countries of Northern Europe, there is less stratification and access to quality education is high, so overall results are high and variance is low. For that reason, when we talk about increasing the quality of education, it is necessary to think not only about the teacher, principals, and programmes of study, but also about the social policies of a country. For the U.S., the recipe for increasing PISA results lies in lowering poverty as a whole.

It is likely that in Russia the results of international comparative studies are also linked to social and economic traits of the country, and not just with education. This is why it is necessary to ask another important question: does a country’s economy currently need high PISA results? Finland is a small country; its entire population is significantly lower than even the number of American schoolchildren with the best results on the PISA. The Finnish economy could provide them all with jobs, it seems. But let us assume that in the U.S., there are simply not as many jobs that must be filled by people with high results in the PISA’s key components. How many of such jobs are in the Russian economy? This is why it is not necessary to chase after high PISA results like this; but this is exactly what is happening in both Russia and the U.S. now.

How to measure the quality of a teacher’s work

The government and business are inclined to believe that quantitative indicators can measure anything. In reality, this is not always possible, particularly when this concerns pedagogical performance, Professor Berliner believes. The idea exists in both the U.S. and Russia that the quality of a teacher’s work can be measured by academic performance or by a student’s subject knowledge. Actually, this is also not quite right.

Student performance and subject knowledge depend on an entire host of factors. Not least important are the socio-economic status and the cultural baggage of the parents. Children of wealthy and well-educated parents do better in school than children of poor and uneducated parents, and this is irrespective of location. A child's motivation to learn also depends on family. It turns out that in testing schoolchildren to measure the quality of teachers, we are actually assessing the family. And by using students’ academic performance to evaluate teachers’ pedagogical capabilities, we end up pushing teachers to give children undeservedly high marks.

It is now impossible to assess the quality of a teacher using quantitative indicators. Instead, Professor Berliner suggests using expert reviews. The experts would mostly include school principals. In both the U.S. and Russia, principals deal with a large amount of paperwork and bureaucratic tasks. This is wrong – principals must first and foremost assist teachers and assess their work. This can begin simply by attending classes regularly to see what a teacher does. It is also not a bad idea to video-record everything that happens in the classroom to see what goes on.

Aside from principals, outside experts must also assess teacher performance. They could consist of the most experienced and qualified practicing pedagogues. They should attend their colleagues’ classes and evaluate what happens there. Children’s subject knowledge does not always speak to a teacher’s abilities, and neither does the atmosphere in the classroom or the student-to-teacher ratio.

By visiting a colleague’s classroom, the expert teacher would also be able to help improve his or her work. For example, a useful practice is to conduct lessons at each other’s school. This helps better understand the challenges teachers face and also find ways to solve them.

According to Professor Berliner, the main problem for Russian teachers is the authoritarian and didactic style of work. This may be linked to the number of children in the classroom. In U.S. public schools, where there are around 30 children in a class, teachers behave much more didactically than in private schools, where class size is 10-15 children. A good approach can nevertheless be found for large classes. For example, teachers can be taught to break children up into smaller groups, each of which is given different tasks.

Ekaterina Rylko, for the HSE News Service