18 October 2016

“You measure what you treasure”: key lessons from Australia’s My School

by Dondon Parafina

The 3-member Philippine delegation (Ana, Don, Deo) to the study visit in Australia organized by UNESCO, hosted by the Australian Curriculum, Assessment and Reporting Authority (ACARA).
The 3-member Philippine delegation (Ana, Don, Deo)
"You measure what you treasure,” says a co-delegate in the study visit to Australia sponsored by UNESCO and hosted by Australia’s Curriculum, Assessment and Reporting Authority (ACARA).

When it comes to public access to data, you should ask for what matters. No just any data, but something that impacts people’s lives.

In the education sector, it’s the learners who matter the most: whether they are learning or not; whether they are being equipped with the right facilities to learn or not.

In this sense, Australia's open data for education platform, MySchool.edu.au, got it so right. "My School”, which has information about over 10,000 schools in Australia, features students’ growth in reading, writing, language conventions and numeracy based on the result of the National Assessment Program - learning and numeracy (NAPLAN). It’s a nationally administered test for all students in years 3, 5, 7 and 9. The results are out there for everyone to see. It has students' test performance results, but they are de-identified to protect the students’ privacy.

And there’s more.

"My School" allows statistical comparison of schools’ and students’ performance according to context-sensitive index of community socio-educational advantage (ICSEA). The index includes parents’ occupation, parents’ educational background, geographical location, and proportion of indigenous students. These are deemed crucial drivers of students’ advantage or disadvantage in learning.

ICSEA is important because it makes the comparison fair and reasonable. When the socio-educational background is similar across schools, it is more likely that the
difference in students’ progress is influenced by the school’s teaching capabilities. Then the principals or teachers will be able to make the right interventions to help low-performing students or identify ways to enhance the top students’ performance even more.

The capability for comparison is one of the strengths of My School, but it’s also the most prone to controversy. Some principals and teachers do not like it, and a teachers’ union even threatened to boycott NAPLAN if My School will continue with the online publication of results. Parents, on the other hand, support and demand for it. Through it, they can make informed choice on the school they should enroll their children in or what kind of improvements they should demand from teachers and school officials.

In other countries, like the Philippines*, the usual education data collected and analysed pertain to resources available for the schools to operate. Classrooms, chairs and tables, textbooks, toilets, computers, teachers and personnel, budget. We compute textbook to student ratio, classroom size, toilet to student ratio, budget per student, and the likes. These are all inputs to learning, which often become the focus of attention or the main points of contention. Whatever we do, they won’t add up to learning and learning growth.

If there’s anything that we should look for when it comes to education data, it should be the measures of learning. Like what My School has.

That, however, is just data access. It’s half of the picture because the other half is accountability. It’s all about what the school principal or the teacher or the parent should do after getting hold of the data, especially the learning performance of the children. How should they use the data to address, say, the condition of the non-performing student?

What should the Principal do, as manager, to improve the learning condition in the school if certain gaps were observed based on the data?

These accountability questions are not simple. The reality of managing schools and student learning is much more complex than what the test results show. The accountability for continuous improvement is a function of many things, including the resources available to the principal and the cooperation of parents and community in school affairs. This broader understanding of accountability can perhaps avoid a situation wherein parents impulsively pull out their kids from schools in reaction to My School data.

It appears to me, at first glance, that My School tends to lodge all accountability on the use of data on the principal and the school level actors. But that may not be correct. We need to know what problems are properly within the mandate and capacity of the school principal to address. The others may have to be elevated to a higher office, perhaps even state policy level.

The point is: if the use of data for accountability is to be taken seriously, we need to make a fuller appreciation of the problems encountered in the schools, as may be revealed in and through the data; and match it with an enabling knowledge of the appropriate way of addressing it or who should be responsible for it.

I hope My School does this as it evolves as a model transparency and accountability mechanism in the education sector. To do so, it has to study further why principals and teachers generally dislike it and find it demoralizing when compared with other schools. On the other hand, the parents need to tell their stories of how My School has helped them participate more actively in schools affairs. These may convince the teachers and principals that open access to data is not out to harm them, but can actually improve schools and students.

*The Philippine delegation to the study visit to Australia shared about their own local open data and citizen participation practices in the education sector. These were the Department of Education’s (DepEd) Enhanced Basic Education Information System (E-BEIS) and Learning Information System (LIS) and ANSA-EAP’s CheckMySchool.

We agreed that there are significant lessons to learn from Australia’s My School.

First, DepEd’s Enhanced Basic Education Information System or E-BEIS, as a repository of various data on education services, is not yet widely shared with the public. Only DepEd offices and units, from central to division to schools, have access to BEIS. DepEd’s website contains only summaries of these data. To be truly relevant, there must be a clear demand for the data that DepEd collects. That will happen if the department will open up the available data for optimized public use.

Second, its data are heavily concerned with input services for learning. E-BEIS has information on enrolment, teaching and non-teaching personnel, classrooms, desks and chairs, toilets, computers, and even land titling, though it has not yet integrated information on textbooks and school finances.

More importantly, E-BEIS is not linked yet with the students’ learning performance. National Achievement Test (NAT) results are not publicly available and accessible. If the E-BEIS is to be positioned as DepEd’s main open data mechanism, it would be best to present its relevance as learning-centered or performance-oriented. Its relevance should consist in providing the opportunity to analyze learning and performance side by side with the input services.

Third, the E-BEIS may also be bolstered if it incorporates school based accountability management. Schools prepare important documents, such as the three-year School Improvement Plan (SIP) and Annual Investment Plan (AIP), which should reflect the schools’ priority improvement and serve as bases for the use of their resources. DepEd must see to it that these are complied with and that the public likewise gets access to these. An online platform to contain the SIPs and AIPs will help a lot in managing these in data format and promote public access for shared use with the communities.

Finally, the CheckMySchool’s service improvement framework may also be linked with the school’s learning performance. Its call for accountability has to include the concern for the quality of learners the schools are producing. It may not be clear yet how we can do it, but the lesson from Australia seems to intuitively tell us that it has to be there or that’s the way it should be.

Learning is our treasure and that’s what we should measure. We access such data to know if we're succeeding in our mission to educate. In another play of words, we say: successful access is accessible success for our precious learners.

Last edited: 21-10-2016
Mr. Redempto "Dondon" Parafina is the Executive Director of ANSA-EAP.

No comments:

Post a Comment