Quality Assurance, Driven by Data: 10 Steps To Open A School Series

It is important to evaluate a school’s performance on an ongoing basis, but we need to make sure it is done authentically using data driven decision making. With strategic and authentic support, organisations transform and improve when coached, mentored and led, focussed on key performance indicators. Ideally decisions should be data driven. Although there may be some areas of school performance easily monitored and measured eg. Financial Performance, there are a number of areas that are tougher to review, reveal and improve. There are also traps and misleading data that can confuse you.

Michael Fullan speaks about the need for meaningful systems of data collection in his book Putting Faces on the Data. We need to “build the bridge from data collection to improved instruction.” Students are people—not data. Assessment data can bury you or give you focused information on how to reach every student. The same applies for data that informs the business side of running schools. It is often more than what the data first presents to understand the full picture.

This article aims to unpack this often complicated topic.

What is Data and Data Driven Decision Making?

Of course data will include very real numbers. Financial reports being the obvious. But there are other forms of data that are less concrete but equally significant.

School Opinion data represented by parents, students and the community including how they might describe the school to others is invaluable. The concrete numbers are easy to find but it is the soft data that is also important. How people behave, how they feel, what is common practice. Perceptions, mood, school culture, values and beliefs in practice. What are the behaviours and what proof exists of the demonstrative actions by all team members. These things are important.

What data is collected? What data is acted upon? What data is shared? What data is analysed?

Financial Data

We have discussed financial data in previous articles and it is probably well covered. In particular a school’s financial position is a clear picture of the schools status at a moment in time. There are some traps and data can be skewed in areas such as EBIDTA but in most cases financial pictures are pretty clear. Valuation becomes subjective when we realise the data is only telling part of the story and does not reflect potential. What is most often not clear is what to do next. It is the other areas that will significantly impact the bottom line when we don’t understand the full picture and all that this encompasses.

When is Data Not Real data?

At first glance data often tells a story but without understanding the full picture that underpins that story you may not be so sure of the data’s validity. Take student achievement data. In 2019 a school may have excellent overall data represented by high averages for literacy. In 2020 that same average may drop significantly. Pressure mounts on school administration and teachers about why results deteriorated so quickly. What if that same school had a high turnover of enrolment? What if the data was not measuring the same students, year by year? This example is a common one in international schools with relatively high student turnover rates. We need to be clear what we are measuring and what the measurement represents. We need to look out for data that may be skewed. Data driven decision making needs to be accurate.

What do you do with the Data?

Ideally all data should serve a purpose. It should inform you about the past but more importantly it should inform new behaviours in the future. There is no point capturing data unless it will influence a change in practice. A big stick reporting approach does not change behaviour. A culture of self and peer reflection needs to be part of a cycle of review and change. We believe strongly in creating a community of peer learning and support. A model of organisational culture that values reflection, openness and judges data as helpful information to learn and grow from.

Data Driven Decision Making in Teaching and Learning

Schools are inundated with information that in this context we might call data. It becomes very difficult for teachers and leaders to know what data is useful, or how to use the data in ways that makes it useful. Teachers also get quite cynical if management collect data just for the purpose of it. It can create extra layers of work without any real benefit. Schools need to know how to select data that can be useful to improve student learning, and also make the data presentable and accessible for teachers to do the analysis of the data. Where possible schools should use already existing processes. The information is most often already freely available it just requires a system to capture it.

Demographic	Processes	Perceptions	Student learning
– Student background – Race/Gender/Sex – Gender – Student and teacher attendance – Staff background qualifications – Student behavior data (referrals, etc) – Physical environment	– Purpose, mission and vision – Leadership policies – Curriculum – Staff assignments – Professional learning, planning, collaboration – Program offerings and access to them – Instructional strategies and materials	– District expectations and evaluation procedures – Student attitudes – Teacher attitudes – Parent, community attitudes – School/district climate – Perceptions, preconceived notions/expectations – Learning style preferences	– Classroom summative assessment results – Formative strategies – Teacher evaluations – Benchmark (interim) test results – Standardized test results – Common district assessment results – Common school-wide assessment results

Digging Deeper

Usually data processes focus on generalities. “Averages” and “trends” are important but these stories don’t often tell us the underlying details. An “average” blends micro data together. What if we want to identify the effect of professional development strategies on improving critical thinking? We would need to investigate and capture data about student performance in areas that reflect “critical thinking.” Overall grades may generally reflect this, but not specifically. What if we want to analyse the number of students who moved from a ‘C’ to a ‘B’ in Mathematics over a set time period? Surely when we believe that all children are important, this data represents distance travelled for large groups of students. A’s are important and somewhat prestigious but some may argue that many bright children may do well almost in spite of the quality of teaching or programs. Our first layer of data may provoke us to look even further. It may tell us what we really need to look for.

Systems Approach to Gathering Data

Technology is obviously a very helpful tool to achieve this however the goal should be “for data to be touched just once.” What we mean is that if information comes into your workplace in some form then find a way to capture it just once then make it easily accessible. Let’s look at an “old school” example. A teacher teaches students on a daily basis including an observation of their learning progress. eg Questioning, observation of work completed. (Data point 1). Teacher tests students and their grade is marked on the paper (Data point 2). Teacher transfers that grade into a paper mark book (Data point 3). Teacher transfers that data into a central database (Data point 4) Teacher creates a report card (Data point 5). Teacher conducts a parent teacher interview (Data point 6).

You get the picture?

The teacher knows the students’ ability at data point 1. Of course we may need to validate the data, capture it in different ways and then present it in different formats however the above example shows how cumbersome the process can be . There are some short cuts. (A longer discussion for another time)

Another thing to consider is to shape processes in ways that allow you to monitor data in a systemic way. In another example let’s consider a request by management to monitor the degree to which all subject departments are focussed on literacy. We want to capture this information on a monthly basis, as our annual strategic plan says “The school will ensure all subject departments focus on and improve literacy within their subject contexts.” A simple way to do this is to create a standardised minutes template for all subject meetings that includes a section on literacy. If the minutes of every subject meeting are recorded in the same way then management has fast access to the reported discussion of literacy in every subject. Of course beyond that, we can review lesson/unit plans, conduct teacher observations, test the students, speak to students etc however we should try wherever possible to systemise the capturing of data to make the systems of capturing information smooth.

In schools with a big focus on data driven decision making they also create a data-friendly culture where teachers and administrators seek reliable data to inform their decisions about curriculum and instruction. Fostering such an atmosphere is a gradual process but much more effective than top down approaches.

Accreditaton

Most international accreditation systems depend on data. In truth the review teams are trying to quickly and succinctly review practice to ensure it is aligned to standards. Further more, they are looking for a demonstrated commitment to changing and improving practice, which includes data gathering processes. Our advice to schools who will one day seek accreditation is to implement systems from the beginning. When you design systems and procedures to embrace thes principles your job is half done. When minutes of meetings including systematic recording of conversations then a quick review shows an external team what you are focussed on. When data is captured and reviewed for internal purposes then it is easily shared with others. Data Driven decision making is an important part of accreditation processes.

Data Driven Decision Making

The real point here however is to make sure we make well informed and effective decisions. The challenge for those who collect data exploration team is to turn this momentum into action. The next steps in the process are to set concrete goals, develop strategies for achieving those goals, determine how to evaluate progress, and roll out the plan. Too often we make assumptions about small pieces of information and we don’t have the complete picture. Ideally data should drive great decisions and improved performance.