Making data work: Five challenges leaders face when building a data-driven-culture
In this series on Making Data Work, I share key insights and tools needed to develop and advance a data-driven mindset and culture of collaborative inquiry within teams, faculties, and schools.
I’ve helped many organisations grow towards more responsive ways of working through cultural, structural, thinking, and process evolutions that have developed greater organisational flexibility and learning capacity. My motivation for writing this series is to help expand the ability of school leaders to rapidly adopt a data-driven mindset in this increasingly technologically complex world we’re having to educate the next generation in.
‘Data-informed practice. Evidence-based learning. Personalised Instruction.’
With the recent advent of teaching roles solely dedicated to ‘data analysis and visualisation’ within schools, ‘data’ might just be the biggest trend in education right now. You’d be hard-pressed to find an educator who doesn’t understand the importance of using data to inform practice, especially with all the hype going on in the industry right now.
What hasn’t been made so clear, however, is how the act of analysing student data in itself only plays a small role in actually using the data to improve student outcomes.
The real challenge arises when leaders attempt to build in the data-driven-mindset into their teams. It’s in the creation of psychologically safe environments where unbiased and collaborative inquiry can take place, that teachers can connect the dots about their students they otherwise wouldn’t normally.
They’ll contribute their own insights.
Connect to the faces behind the numbers.
Centre learning around the needs of their students.
Allow me to start off with the exploration of five key challenges we’ve observed school’s face when embarking on a journey with data.
There are way more than five, but I’ve picked the most important ones that you need to know. As we progress through this series, I will expand on how we’ve addressed these challenges and what you can do if you want to embed the data-driven-mindset into your teams
#1. The Five Dysfunctions of a Team
Collecting and using evidence of learning to target and differentiate teaching is an incredibly technical and time-consuming endeavor. Very challenging for a teacher to do alone.
Working together in Professional Learning Communities/Teams not only saves time and effort, but also allows teachers to draw on a broader range of experience and expertise, and to test interpretations and approaches with each other.
It allows for a teacher’s practice to evolve from it being about ‘my kids’ and more about ‘our kids.
However, this kind of collaboration can be very confronting for many teachers. Being transparent about their students’ performance means having to be open to the possibility of having their teaching practice scrutinised.
It means team members have to be vulnerable.
Patrick Lencioni’s: ‘5 Dysfunctions of a Team’ gives leaders a model for how to create safe environments that enable effective collaboration of their teaching teams.
Let’s address the dysfunctions together from the bottom up:
A) Vulnerability allows for trust to develop
Trust that they won’t be judged the next time they express thoughts and ideas.
B) Trust removes the fear of conflict
Conflict, a healthy debate about choosing what is right for their learners will no longer feel like an attack.
C) Conflict = Contribution = Commitment
Commitment arises when all team members have had the opportunity to debate opinions and share ideas before agreeing as a collective on the direction the team should take.
D) If we’re all committed, we’re all accountable.
Accountability won’t just happen at the individual level. With the presence of vulnerability, trust, and ability to have conflict, team members will hold each other accountable.
E) Our students. Our results.
The inevitability of whatever happening in the staffroom eventually finding its way into the classroom.
If collective efficacy is the number one driver for student outcomes as stated in John Hattie’s 252 Influences on Student Acievement, how can collective efficacy be fostered within a team without first addressing their ‘5 Dysfunctions’?
#2. Time: I want to but I just don’t have the….
“We know that this doesn’t work and that doesn’t work, but we don’t have time to create a better process. I’d find myself in classrooms explicit teaching, going to myself, “I know this doesn’t work,” but I’m just stuck here doing the same thing.”
Emma Mcintyre, Enrichment Leader at Hoppers Crossing.
It breaks my heart to know that she’s not the only one who’s experiencing this.
It’s no wonder why a fifth of teachers leave the profession in the first two years, with another 8% of teachers leaving the profession annually.
As the scope of teachers’ responsibilities continues to expand, encompassing complex and technical tasks that the profession has never had to do before – data analysis being one – leaders must find a way to simplify complexity for their teams, enabling them to do more with less.
I’d like to leave point 2 open-ended and pose this question instead.
How might we, as leaders:
#3. Low Data Literacy: But can you blame them?
Here is a breakdown of the four of the most common tools you’ll find in any school. We’ve found that schools have at least three out of four of these tools running in parallel. And this is just accounting for Maths.
What do you see when you skim through this table?
|NAPLAN||– Group Summary report
– 5-year trend Growth report
– Relative growth report
– Item level analysis report
– Student response
– School comparison
– Student profiles
|– Bands range from 1 – 10
– Scale scores range (-128 to 1000)
– ‘Top 2 bands’
– ‘Bottom 2 bands’
– ‘High to medium growth’
– Skill assessed: not linked to any skill in VCAA & ACARA Curriculum databases
|ACER – PAT||– Test usage report
– Bands Report
– Norm Reference Report
– Group report
|– Bands: 64 and below, 65 – 64, 75 – 84, 85 – 94…..
– Scale Scores: (0 – 155 and above)
– Norms: gives the margin of error for each scale score
– Percentile Rank: a simple means of indicating rank order and position of students result in relation to a norm reference sample
– Difficulty score: the level of difficulty of the test relative to the level of ability of students
– Stanines: Divides the total student distribution of abilities into 9 categories with 1 being the lowest, 5 the midpoint, and 9 the highest.
|Essential Assessment||– School Overview
– Class Overview
|– ASR = Achievement Standard Results
– ZPD = Zone of proximal development
– Average result
– Expected result
– Pre/Post result
|Math Pathways||– What students are working on
– Student module grid
– Student levels
– History of activity
– Test results
– Projected Pathways
– By key concept
– By Module
– By growth rate
– By levels
– Goal Rating
Inconsistent. Complex. Confusing. Time-consuming.
Even the most well-intentioned teacher who wants to analyze the data to prepare for his/her classes would be hard-pressed to understand the story behind each report and derive common meanings from each of the different measures.
Surely by now, there’s a tool out there that saves teachers hours of manual labor by simplifies the process from collection, to analysis, through to targeted insights and recommendations in a simple and easy to understand way?
Something that doesn’t detract from a teachers practice, but rather enhances and enriches what she already does?
Glad you asked. In the next chapter, I’ll expand on how Edapt’s Learning Analytics Platform is saving teachers hours of time and effort, and how you can use it in your inquiry cycles.
#4. Change: Why should I?
Introducing a new improvement cycle, teaching strategy or analytics tool means change. Mandating that it be used won’t mean it will be adopted. Or at least in the way leadership envisages it to be.
Changing the way someone does something is incredibly complicated. Especially if that’s the way they’ve been doing it for years.
Doing so for a collection of individuals, each with their own set of nuanced idiosyncrasies, beliefs, fears, and desires is exponentially more complex.
This is a universal challenge that leaders from any industry face, whether it be Accounting, Finance, Technology, or Education.
University taught us how to become specialists in our craft. It was through the merit of our work that resulted in our promotions to exciting leadership positions. Unfortunately, neither school nor university prepared us for the day where we’d have to lead teams and manage change.
This is also why consultants exist in all industries. We work with leaders in developing an objective view of their bank, hospital, school (aka system) and then manage the people side of change to reach the desired outcomes.
The answer lies in the ability to first see the ’system’ that we’re working within and its interconnected parts, be that its people, processes, practices, structures, roles, responsibilities, documents.
Then working backward from the desired outcome, developing a strategy that both maximises the adoption (of the tools, process, practice) and causes the least amount of friction possible.
#5. Mindset: Separating the doing from the being
So what does it mean for teams to truly embody a data-driven mindset?
Where teams no longer just go through the motions of checking off a series of sequential tasks from FISO’s Improvement Cycle because that’s what they’ve been told is ‘right.’
A data-driven-mindset starts when one is able to separate the doing from the being.
Tools serve to extend the ability of a teacher in a given context.
Practices are applications of specific strategies to deal with specific situations that one might find themselves in.
Frameworks serve as conceptual structures that act as guides for how to create better student outcomes.
Tools, practices, and frameworks are useful for improving what teams do.
This is not, however, what will shift culture.
Uniting teachers under a common purpose, vision, set of values and principals will ensure what they do is governed by why they do it.
That is what will shift a culture.
This introduction to the five challenges are key roadblocks to look out for when building a data-driven culture centered around collaborative decision making and will enhance your ability to think divergently and creatively in your pursuit of better student outcomes.
I’m not going to lie, change is hard. Integrating evidence into teaching in every classroom requires action across the entire school.
Improving overall school performance can take six to seven years.
Implementation requires the right tools, team maturity models, and change processes. Edapt uniquely combines a cutting edge learning analytics platform with a framework for change which is already making an impact at several schools.
You can find out more about it here
In the next chapter, I will go into more detail about our learning analytics tool, the model for school improvement born out of modern management consulting practices, and how schools are using Edapt to enhance student growth.