Client story Catholic Diocese of Maitland-Newcastle relies on data lake and AI to explore drivers of student learning
By Insight Editor / 11 Nov 2020 / Topics: Customer experience Data and AI Data and AI
Many of us recall that line on a school report that revealed to our parents that perhaps we had not been achieving all we were capable of.
It was based on the gut feel of the teacher at the time – but nevertheless almost always resulted in a parental lecture and the occasional reminder to ‘pull up your socks.’ Not very scientific, not very strategic.
But what if it was possible to use data to predict for a primary school child in Year 5 a defined range for possible achievement in Year 12 exams? How incredibly valuable this would be to motivate the child to aim for the top of the range.
What if gifted children were more easily identified and less at risk of falling through the cracks? What if data could identify a specific learning deficiency for a child, where this factor was the road block to further academic growth? A timely, targeted, evidence-based intervention could be life changing. Armed with such insight parents and teachers could focus their efforts more acutely, supporting all students to achieve much more of their potential.
The Catholic Diocese of Maitland-Newcastle is confident all of this is possible – and also believes data plays an important role in ensuring optimal outcomes across a slew of different areas. In response, it has undertaken a significant digital transformation with data at its core, as well as working toward user directed analytics to ensure evidence-based insight drives action.
The Catholic Diocese of Maitland-Newcastle, serves a community of 150,000 Catholics in the mid north of NSW providing pastoral support, health care, social welfare and educational services. Just shy of 20,000 students attend its 58 schools spanning Newcastle as well as the Lake Macquarie, Hunter, Upper Hunter and Manning areas.
Chief executive Sean Scanlon explains that the decision to set up a data and analytics capability for the Diocese was born of a desire to improve educational outcomes for its students. “It needed to be something where we had evidence and rational data, and analyse that data to understand how we were performing, particularly in the education space so that we could target each and every child’s education,” he says.
Data was always collected about student performance, but it was often siloed, difficult to access and scattered through an array of disparate source systems and spreadsheets. It provided an incomplete picture of student performance and progress and as Scanlon notes; “Without measuring those things, you couldn’t hold people accountable and we couldn’t see what we were achieving, and we couldn’t set targets for improvement.”
The same held true for the Diocese’s work in social services, supporting people in need or vulnerable. “How can we measure how we’ve improved their lives or done things that target the need, but also allow us to then go to the government and say, ‘Look, here’s the evidence. Here’s the data that we’ve analysed and looked at, that shows that what we’re doing works’.”
Working with Microsoft partner Insight and under the leadership of Zanné Van Wyk, Head of Data and Analytics for the Diocese, the organisation has deployed Azure Data Service foundations. Data from individual schools has been ingested and Power BI for self-service analytics has been rolled out, with access to data governed through Azure Active Directory.
Paul Greaves, assistant director of the Maitland-Newcastle Catholic Schools Office and project sponsor for the data and analytics student intelligence project says; “Maitland-Newcastle Catholic Schools system has set some specific targets around improving student academic achievement. The creation of a ‘Leading Learning Collaborative,’ which integrates the work of world-leading education researchers such as Lyn Sharratt and Michael Fullan, is the vehicle we have designed to carry this initiative and the fuel in the tank is data mixed with self-service analytics.
“The Student Intelligence Project is changing the way our system leaders, Principals and middle leaders interact with data and analytics as it addresses a number of pain-points we were previously experiencing. For example, ingesting and storing a multitude of data in a data lake from which we can draw specific dimensions for analysis means leaders no longer have to navigate a plethora of siloed data sources – each with its own usernames, passwords, formats, reports and often limited capacity to interact with data. Typically, if triangulation of data was the aim, it inevitably meant export into Excel and required time-intensive data manipulation and high-level technical skill.
“In Power BI we now have an environment where data sources are integrated and readily accessible. We have a consistent platform where educational decision makers can now focus on data as it relates to educational outcomes for students, rather than being limited by the aforementioned data processing tasks. “
Single source of truth
The benefits were expected immediately because everyone in the organisation would have access to a single source of truth, one comprehensive data source to support decision making and drive efficiency.
Care has also been taken to incorporate as much context as possible into the multi-dimensional data model that is being built in Azure. This will help identify problems, find solutions and stage appropriate interventions.
Van Wyk says; “We believe that we will be able to predict for a primary school child in year five, how they will do in HSC in year 12, using the datasets that we have available. If we have that prediction online and we can see that someone is below what we feel they can achieve, we can then already in primary school start those targeted interventions to ensure we prepare that child for HSC and improve what we at this point are predicting as maybe lower than the norm.
“Also, what we’re working now is in-depth analysis about gifted education children. We know who the gifted education children are in the schools.
“Now that we’ve got a broader context around other tests, we realise there’s children being missed as gifted, just because their marks don’t show it. The work that we’ve done just around having a better understanding of what defines a gifted child in the schools over the last six months was phenomenal, absolutely phenomenal.”
Greaves adds that;
“Our most impressive example we can cite thus far in our journey is that of our Gifted Education Leader at a system level. Weeks of downloading data from a variety of sources into spreadsheets, where further complicated matching and manipulation had to occur, would finally allow an educational lens to be applied to the data.”
The work of the Student Intelligence Project in creating the dimensional model now allows our Gifted Education Leader to readily surface the required data and engage in analytics which drive action in the form of targeted and appropriate interventions.
Van Wyk is expecting the data also to reveal some interesting patterns about the impact that remote learning has had on students as they navigate the COVID-19 pandemic. Similarly the data should help shed light on student progress despite the lack of NAPLAN testing in 2020.
Generally, NAPLAN tests provide a level-set for schools to see how their students are going, and whether they need extra help. This year no NAPLAN tests have been taken.
Van Wyk says however that the student data collected by the Diocese provides an alternative measure of student progress.
It means says Scanlon that: “You’ve got an opportunity to bring kids back on track if they’re falling behind or if there are other issues. We’ve got more dimensions to this than NAPLAN has targeted, the wellbeing aspects and students missing school could be a dimension.
“This allows us to have greater insight and therefore to act more quickly to deal with any issues that might arise or to look at what’s happening in the students’ life and drill down to the individual student rather than looking at a class or even a school and seeing how it’s performing. That’s the expectation,” he says
It will also help schools meet their obligation under the memorandum of understanding with the State Government that children achieve appropriate growth in their education. “That’s the requirement, so we need to be able to measure and check that, and see that we’re on track,” adds Scanlon.
Schools are also required to report on student attendance – not an easy task when some children are in the classroom and others are remote learning. Van Wyk says that by pulling the Diocese’s Office 365 data, including Teams use, into its data lake it was possible to see easily which children were attending lessons – even remotely.
Predictive analytics
The Diocese has developed a three-year strategy for its data and analytics transformation and is now at the mid-way mark.
Its goal is not to deliver predesigned dashboards and reports – rather to offer self service analytics and predictive analytics to ensure maximum impact,
“That’s why we have a data scientist as part of the team,” says Van Wyk. While she herself developed the strategy, she engaged Microsoft partner Insight to validate it, and help develop and deploy the Azure based solution.
That solution leverages Azure Data Factory, Azure Databricks, Keyvault, Storage account – Data Lake, Azure SQL DB, Azure Analytics Services, Logic Apps, Azure Data Catalogs, DevOps – Repos and Pipelines. The governance surrounding the system is critical with access to often sensitive data managed through Active Directory.
Working with Insight the Diocese is now refining its understanding of business processes across the organisation and building a multidimensional data model. From that new key performance indicators are emerging that will be used to steer the business.
Van Wyk’s goal however is to make more use of predictive analytics. “Previously, for example, finance will come in and they will say, ‘Let’s put 10 per cent on this year’s budget for next year.’ We believe we can use predictive analytics to either seek targets, benchmarks, or baselines, for example, for KPIs or budgets and that can be productionised back into your dimensional model.”
The predictive analytics approach can also be applied to learning. According to Khaled Auf, Data & AI Practice Lead, Insight; “They are extracting data collected across the learning lifecycle and applying it into the development of predictive models for targeted ‘interventions’ in time to make a difference, not months downstream of an issue.
“With Azure, they are looking to have a deep understanding of business problems; build trust of decision makers; effectively leverage data and analytics – explaining results in simple, powerful ways; and establishing a data and analytics governance framework"
The partnership with Insight has been exceptional, says Van Wyk, resulting in a solution that aligns with her goals, and came in on time, and in budget.
She says; “It’s about time, cost, and value. They got it. Within a week, we had an architecture. The consultants that they gave us were incredibly knowledgeable and they had a very practical plan on how to implement this, very organised and structured. We delivered this within the timeframe that we discussed, within the budget that we discussed and the business value that we discussed. “
Greaves says that; “Having education leaders spending a majority of their time collecting and manipulating data is never a good investment. We have recently developed a ‘Data Framework’ for our system where the application of ‘Data-Cycles’ in Professional Learning Teams (PLTs) is at the heart of activity to improve student academic outcomes. The Student Intelligence Project is critical in creating the conditions where integrated data is accessible in a platform which allows us to ask questions of the data in a timely manner and thus allows our investment in educational leaders to be better realised.
“Our pressing leadership challenge at a system level is how do we now engender shared beliefs and understandings amongst our system and school leaders around data and the Power BI platform? Recently we have delivered Data Literacy Professional Development workshops to over 150 system leaders, Principals, Assistant Principals and data leaders in schools where our focus was to simply introduce them to the Power BI tool. The fact that so many volunteered to participate in the workshops indicates a clear desire to become more data literate. Our approach has certainly piqued interest and is driving a desire for many to further explore possibilities.
“Our approach in the workshops in the future will continue to be the presentation of actual examples of ‘Data-Cycles’ at work where student outcomes have been impacted in a positive manner. This is what will get our leaders onboard, rather than technical descriptions of how to use the software. If leaders think it matters, they will cause themselves to become familiar with these more technical aspects. Bringing about intrinsic self-actualisation in lour leaders should be our own leadership focus.”
COVID complication
One complication that could not have been foreseen was the COVID-19 pandemic which forced a lot of the to-ing and fro-ing between Van Wyk’s team and Insight to take place online rather than face-to-face.
At the same time she was also building the Diocese’s new data and analytics team. “This is a team sport. You can’t deliver it as an individual. You need to brainstorm and work things out and have a few fights because everyone has strong personalities. They’ve got their own experience. The data scientist is from Chile. I’m from South Africa. The BI specialist is from Venezuela and the data engineer from India.”
She’s also tapped expertise from across the diocese to inform the analytics models including from a 90-year-old nun and an 80-year-old professor.
It may appear to be a Tower of Babel – but the language everyone on the team shared was data and analytics along with a common mission to deliver insight and value for the Diocese.
More broadly as a result of COVID the organisation itself had to rethink the way it operated says Scanlon, and has embraced Office 365 and Teams to support remote working with no sacrifice of communications or connection. He acknowledges that ordinarily it might have taken 18 months to deploy the new digital way of working; the restrictions of the pandemic meant it was achieved much faster.
“We were forced to use a lot of this technology. People now, I think find it second nature and that creates a lot of flexibility around how people might work. I think that’s going to be one of the most important things in certainly our administrative services.”
The Diocese is also using its remote learning experience as the platform to rethink how it can support students who might in the future be off sick or away for other reasons.
And, thanks to the data and analytics platform – it will be clear how those student’s learning has been impacted and whether further action is required to ensure they always achieve their full potential.
