Innovative Teaching in the Time of COVID-19

By co-developing online data science tutorials and automated feedback mechanisms with former students, Mine Çetinkaya-Rundel from the Centre for Statistics has shown that first-year undergraduates can still receive high-quality learning experiences in the midst of a pandemic.

With many university staff across the UK experiencing stress and extreme workloads associated with moving teaching online, and a number of students complaining about the quality of their learning experiences during the pandemic, online teaching has become a subject of intense debate for everyone in university education.

A large part of what has made online teaching so challenging is the lack of warning the pandemic provided. Suddenly and urgently, teaching staff had to prepare and deliver classes online from home, introducing a host of practical and technical problems that many are still grappling with even now.

Opportunity for innovation

Mine Çetinkaya-Rundel from the Centre for Statistics has delivered Introduction to Data Science, an introductory level course on data science and statistical thinking from the School of Mathematics, since she joined the University in 2019.

Not only did Mine face the same hurdles everyone in university teaching has faced in delivering the course completely online, she also had to deliver it to up to 300 students – three times as many as the year before. “I was told early on that we needed to reduce the marking, as the core staff would not grow proportional to the number of students,” she recalls.

Mine immediately flipped this obstacle on its head, seeing it as “an opportunity for innovation”. She started thinking about how to deliver the same amount of exercises and feedback to students without adding to the burden on human resources, and soon realised that the key was to introduce as much automated feedback on student exercises as possible.

Student call to arms

As an award-winning specialist in undergraduate and graduate statistics teaching, and curriculum development, Mine was well-qualified to solve the problem. She is a partner on OpenIntro, a non-profit company that delivers educational products that are free, transparent and lower barriers to education. At OpenIntro, she has co-authored OpenIntro Statistics textbooks and the statistics curriculum’s associated R package, openintro. Mine is the also the creator and maintainer of Data Science in a Box, another free open-source resource, this time containing interactive tutorials and course materials for teaching or learning introductory data science.

With this wealth of experience, Mine could have transferred the course online and introduced automated feedback alone. However, she was keen to ensure the materials met new students’ expectations, and saw an opportunity to engage with existing students who might be under lockdown at home twiddling their thumbs. “Who better to work on the course materials than people who have seen and learned from them?,” she asks.

As a result, she sent out an email to a selection of 2019’s cohort asking if they would be interested in updating the 2020 course materials with her as a summer project. Lee Suddaby and Zeno Kujawa answered her call.

Co-development collaboration

To begin, Mine reviewed the existing assignments to see which components could be delivered to students in a format where they receive automated feedback on their code and intuition, and which still needed human assessment. Next, Lee and Zeno converted the homework assignments into auto-graded interactive tutorials. To do so, they first familiarised themselves with the R package learnr and its companion gradethis – designed to help teachers engineer interactive tutorials and exercises, and be able to automatically grade student answers.

Requiring the undergraduate pair to figure out the best way to phrase a question and provide feedback for each exercise, they then wrote the tutorials separately in parallel, before reviewing the other person’s work, iterating again, and finally handing over the tutorials to Mine for feedback and sign off. “The structure meant better quality work to review for me, and I think they learned a lot from each other too,” says Mine.

The result of all this summer labour is a 50% reduction in human marking of homework assignments. “But for me, this was more about making sure the content exposure isn't reduced with everything being remote,” says Mine. “Ultimately, what I've observed so far is that students are working through these exercises and receiving feedback straight away, meaning we are creating opportunities for them to practice, to learn.”

Seeing opportunities for further improvements to the course, Mine is now working with other students this semester to add more tutorials that focus on statistical modelling topics in order to fully cover the course curriculum. She is also exploring ways to incorporate automated data-logging into optional exercises, which has proved challenging up to now. “We want to gain some insight into how students are doing and which topics appear challenging,” she says. “That way, we can say ‘X percent of you did the exercises, and a large chunk of you did well in Y but not so well in Z and here's why’”.

Thinking more broadly, Mine is keen for other university educators to replicate elements of the course itself: “These tutorials could be used in any course covering similar material to mine,” she says. Towards this aim, the tutorials are now being distributed with the dsbox package, the companion R package to Mine’s Data Science in a Box resource. Moreover, Mine views student curriculum co-development as a potential blueprint for others developing courses going forward, with advantages for both students and educators.

For Lee and Zeno, the opportunity to apply and extend what they had learned in their class about data science workflows involving R and version control has been invaluable. Not only did they have fun learning about the tooling that an industry or academic data science team uses, the project also provided in-depth experience and knowledge relevant to future careers in data science or data science education.

And for Mine, involving students has enriched her course. “As I'm moving further and further away from their age group, it’s getting harder and harder to for me to tell what might be interesting to students, but Lee and Zeno know,” she says. “We actually swapped out one of the datasets I had with a more local Edinburgh-focused one because they thought that would be more interesting to the students taking the course.”

“It takes motivation, it takes dedication and it takes skill, but I think ultimately co-developing the curriculum with students is worth it for everyone involved.”

Edinburgh alumnus Benjamin Skuse is a freelance science writer based in Somerset, UK. https://benjaminskuse.wordpress.com