Spreading the word

We’ve had the opportunity of presenting our work on Learning Analytics visualisations and dashboards at quite some events lately. Here’s a quick rundown.

In June we visited Harvard to attend the Learning Analytics Summer Institute. During the first day, al phd candidates could give a 1 minute quick intro of their work. My 1 minute of fame, together with the poster session, caught the attention of quite some people. Among them, Zachary Pardos, a professor at Berkeley, who is working with MOOC data and was interested in combining our visualisations with their data. More on that soon!

We also gave a half-day workshop on Visual Learning Analytics, explaining both the basics and pitfalls of data visualisation techniques, applied to Learning Analytics. Quite a nice turn-up, and a very active, enthusiastic group.

LASI workshop

In August we presented our teacher dashboard at OBIE2014 at ICWL2014. Sadly I couldn’t go to the event, so Jose volunteered to present our work. More details on his experiences on his blog.

September was Graz, Austria. At the weSPOT consortium meeting we discussed the progress of the European project. Lots of interesting discussions about our current dashboards, and a date for at least one really interesting evaluation setting (our tools are evaluated in different pilots, but when we get to do them ourselves we’re more excited ;)). That same week was EC-TEL2014, where we demoed some of our more recent work at the ARTEL2014 workshop.

LARAe.chi14

These events resulted in lots of interesting discussions: how does one evaluate these visualizations, what do we want to evaluate (we’re computer scientists aka not pedagogical folk), how do we make visualisations more accessible for teachers and students in schools,… Lots of questions, few answers. At least everyone agreed.. it’s not that simple.

Anyway, that leaves room for lots of research. So at least I know what I’ll be doing these next few years … ;)

Slides/papers/etc of these events:

2nd Learning Analytics Summer Institute (LASI2014)
Visual Learning Analytics workshop
slides part 1 / part 2

1st International Workshop on Open Badges in Education (OBIE2014) @ 13th International Conference on Web-based Learning (ICWL2014)
slides / paper

4th Workshop on Awareness and Reflection in Technology-Enhanced Learning (ARTEL2014) @ 9th European Conference on Technology-Enhanced Learning (ECTEL2014)
slides / paper

Engaging students in Learning Analytics

This article was originally posted on Medium.

LARAe.chi14

Learning Dashboards, which visualize Learning Traces left behind by students, have many applications, e.g. they help teachers keep track of their students (with all privacy concerns that follow) and they assist students in keeping track of their progress.

Teachers love these dashboards. The more data available, the better it can help them steer a course, find struggling students and assist in grading. Students, however, tend to experience elaborate dashboards as a Big Brother tool straight out of 1984 (even though none of them probably read the book or saw the movie).

But it is important to get students aboard. The Quantified Self (QS) movement has shown that tracking and being aware of your own data can help change and improve your ways, and learning should be no different.

But students are busy. They have no time for this QS nonsense. They have exams, tasks, projects, … they are pretty much the “busiest” people you will ever meet. So introducing “one more thing” that, to them, has no direct impact on their ultimate goal, a good grade? Probably not going to be a success.

I believe one way of solving this problem is by taking a little detour. Figure out what it is students want, what is lacking in their workflow and how we can help. And if we find a problem we can solve, we might as well do it in a Learning Dashboards kind of way. (Of course, if the problem cannot be solved by putting Learning Traces to a good use, find another problem)

Meet LARAe (Learning Analytics Awareness & Reflection environment), a dashboard created specifically for teachers and students of our User Interface course of 2014. From our previous work and by talking to both teachers and students, we pinpointed a bunch of issues in their workflow and created a dashboard that attempts to do just that: improve their workflow.

In this course, students generate lots of data in the form of blog posts, comments and tweets. 13 groups of 3 students, blogging 3-5 times a week with each group commenting on posts of the others, difficult for both teacher and student to keep track of. Many use RSS readers, some manually explore the blogs on a weekly basis. A workable process, but not ideal for everyone.

So LARAe tries to make this workflow a bit more straightforward. When it does, teachers start using it, and so do students. And suddenly everyone is looking at Learning Analytics data…

Being regularly confronted with these visualizations of Learning Traces, students might start visiting the dashboard for reasons other than their original intent. They become curious and start to explore, gain awareness of their activities and that of others. And maybe they learn something along the way, improve their learning process and who knows, get that better grade.

Learning Dashboards have many applications. Taking them beyond mere visualizations of numbers and statistics will not only help broaden their range of applications, but also get them in the hands of more people. Like students. And in the end, it is all about the students.

Privacy Concerns with Learning Dashboards

demoday_larae3

Let me first get the name LARA.e out of the way. As we keep developing new learning dashboards, we constantly need to create new names which can get confusing very quickly. Therefore we rebranded the entire thing as LARA.e, which stands for Learning Analytics Reflection & Awareness environments. We’ve mentioned Navi Badgeboard (LARA.e1), a dashboard helping students become more aware of their activities and achievements in class and Navi Surface (LARA.e2) which creates a discussion environment by playing the role of a catalyst for reflection. The latest dashboard to join the family is called just plain LARA.e3.

LARA.e3

LARA.e3 (see screenshot above) focusses on the teachers and provides them with a detailed overview of all activity traces of the class. It displays a massive amount of information regarding (in our case) blog posts, comments, tweets and badges, but can be modified to visualise any type of learning traces through multiple visualisations. Global filtering options allow the teacher to drill down on time frames, students and group. Students and groups can be easily compared and with the addition of course grades (left out of the screenshots for privacy reasons), student activity can be compared to their actual results in class. LARA.e3 can be displayed on large displays, smart boards, digital tabletops and more, allowing the teachers to consult the data individually, with colleagues as a base of discussion or in class with the students.

LARA.e3 is all about awareness and reflection regarding the teacher: awareness about activities across the entire class can help teachers intervene regarding group or individual student issues. Reflection about the course structure and content helps teachers get a deeper understanding which in turn helps find and address issues with the course.

Any student reading this blog post will most likely cry Big Brother, something I’ve tried to address in the previous post. We do have access to all their data and with LARA.e3, we do see everything. And while we can tell them it’s all about self-awareness and this doesn’t affect their grades, it’s impossible to really convince them not to worry or be wary of our motives.

It’s a trust issue, present in student-teacher relationships just as it is in employee-employer relationships (We’ve all heard the company badge/fire security excuse before right?). The only real solution I see is anonymization.

To convince students of their anonymity, they need full control over their identity. The teacher nor other students should be able to figure out the true identity of the users displayed on the dashboards by any means. Only then might we create a real trust and convince them of a true Quantified Self learning environment, where the focus is self-awareness without the underlying fear of being watched.

Anonymous data can still provide us with interesting insights. While LARA.e3 will not help find the individual straggler, intervention can still be possible at a class level. But every other benefit of the dashboard remains.  Personal student dashboards (e.g. LARA.e1) do not lose any of their functionality. Another plus can even be better data as less students might attempt to game the system.

To end this post, I’d like to ask if any teachers reading this have any interesting stories of students’ reactions to the introduction of learning analytics in their courses and how they handle this trust issue in case there was any. To the students, feel free to share your thoughts on anonymous dashboards (or the not so anonymous LARA.e3).

Learning Dashboards

1984

This post is written for our thesis students. But do feel free to join in on the conversation!

As you may or may not know, learning analytics and learning dashboards, a Quantified-Self take on learning, is an important part of our research here at the HCI research group. Most of you have taken Prof. Duval‘s classes and have had to blog, tweet, read and comment on each others’ work. For your thesis, we’re asking you again to blog and track time with Toggl. But the reasoning behind all this might not be that clear to all of you.

First of all, these activities force you to stand still and think about what you are doing. This is certainly the case for blogs, where you have to do proper research and think before you start writing anything down. This alone will make you understand the subject matter better and already puts you on the path to becoming a better learner.

But what about Toggl? And Twitter? What uses have they except for tracking your every move? Are we really watching you? Is HCI Big Brother?

The data we track has one main goal. To create learning dashboards to improve your learning process. This way we can feed the data back to you to provide you ways of finding patterns in your habits and change your ways to become a better learner.

Let’s explain this with two examples:

You go out drinking quite late. You have an early class next morning. The professor isn’t motivating you much with that book he’s reading from. You have a heavy, greasy lunch. Your afternoon is free so you decide to work on your thesis. Hours go by but you eventually feel like you haven’t accomplished anything.

Quite a depressing example. Here’s another:

You’ve had a great night sleep. You have an early class and it’s super interesting! The professor’s a genius! Your afternoon is free, the weather is awful so you decide to head to the library. You work a few hours on your thesis and feel super productive.

While in these extreme circumstances it’s obvious what affects your motivation and productivity, there are many more factors that influence your learning.  And with dashboards, we can present you that information through interesting visualizations that allow you to find the pieces of data relevant to your learning, so you can figure out what parameters influence you positively, or negatively. You can figure out what works for you, and what doesn’t!

We’re not just talking general guidelines. Too much alcohol will make most people unproductive. But noise? Music? Weather? These things can have a different effects on people.  And there’s a good chance there are factors you never even considered!

To keep the overhead low, we decided we can use Toggl once more. We’d like to define a bunch of categories which would help you log certain personal and environmental information, like mood (happy, sad,…), physical state (hungover, active, awake,…), location (library, home, parents, friends,..), noise (roommates, music, birds), how your time was spent (productive, waste of time,…)… The list hasn’t been set in stone, but you get the idea.

And before we start, we’d like your feedback on this matter. What would be interesting data you wished you’d have a better view on? What do you think affects your motivation and productivity? And what are your general thoughts on learning dashboards anyway?