LISSA: a student-facing Learning Analytics dashboard based on readily available higher education data

This article is based on the paper “Learning Analytics Dashboards to Support Adviser-Student Dialogue”, published in IEEE Transactions on Learning Technologies in June 2017. I presented this work recently at the Dataviz Belgium Spring Meetup 2019, Mechelen, Belgium. It was originally published on Medium, but deserves a spot on my own blog…

During my PhD I mainly focused on Learning Dashboards. Most of these the research on these dashboards focuses on teachers and researchers as users, but my goal was to empower the students, by giving them insights into their own process through data visualisation. I designed and evaluated a number of dashboards, but one stood out: LISSA, or “Learning dashboard for Insights and Support during Study Advice”. LISSA supports the conversation between study advisers and students, and has been deployed across campuses of KU Leuven.

We involved 17 study advisers in our study. These study advisers are responsible for both the study advice and content-related support for first-year students in a particular program. They are experts in both the content of the first-year courses, the current organisation of the program, and the regulation, both program-specific as university-wide.

The study advisers helped us get insights into how they work, and what their requirements were during these advising sessions. These sessions are private conversations with a student (occasionally with parents) taking place in an office environment. These students typically do not have a flawless study career: they have trouble studying, would benefit from a personalised program plan, did not achieve enough credits through the year, or simply wish to re-orientate towards a new program.

The study advisers have multiple tools and websites to their disposal. But combining and interpreting these multiple channels of information for each specific student requires effort and time, and is error-prone. In addition, data is often incomplete.

The dashboard

Using a user-centred, rapid-prototyping design approach, we started of with Sketch for our initial designs and D3.js / Meteor to create our final interactive prototype. Here are two screenshots of the final design (in the meantime current PhD students redesigned the interface):

Year Overview

LISSA provides an overview of every key moment in chronological order up until the period in which the advising sessions are held: the grades of the positioning test (a type of entry-exam without consequence), mid-term tests, January exams, and June exams. A general trend of performance is visualised at the top: the student path consists of histograms showing the position of the student among their peers per key moment.

Every course is represented by its name and grade (out of 20). A green, orange, and red colour coding represents successful exams, tolerable grades (students can request to pass a limited number of 8–9/20 grades) and failed courses. The course is accompanied by a histogram visualising the performance of peers and the position of the student among them (black highlight).

Planning

For June sessions, it is important to plan the re-sits during September. Too few exams result in a credit threshold issue, while too many will most likely result in failure. The check-boxes next to the failed exams re-sit planning let adviser and student select several courses. The “re-sit exam success rate graph” uses historical data to provide insights into the number of students succeeding the selected number of exams in the past.

Prediction

The stacked prediction bar provides historical data of students with a similar profile (based on the number of exam passed or failed) to the student: it shows the distribution of the duration of the bachelor program (three-four-five years or drop-out/ “NIET”) with similar September re-sits.

Data Sources

To visualise the key moments, data regarding student grades is required. This includes all first-year students of the current year to populate the courses and course histograms, the student path, and course histograms. All grades regarding the January, June and September periods are available in the KU Leuven data warehouse.

The stacked prediction bar is based on the first-year student grades of previous years. This provides the data needed to predict the three, four, five, or more years length of a Bachelor degree.

We created a data process pipeline using Python scripts to convert the different files and formats into a simple representation that is imported into a MongoDB.

Lessons Learnt

The role of the Learning Analytics data

LA dashboards are often developed for specific institutions with certain data requirements. The Learning Analytics data necessary to deploy LISSA is very basic: grades of students across key moments and data regarding student success (derived from historical grade data). This data is usually available in most higher education institutions, but limited to staff. Yet, we have shown that this data placed in a student advising context, can help support students, provide insights into their progress and help plan their future.

LISSA is based on factual data. Exam success rate and bachelor duration show what has happened historically as facts and provide no calculated estimations. This reliable way of visualizing the data provides reassurance among both study advisers and students about the advice they are giving and receiving.

Personal background data regarding socio-economic status, parents’ education, gender, and high school achievement can provide further insights and help the study adviser understand the student’s situation better. However, this unalterable data does not provide the students with actionable insights. It is therefore important to investigate how to integrate this data in an ethical manner.

The role of the Study Adviser

LISSA facilitates insights at multiple levels, but these insights benefit from guidance by the study adviser. Even though the data is objective, there is still a need for critical and reflective interpretation by domain experts . Overconfident students might interpret an overall negative result as a surmountable problem, whereas the study adviser could advice and plan a more achievable program, preventing the student from wasting years on incorrect choices. LISSA can portray a student in a negative way, while a discussion with the student might reveal problems that are easily resolved, e.g. a change in study method, a new program, or a change in attitude. Without the study adviser’s guidance, such students might choose not to continue their Bachelor program.

LISSA still leaves room for personal opinions and tacit experience, as they still play an important role during advising sessions by allowing study advisers to e.g. emphasise certain results to push them on the correct path. Many external factors, such as information gathered through discussion and previous study adviser experiences with students, impact the decision to deviate from the factual data or interpret it differently.

Transparency

During the semi-structured interviews and workshops, ethical issues arose regarding confronting the students with the data.

Some study advisers did not show LISSA to students with a very high number of failed courses. While some students might benefit from an “eye-opener”, study advisers prefer to use LISSA as a motivational tool.

An important role of LISSA is the ability to position a student among peers. In general, the use of histograms was considered very useful and positioning had positive effects such as motivating the student when a bad grade is still good among peers, or a course failure rate is very high. Some study advisers worried about demotivation: a student on the low end of the histogram might see succeeding the course as an unachievable goal.

There is however a demand by students to gain access to LISSA outside the sessions. But study advisers fear that the data visualised can be greatly misinterpreted without their guidance: students with fear of failure or over-confident students might interpret the data incorrectly. Parents can play a negative role into either pushing too hard, or interpreting mediocre results as insurmountable. The lack of knowledge about higher education with parents without a degree might stop a student pursuing an achievable degree. These problems might result in wrong decisions regarding exam and study career planning. Study advisers do see potential in providing reduced information, but what this reduction entails must be further discussed and evaluated.

For more details regarding the process, evaluation and findings, check out the original paper. Need help visualising your data? Send me an email, I’m for hire!

Why Deepfakes are a good thing

[Also on Medium]

If you should believe the media, Deepfakes equals the end of days. Fake news, fake evidence, this technology will create a world of disbelief, blackmail,… trust goes out the window, we’re all doomed. Period.

The media likes to scare us, as, well, scary stories sell. And for once, it might actually help get the positive message across.There’s a way to use Deepfakes for good, and I’m not just talking about my wife on the Tonight show.

We believe everything

Whether it’s the news, a magazine, a book, gossip, it’s all just words, spread around, by friends or “reliable” sources. And we (should) all know to take everything we hear and read with a grain of salt. Yet, we let them lead our thoughts and our beliefs, which influence important life choices such as who we vote for, who we love, who we hate. Words are the most easily manipulated form of message (why not buy a book on how to manipulate people). And still, millions of people live by words that cannot be proven to be true, on a daily basis, and would even give their lives for them.

Roswell’s Flying Saucer 1947

We used to consider photographs real. Then Photoshop happened.

Trump & Saruman

Video is real. Even though we know 90% of what we see in Hollywood movies is fake, every other video is real right?

We must become more aware

Fake news is a big topic. While we know in our hearts never to trust politicians and news outlets, we still do, we still believe every word they say but go crazy when we finally find out it’s all lies. We don’ blame ourselves, though.

Deepfakes just added another level of worries. Video can be faked?.. Is anything we see and hear real anymore?

No. Nothing is. Just as you shouldn’t believe every single word on tv, in the newspaper or [insert your favorite religious book here], video is just another medium ready to be manipulated.

But don’t blame Deepfakes. Don’t demonise the technology. Thank Deepfakes!

Think about this: If someone was able to create this technology in their bedroom throwing together a bunch of existing tools, someone with a bigger budget must have pulled it off a long time ago. There’s no doubt large organisations with massive resources haven’t explored these techniques. Who knows, maybe we’ve seen some of their work, on the news, without knowing it.

So thank Deepfakes, for making us aware of this, making us realise once again that we can’t take everything we see and hear for granted. For creating a problem for us to solve, early on, before it becomes so big, and has influenced so many of us incorrectly, that it’s too late.

It will take time. A new skill we must all learn. So doubt that next video you see on the Internet. Hell, doubt everything you see, read, or hear. Be more critical! Think for yourself.

Family fun with deepfakes. Or how I got my wife onto the Tonight Show

[Also on Medium]

[ Update 3 Feb 2018: added two new creations at the bottom of this post. Last one turned out really well ]

I’ve first heard of deepfakes a good week ago. Thanks Twitter. Thanks Tim Soret.

Yes, it’s pretty damn cyberpunk. But from a superficial point of view, /r/deepfakes (extremely NSFW! You have been warned) consists of people using an app created by user “deepfakes” to create fake celebrity porn.

This has caused a shitstorm on the Internet, media discussing the legality of it all, websites taking down the deepfake creations, and people panicking as they realise AI is going to screw us all up (newsflash: it’s already been happening in much less obvious ways). And meanwhile, Nicolas Cage is taking over Hollywood.

While everyone’s debating whether this is good or bad, I just had to find out more. First thing that came into my mind? How can I apply this to everyone I know (in a non-porn way, in case you wondered).

How does it work?

The deepfakes app is a deep learning algorithm that learns how to reconstruct faces. Give it a bunch of pictures, let it run a few hours, and it spits out fuzzy copies of those images. Do note, it doesn’t create a copy. It learns what a face looks like, in different expressions, and is able to output that face solely based on that. There’s a detailed explanation on Reddit but let me try and dumb it down.

Think of it like this: imagine if you could stare at someone for 12 hours straight, observing all their expressions and absorbing that into your brain. Then that person asks you to sketch his face on paper, smiling, crying, any expression you’ve observed. What do you do? You immediately generate a photographic quality sketch on paper, from the mind! (using a pencil)

It’s insane!

While that’s pretty cool, it only gets better. See that “encoder” part? The FakeApp uses one encoder for all faces. It’s the decoder that’s kept face specific. And here comes the really cool part. Let it learn two faces, and things become more interesting.

Right, now see how this works. The encoder takes an image of a face, let’s it run through its “brain”, and the decoder spits it out again. In the example above, it can do so with faces of Anne Hathaway, and Elke, my wife. Ok, so far so good. But now let’s feed it a picture of Anne, but use the decoder that generates Elke’s face!

You just created a new photo of Elke. A photo that never existed, in the same angle, the same expression, as Anne! Amazing!

Family fun

Sure, putting celebrities’ faces on your favorite porn stars is an interesting use case. But we can leverage these celebrities for other things, such as inserting your friends and family into blockbuster movies and shows!

For the best result, you must first find an actor/actress that has a similar head shape as the person you wish to insert. In case of Elke (my wife) I accidentally noticed, while watching the Dark Knight Rises, that Anne Hathaway might be a good fit. I guess you know Anne, so here’s Elke:

All I needed was about 850 photos of Elke, a few 1000 of Anne, a lot of computing time, et voila: Elke’s on the Tonight Show starring Jimmy Fallon.

Bonus effect: now we know what Elke looks like with short hair :D

Here’s a little comparison gif:

I personally think it’s fun, can be innocent, and even makes for a nice surprise/gift. Remember, any tool can be used for evil. And as long as we’re not banning guns, this should not be a high priority, amirite?

There’s so much you can do with this technology. You know those dumb emails people send around where they replaced dancing elves heads with their own, or even worse, yours? Well, now you can put your best friend into his favourite movie: have her dance with Patrick Swayze and have the time of her life, or have an alien burst out of his stomach. It’s all within your reach now!

Beyond just pure fun, I can only imagine how people will start turning this tech into business ideas. Fashion will be huge (what would I look like with this kind of hair, this kind of dress…), fitness could be interesting (do I look good with muscles, will I really look better skinny), travel (this is you standing on a beach is going to be quite convincing). It’ll bring advertising to a whole new level. No need to imagine what if, they’ll tell you what your “better” life will look like! And it’ll be hard to get that picture out of your head…

Update: in the mean time, I’ve created two more. Elke’s a huge fan of Steve Carell, and I suddenly realized Anne Hathaway co-starts with him in Get Smart. First attempt was okay:

Then I wanted to try this one (original video):

And I think it turned out great:

 

Help us augment live streams of your game! For science!

TL;DR: HCI research group at KU Leuven looking for game studios to provide access to live game logs for 2 Master theses, or even collaborate with our group around data visualisation for streaming research

Every year our HCI research group defines a number of Master thesis topics for Computer Science and Engineering students. Our topics include data visualisation, recommender systems, augmented reality, learning analytics, and e-health.

As my expertise is data visualisation, and video games have always held a special place in my heart, it only made sense to merge the two. That’s why this year we have two Master students working on  “Designing live data visualisations for the new spectator sport: video games”.

The general idea is to use live (interactive) visualisations during a game to help a specific audience get a better understanding of and new insights in what is going on. Some examples:

  • help people who are new to the game to get a better understanding of the game
  • assist the streamer in informing their audience better
  • help the streamer’s fans get more insights into their specific idol’s activities
  • facilitate a better overview to assist commentators

… anything goes, but we’ll be narrowing it down ASAP, as both theses must be finished by June 2018 ;)

For this topic my students require access to activity logs of a game, preferably during live play. The availability and type of data will play a big part in the choice of game (type) and target audience of the visualisation.

So what we need is a game studio (big or small) that is developing a game with Twitch/e-sports in mind, or a studio that has an established game with e-sports/twitch popularity. What we want from you is access to detailed game logs or an API, for science!

If you think you could help in any way, let us know! We can work independently, sharing our research results in a way that can help you, or we can even look for official collaboration options (we could even apply for funding, certainly if you have offices in Belgium), within and beyond the scope of these theses (this might actually be very interesting for small Flemish indie studios because of government funding possibilities).

Interested? Want to know more? Shoot me an email, or leave a comment below!

In case you wish to learn more about us, read on.

A  Master thesis at the Human-Computer Interaction group.
Our process is that of design-based research, iteratively designing prototypes with a heavy focus on the user, involving them in the design process and the evaluations. The result should be a fully working prototype that has been proven to be an added value to the target audience, by being usable, useful, and generating meaningful insights. The student also writes a 50-70 page thesis text, reporting on the current state of the art, the thesis process and results, and the research contribution. To pass the thesis (a requirement to finish the Master program) the student has to successfully defend the work in front of a jury of academics.

About our research group:
http://augment.cs.kuleuven.be

About our university:
https://www.kuleuven.be/english/

About me:
http://svencharleer.com/blog/about/

My PhD Acknowledgements <3

Four point five years. Summarised in 131 pages and 18 scientific publications. That and a new title.

While the new title does sound cool (how else can one be taken seriously when announcing world domination), the biggest result is the little book: the dissertation, in my case, on Learning Dashboards. But while I hope the pages reach the intended audience (the Learning Analytics community), I wonder if anyone beyond that will read it.

Probably not. And that’s fine.

But more than just research and writing went into this achievement. A lot of people, friends, family, and colleagues, were part of the process. Pulling and pushing me along the path that led me here. So even if the main contribution doesn’t reach a large audience, I hope the acknowledgements in my book do. Without these people, I would not be where I am today, nor would this small advancement in Learning Analytics research.

So I could be ignorant and assume you’ll download and read the book ;) But I’ll just leave you with the first few pages, the important ones about.. the important ones.

Here we go…

Twenty years ago, I figured, let’s do Physics. I loved all things space and dinosaur related, and had an awesome Physics teacher. However, that same teacher told me studying Physics would most likely land me in Finance. So, I followed my other passion, Computer Science.

 

But after completing my degree and spending years in the private sector, I wondered if I had made the right decision. While I had a passion for programming, the lack of creativity that comes with a software engineering job (beyond the code) was killing me. Moving to Nottingham to pursue a career in games did not improve the situation either: a developer just develops, it seems. Side projects (indie game development and art academy) were an attempt to bring some creativity back into my life, but I needed a serious change where I spent most of my time: the day job.

 

The PhD was an unexpected opportunity that presented itself in my mailbox. Two weeks it sat there until I finally decided to reply. What followed was life changing. Research meant I could explore the unknown, build things no one had before, and join the user in their experience with our new creations. I got paid to create visualisations, play with new technology, and spend time thinking of all the crazy things we could accomplish with it. Both the nerd and the artist in me were satisfied. This might just be where I belong. It only took me 36 years to figure it out…

 

Thirty-six years is a long time. I owe where I am today to a lot of amazing people: for the opportunities, the support, the patience, and the listening.

 

I would like to thank Erik. In 2013, I somehow convinced him I would be the right guy for the job. When times got tough, he would keep convincing me I was. “I wasn’t that smart either and look where I am now”, a pep talk I will never forget. From all the “bosses” I’ve had, he was one of the few who genuinely cared about his people, at a personal, family, and career level. Thanks for letting me get to know you and your amazing family. Your awesome ideas will live on in our work, we will all make sure of that.

 

If it wasn’t for Bert, I would have never even considered a PhD. But it was his better half, Katrien, who got me in the room with Erik. I owe a lot to her. She stepped in just as I was close to jumping ship. She was the motivating force I needed, and pushed me across the finish line. Thanks to both Katrien and Tinne, this last year and a half of the PhD has been amazing. We’ve published great papers, made a name in the community, and put our stamp on student advice at the university.

 

Joris. What was it Jose said, thanks for the coffee? I’ll do one better: bourbon at Harvard, such an amazing trip! He never once doubted me, and I will never forget his endless “het komt goed” (it will be all right). And who’d have thought, it did! (I guess this calls for another round of drinks in Boston!).

 

I would also like to thank Andrew, Bieke, Yolande, and Martin, for taking the time to read my dissertation, providing valuable feedback, and a memorable private defence.

 

But my biggest thanks goes to Elke. I could not have achieved this without the love of my life (15 years this year, 10 as my gorgeous wife). Always supporting my crazy decisions. She quit a promising career to follow me abroad and let me pursue my game developer dream back in 2008. And in 2013 she supported me again in my biggest career change, when I gave up a well-paid, secure job to become a student once again. We were not expecting the PhD to be such a roller-coaster. I experienced moments of joy and despair, feelings that would affect her as much as it did me. But she always had my back, endured the after-work rants, and supported me in every way possible. Without her, I would never have managed.

 

Hazel joined us (in the womb) at the start of the PhD. Kids do not make things easier. But they do give you a reason to keep going. During dark periods of the PhD, she was always there to put a smile on my face (or add to the misery with sleepless nights. She’s a little monster like that). Hazel, if you read this when you are older, we love you and we will make sure you get to follow your dreams just as we could.

 

My parents, Marinette and Guy: they have always been there for me, supported me, and believed in me (and also provided me with all the nerdy hardware a kid needs to keep his technology interest going). And my grandparents, Meke, Vake, Peter Wieke, and Bobonne. Meke is not here to see this, but if someone believed I could pull this off, it was her.

 

My parents-in-law, Monique and Daniel, and the “Moekes”, for treating me like one of their own. Monique, the things you have missed out on, it is not fair. We miss you, words cannot describe.

 

Kurt, thanks for showing up at the defence (if you didn’t, this is going to be awkward). You’ve been that one true best friend. Always there in time of needs. And always making me look good at the board game table.

 

Franky, the bastard who pulled me out of the Flemish, secure mindset, and lured me to England. I ended up working long unpaid hours in Nottingham and lived amongst criminals and drunks. But I regret nothing!

 

José, for his unique perspectives on things, telling it like it is, and your attempt at keeping me sane through the PhD (it did not work).

 

Thomas for the babysitting and being Hazel’s coolest uncle. And “Tantan”, for taking care of Hazel all those Mondays, and for all the things you have done for Elke.

 

Kris, for bringing that new addiction into my life. The Nets won’t run themselves! Lies, for being Hazel’s awesome godmother. Jim, thanks for letting me win sometimes. Wait, no.

 

Sean, Jenna, Greg, and Johnny. One day Rad Lab Games will rise again!

 

François and Denis, we will make that dinner happen and bathe in Brasschaat’s sushi! Until then we will just shoot people online.

 

NorthgateArinso: Fred, Schtroumpf, Maarten, Karo, and Tom. My first and fourth job (thanks Samir), and also my last job before I ran off to academia (I am not implying anything!).

 

Everyone at Monumental Games and iChoosr, even though the stops were short, they were life changing.

 

My current, former, and visiting colleagues at the coolest lab of the Computer Science department: Yucheng, Karsten, Francisco, Gonzalo, Robin, Tom, Sam, Bruno, Victor, Gayane, Chen, Sten, Till, Frans, Samara, and Oana. And all the amazing people of the weSPOT, eCloud, and ABLE projects.

 

The Blade Runner soundtrack, for getting me through numerous paper deadlines.

 

And Bert. For getting me into this mess in the first place.

 

Dude, sucking at something is the first step towards being sorta good at something.

– Jake the Dog, Adventure Time

I love you, Pumpkin.

– Honey Bunny, Pulp Fiction