LISSA: a student-facing Learning Analytics dashboard based on readily available higher education data

This article is based on the paper “Learning Analytics Dashboards to Support Adviser-Student Dialogue”, published in IEEE Transactions on Learning Technologies in June 2017. I presented this work recently at the Dataviz Belgium Spring Meetup 2019, Mechelen, Belgium. It was originally published on Medium, but deserves a spot on my own blog…

During my PhD I mainly focused on Learning Dashboards. Most of these the research on these dashboards focuses on teachers and researchers as users, but my goal was to empower the students, by giving them insights into their own process through data visualisation. I designed and evaluated a number of dashboards, but one stood out: LISSA, or “Learning dashboard for Insights and Support during Study Advice”. LISSA supports the conversation between study advisers and students, and has been deployed across campuses of KU Leuven.

We involved 17 study advisers in our study. These study advisers are responsible for both the study advice and content-related support for first-year students in a particular program. They are experts in both the content of the first-year courses, the current organisation of the program, and the regulation, both program-specific as university-wide.

The study advisers helped us get insights into how they work, and what their requirements were during these advising sessions. These sessions are private conversations with a student (occasionally with parents) taking place in an office environment. These students typically do not have a flawless study career: they have trouble studying, would benefit from a personalised program plan, did not achieve enough credits through the year, or simply wish to re-orientate towards a new program.

The study advisers have multiple tools and websites to their disposal. But combining and interpreting these multiple channels of information for each specific student requires effort and time, and is error-prone. In addition, data is often incomplete.

The dashboard

Using a user-centred, rapid-prototyping design approach, we started of with Sketch for our initial designs and D3.js / Meteor to create our final interactive prototype. Here are two screenshots of the final design (in the meantime current PhD students redesigned the interface):

Year Overview

LISSA provides an overview of every key moment in chronological order up until the period in which the advising sessions are held: the grades of the positioning test (a type of entry-exam without consequence), mid-term tests, January exams, and June exams. A general trend of performance is visualised at the top: the student path consists of histograms showing the position of the student among their peers per key moment.

Every course is represented by its name and grade (out of 20). A green, orange, and red colour coding represents successful exams, tolerable grades (students can request to pass a limited number of 8–9/20 grades) and failed courses. The course is accompanied by a histogram visualising the performance of peers and the position of the student among them (black highlight).

Planning

For June sessions, it is important to plan the re-sits during September. Too few exams result in a credit threshold issue, while too many will most likely result in failure. The check-boxes next to the failed exams re-sit planning let adviser and student select several courses. The “re-sit exam success rate graph” uses historical data to provide insights into the number of students succeeding the selected number of exams in the past.

Prediction

The stacked prediction bar provides historical data of students with a similar profile (based on the number of exam passed or failed) to the student: it shows the distribution of the duration of the bachelor program (three-four-five years or drop-out/ “NIET”) with similar September re-sits.

Data Sources

To visualise the key moments, data regarding student grades is required. This includes all first-year students of the current year to populate the courses and course histograms, the student path, and course histograms. All grades regarding the January, June and September periods are available in the KU Leuven data warehouse.

The stacked prediction bar is based on the first-year student grades of previous years. This provides the data needed to predict the three, four, five, or more years length of a Bachelor degree.

We created a data process pipeline using Python scripts to convert the different files and formats into a simple representation that is imported into a MongoDB.

Lessons Learnt

The role of the Learning Analytics data

LA dashboards are often developed for specific institutions with certain data requirements. The Learning Analytics data necessary to deploy LISSA is very basic: grades of students across key moments and data regarding student success (derived from historical grade data). This data is usually available in most higher education institutions, but limited to staff. Yet, we have shown that this data placed in a student advising context, can help support students, provide insights into their progress and help plan their future.

LISSA is based on factual data. Exam success rate and bachelor duration show what has happened historically as facts and provide no calculated estimations. This reliable way of visualizing the data provides reassurance among both study advisers and students about the advice they are giving and receiving.

Personal background data regarding socio-economic status, parents’ education, gender, and high school achievement can provide further insights and help the study adviser understand the student’s situation better. However, this unalterable data does not provide the students with actionable insights. It is therefore important to investigate how to integrate this data in an ethical manner.

The role of the Study Adviser

LISSA facilitates insights at multiple levels, but these insights benefit from guidance by the study adviser. Even though the data is objective, there is still a need for critical and reflective interpretation by domain experts . Overconfident students might interpret an overall negative result as a surmountable problem, whereas the study adviser could advice and plan a more achievable program, preventing the student from wasting years on incorrect choices. LISSA can portray a student in a negative way, while a discussion with the student might reveal problems that are easily resolved, e.g. a change in study method, a new program, or a change in attitude. Without the study adviser’s guidance, such students might choose not to continue their Bachelor program.

LISSA still leaves room for personal opinions and tacit experience, as they still play an important role during advising sessions by allowing study advisers to e.g. emphasise certain results to push them on the correct path. Many external factors, such as information gathered through discussion and previous study adviser experiences with students, impact the decision to deviate from the factual data or interpret it differently.

Transparency

During the semi-structured interviews and workshops, ethical issues arose regarding confronting the students with the data.

Some study advisers did not show LISSA to students with a very high number of failed courses. While some students might benefit from an “eye-opener”, study advisers prefer to use LISSA as a motivational tool.

An important role of LISSA is the ability to position a student among peers. In general, the use of histograms was considered very useful and positioning had positive effects such as motivating the student when a bad grade is still good among peers, or a course failure rate is very high. Some study advisers worried about demotivation: a student on the low end of the histogram might see succeeding the course as an unachievable goal.

There is however a demand by students to gain access to LISSA outside the sessions. But study advisers fear that the data visualised can be greatly misinterpreted without their guidance: students with fear of failure or over-confident students might interpret the data incorrectly. Parents can play a negative role into either pushing too hard, or interpreting mediocre results as insurmountable. The lack of knowledge about higher education with parents without a degree might stop a student pursuing an achievable degree. These problems might result in wrong decisions regarding exam and study career planning. Study advisers do see potential in providing reduced information, but what this reduction entails must be further discussed and evaluated.

For more details regarding the process, evaluation and findings, check out the original paper. Need help visualising your data? Send me an email, I’m for hire!

Another job, another commute. Visualized

I’ve had quite a few different jobs.. well, about five… But that’s quite a lot for your typical Belgian (we fear change!). This resulted in a range of long car commutes, short bicycle commutes, extreme long train commutes… You’d think I’d learn and go for shorter with every job change, but alas… I didn’t.

My latest commute is even the longest yet. But working at Augment, Erik Duval’s and now Katrien Verbert’s research group at KU Leuven, has been worth it. And it probably is the most fun/satisfying/creative/soul shattering/depressing (yes) job I’ve had. Good times!

Anyway, it got me thinking that I should try to visualise the differences between my commutes. No rocket science, but I thought it would be fun to walk you through them.

commute_legend

First, a quick explanation. The colors are explained in the image above, the thickness of a line indicates the difficulty of that part of the trip (traffic jams for car, waiting for connections in case of the train, and cycling just requires effort,.. always). The partial donuts indicate ring roads and how much distance is spent on them. All of these are very rough estimates, enough to tell my story. The bar on the right indicates the best possible time (green) to the worst possible time (red). These are based on what Google Maps estimates during morning rush hours (again, no exact science here). Okay? Let’s go.

Job #1: Northgate Arinso

commute_1

Ah, Brussels and its traffic jams. Our country is small, but our traffic jams are amongst world’s biggest. And we chose to live north of Brussels, which meant I’d have to head through the longest and busiest part of the Brussels Ring. I was young and stupid,  and spent 4.5 years in horrible traffic jams. Seriously hated it in the end. But it did give me a lot of time to think. Think about what my next job should be! (staring at the trunk of the car in front of you is quite.. zen and inspiring?)
That thick part at the start of the trip? Two lanes merging onto one on a bridge. Could add an hour to your route if you didn’t know the shortcuts. Horrible! (they widened the bridge AFTER I moved job/house of course)

Job #2: Monumental Games

commute_2

I managed to land a job in the games industry, a dream of mine since I was a kid. Added bonus: I went from 2-3 hours a day wasted in the car to my shortest commute ever. Was even short enough  to have lunch at home with Elke! I had to buy a bicycle (I was a.. car person, but on a puny game developer salary.. no dice), but in the end the overtime was worse than my previous commute time. So.. I left!

Job #3: iChoosr

commute_3

There were many reasons why I left the game industry (there are things I miss: many of my colleagues, making games, and a bit of Nottingham. And things I don’t miss: rain), but it was time to move on, and we arrived back in Belgium: Antwerp. Being even closer to work, getting the bicycle out of the basement was sometimes just not worth the time. Elke’s work was right next to ours, so we could meet up for lunch every day. Second best lunches ever!

commute_4

We ended up buying a house, away from the city. So the commute got longer. Still, two bicycle rides and a trip on the train weren’t too bad… But the time it takes for the short distance was quite ridiculous (welcome to Belgium).

Job #4: Northgate Arinso

commute_5

Leaving iChoosr was a bit weird (I was the first at the company to ever leave..), but I thought, hey, commutes, I can handle those! I’ve done horrible commutes for almost five years! .. But I did seem to forget we relocated: we moved to the north of Antwerp, which added another ring road to my trip… It took me 3 months to realise what I did. It took me another year to find an exit.

Job #5: KU Leuven

commute_6

Okay, it’s even further away. But I get to do research! Be creative! Get a PhD in the end (hopefully). Two ring roads again, but this time, the Brussels part is quite short.

commute_7

Universities do not give you cars, but they do give you train subscriptions! So, thanks to the amazing services of the NMBS (Belgian railways), with its weekly/daily delays and cancelled trains, it can take up to three hours to get to work (or you just can’t get anywhere,.. it happens more often than you think). This can get quite frustrating, but the train has become my second office. Not a complete waste of time..

 

Now, if I had more time, it would be great to visualise my love for each job/company/colleagues, and weigh it against salary, commute distance etc. But hey, I don’t want to burn any bridges. And even so, every job I’ve had was amazing in its own way. I made great friends, learned things! Not regretting any of it..

I will end with this: the actual locations of the companies. And here, I’ve improved. I present to you, from left to right: Northgate Arinso, Monumentel Games, iChoosr, Northgate Arinso again,.. and KU Leuven.

maps2

Yep. We’ve gone greener. However, the access to good lunch places is equally bad on both the university campus and the industrial area in which Northgate Arinso was located. The prettiest building I’ve worked in (on the outside anyway) deserves some attention as well: “The Prudential Building” in Nottingham.

nb00161

Hope you enjoyed the trip (pun intended, of course). If you like this type of visualisation, I’d love to see it applied to your own commutes. Or anything trip related really. Do leave comments!

Oh, and lessons learned? Err… Balance is everything! But home working is the best?!