Category: Research

Real-Time Dashboards to Support eSports Spectating

Leave a comment

This must have been my favorite project during my time at academia.

This article is also available on Medium.

The League of Legends and Counter-strike: Global Offensive dashboards in action! Read on to learn more.

My PhD was funded through multiple (European and smaller) projects, on topics such as Learning Analytics, Digital Humanities, and Unemployment. And they were interesting: I designed data visualizations on tabletops, desktop, and mobile, I researched the user’s perception and experience… all fun and games.

But no, not games.

I’ve always had a soft spot for games. I moved abroad for two years (and had my wife give up a great job) just so I could live the dream and be a game developer (a story for another time perhaps). And combining my research with games was something I had dreamed of for quite a while.

Alas, no funding to support this research. But life finds a way, right? ;)

Students to the Rescue

Students! Of course! Every year we had new Master in Engineering students working on both fun (and sometimes boring) Master thesis topics. So I introduced a new one: eSports dashboards to support spectators. The high paced action of different players across large battlefields is not easy to follow. So how could we help spectators get better insights in what was going on, real-time, during these hectic games played by professional gamers.

A huge success among students, but sadly enough spots were limited, and two students “won the lottery” (we have no say in the matter. It’s quite random): Hans Cauwenbergh and Bram Luycx.

And we got lucky. This turned out to be one of the best collaborations with students ever.

The Dashboards

Hans jumped on League of Legends, while Bram took on Counter-strike: Global Offensive. We designed and set up a couple of surveys on Reddit in order to get insights in spectators’ motivations and perceived usefulness for different statistics provided by the respective games to inform the design of the dashboards. And then, they were off!

Viewer preferences (167 responses for League of Legends, 596 for Counter-strike: Global Offensive)

League of Legends

a) Gold distribution, b) Damage dealt, c) Vulnerability.

The dashboard is the result of five iterations: two digital non-interactive designs, two interactive prototypes, and the final fully functioning prototype developed with React.js and D3.js. Iteration one to four were developed using the interface design tool Figma. 22 participants took part in the evaluations of the iterations of the prototypes.

The main view is split in three modules: Gold, Damage, and Vulnerability. All modules are continuously updated in real-time. Gold and Damage are quite obvious. Let me explain Vulnerability.

Vulnerability provides an indication of how vulnerable each player is. A simple linear equation combines the position disadvantage of the player to the rest of the team and/or enemies, current health and mana points, defensive statistics and summoner spells availability. These values are combined and visualized around the player icon as a summary indicator. This view lets spectators make predictions regarding the state of the game.

Hans also added an interesting Jungler visualisation:

Counter-strike: Global Offensive

a) Round progression, b) Economy button, c) Map button, d) Scoreboard, e) Money Distribution, f) Average Damage per Round

This dashboard is the result of three iterations: a paper prototype, a non-interactive digital prototype designed using Figma, and a fully functioning prototype developed with React.js and D3.js. 14 participants took part in the evaluations of the iterations. The dashboard consists of Round Progression, Economy, Map, Scoreboard, Money Distribution, and Average Damage per Round.

Results

Through semi-structured interviews and eye-tracking we managed to get further insights into the perception and experience of participants spectating these eSports games while using our dashboards. It resulted in an interesting list of design goals for spectator dashboards that we published and presented at CHI PLAY 2018 in Melbourne, Australia.

LoL and CS:GO eye tracking results from each participant after the evaluation. The attention heatmap indicates the areas of interest where participants focused more during the whole match.

I’ll refrain from turning this digestible blog post into a long scientific article. But if you are interested in the guidelines, you can find the article here. Or go through the slides below!

If there is enough interest, I’ll post a summary of the guidelines on Medium ;) Just let me know in the comments!

Hans (full-time) and I (freelance part-time) haven’t stopped here though and are working for GRID eSports Gmbh where a lot of eSports data magic is happening!

Next to that, I’m a Freelance Data Visualisation Experience Designer/Adviser. I’m for hire, so if you have any dataviz needs, send me a message!

LISSA: a student-facing Learning Analytics dashboard based on readily available higher education data

Leave a comment

This article is based on the paper “Learning Analytics Dashboards to Support Adviser-Student Dialogue”, published in IEEE Transactions on Learning Technologies in June 2017. I presented this work recently at the Dataviz Belgium Spring Meetup 2019, Mechelen, Belgium. It was originally published on Medium, but deserves a spot on my own blog…

During my PhD I mainly focused on Learning Dashboards. Most of these the research on these dashboards focuses on teachers and researchers as users, but my goal was to empower the students, by giving them insights into their own process through data visualisation. I designed and evaluated a number of dashboards, but one stood out: LISSA, or “Learning dashboard for Insights and Support during Study Advice”. LISSA supports the conversation between study advisers and students, and has been deployed across campuses of KU Leuven.

We involved 17 study advisers in our study. These study advisers are responsible for both the study advice and content-related support for first-year students in a particular program. They are experts in both the content of the first-year courses, the current organisation of the program, and the regulation, both program-specific as university-wide.

The study advisers helped us get insights into how they work, and what their requirements were during these advising sessions. These sessions are private conversations with a student (occasionally with parents) taking place in an office environment. These students typically do not have a flawless study career: they have trouble studying, would benefit from a personalised program plan, did not achieve enough credits through the year, or simply wish to re-orientate towards a new program.

The study advisers have multiple tools and websites to their disposal. But combining and interpreting these multiple channels of information for each specific student requires effort and time, and is error-prone. In addition, data is often incomplete.

The dashboard

Using a user-centred, rapid-prototyping design approach, we started of with Sketch for our initial designs and D3.js / Meteor to create our final interactive prototype. Here are two screenshots of the final design (in the meantime current PhD students redesigned the interface):

Year Overview

LISSA provides an overview of every key moment in chronological order up until the period in which the advising sessions are held: the grades of the positioning test (a type of entry-exam without consequence), mid-term tests, January exams, and June exams. A general trend of performance is visualised at the top: the student path consists of histograms showing the position of the student among their peers per key moment.

Every course is represented by its name and grade (out of 20). A green, orange, and red colour coding represents successful exams, tolerable grades (students can request to pass a limited number of 8–9/20 grades) and failed courses. The course is accompanied by a histogram visualising the performance of peers and the position of the student among them (black highlight).

Planning

For June sessions, it is important to plan the re-sits during September. Too few exams result in a credit threshold issue, while too many will most likely result in failure. The check-boxes next to the failed exams re-sit planning let adviser and student select several courses. The “re-sit exam success rate graph” uses historical data to provide insights into the number of students succeeding the selected number of exams in the past.

Prediction

The stacked prediction bar provides historical data of students with a similar profile (based on the number of exam passed or failed) to the student: it shows the distribution of the duration of the bachelor program (three-four-five years or drop-out/ “NIET”) with similar September re-sits.

Data Sources

To visualise the key moments, data regarding student grades is required. This includes all first-year students of the current year to populate the courses and course histograms, the student path, and course histograms. All grades regarding the January, June and September periods are available in the KU Leuven data warehouse.

The stacked prediction bar is based on the first-year student grades of previous years. This provides the data needed to predict the three, four, five, or more years length of a Bachelor degree.

We created a data process pipeline using Python scripts to convert the different files and formats into a simple representation that is imported into a MongoDB.

Lessons Learnt

The role of the Learning Analytics data

LA dashboards are often developed for specific institutions with certain data requirements. The Learning Analytics data necessary to deploy LISSA is very basic: grades of students across key moments and data regarding student success (derived from historical grade data). This data is usually available in most higher education institutions, but limited to staff. Yet, we have shown that this data placed in a student advising context, can help support students, provide insights into their progress and help plan their future.

LISSA is based on factual data. Exam success rate and bachelor duration show what has happened historically as facts and provide no calculated estimations. This reliable way of visualizing the data provides reassurance among both study advisers and students about the advice they are giving and receiving.

Personal background data regarding socio-economic status, parents’ education, gender, and high school achievement can provide further insights and help the study adviser understand the student’s situation better. However, this unalterable data does not provide the students with actionable insights. It is therefore important to investigate how to integrate this data in an ethical manner.

The role of the Study Adviser

LISSA facilitates insights at multiple levels, but these insights benefit from guidance by the study adviser. Even though the data is objective, there is still a need for critical and reflective interpretation by domain experts . Overconfident students might interpret an overall negative result as a surmountable problem, whereas the study adviser could advice and plan a more achievable program, preventing the student from wasting years on incorrect choices. LISSA can portray a student in a negative way, while a discussion with the student might reveal problems that are easily resolved, e.g. a change in study method, a new program, or a change in attitude. Without the study adviser’s guidance, such students might choose not to continue their Bachelor program.

LISSA still leaves room for personal opinions and tacit experience, as they still play an important role during advising sessions by allowing study advisers to e.g. emphasise certain results to push them on the correct path. Many external factors, such as information gathered through discussion and previous study adviser experiences with students, impact the decision to deviate from the factual data or interpret it differently.

Transparency

During the semi-structured interviews and workshops, ethical issues arose regarding confronting the students with the data.

Some study advisers did not show LISSA to students with a very high number of failed courses. While some students might benefit from an “eye-opener”, study advisers prefer to use LISSA as a motivational tool.

An important role of LISSA is the ability to position a student among peers. In general, the use of histograms was considered very useful and positioning had positive effects such as motivating the student when a bad grade is still good among peers, or a course failure rate is very high. Some study advisers worried about demotivation: a student on the low end of the histogram might see succeeding the course as an unachievable goal.

There is however a demand by students to gain access to LISSA outside the sessions. But study advisers fear that the data visualised can be greatly misinterpreted without their guidance: students with fear of failure or over-confident students might interpret the data incorrectly. Parents can play a negative role into either pushing too hard, or interpreting mediocre results as insurmountable. The lack of knowledge about higher education with parents without a degree might stop a student pursuing an achievable degree. These problems might result in wrong decisions regarding exam and study career planning. Study advisers do see potential in providing reduced information, but what this reduction entails must be further discussed and evaluated.

For more details regarding the process, evaluation and findings, check out the original paper. Need help visualising your data? Send me an email, I’m for hire!

Help us augment live streams of your game! For science!

Leave a comment

TL;DR: HCI research group at KU Leuven looking for game studios to provide access to live game logs for 2 Master theses, or even collaborate with our group around data visualisation for streaming research

Every year our HCI research group defines a number of Master thesis topics for Computer Science and Engineering students. Our topics include data visualisation, recommender systems, augmented reality, learning analytics, and e-health.

As my expertise is data visualisation, and video games have always held a special place in my heart, it only made sense to merge the two. That’s why this year we have two Master students working on  “Designing live data visualisations for the new spectator sport: video games”.

The general idea is to use live (interactive) visualisations during a game to help a specific audience get a better understanding of and new insights in what is going on. Some examples:

  • help people who are new to the game to get a better understanding of the game
  • assist the streamer in informing their audience better
  • help the streamer’s fans get more insights into their specific idol’s activities
  • facilitate a better overview to assist commentators

… anything goes, but we’ll be narrowing it down ASAP, as both theses must be finished by June 2018 ;)

For this topic my students require access to activity logs of a game, preferably during live play. The availability and type of data will play a big part in the choice of game (type) and target audience of the visualisation.

So what we need is a game studio (big or small) that is developing a game with Twitch/e-sports in mind, or a studio that has an established game with e-sports/twitch popularity. What we want from you is access to detailed game logs or an API, for science!

If you think you could help in any way, let us know! We can work independently, sharing our research results in a way that can help you, or we can even look for official collaboration options (we could even apply for funding, certainly if you have offices in Belgium), within and beyond the scope of these theses (this might actually be very interesting for small Flemish indie studios because of government funding possibilities).

Interested? Want to know more? Shoot me an email, or leave a comment below!

In case you wish to learn more about us, read on.

A  Master thesis at the Human-Computer Interaction group.
Our process is that of design-based research, iteratively designing prototypes with a heavy focus on the user, involving them in the design process and the evaluations. The result should be a fully working prototype that has been proven to be an added value to the target audience, by being usable, useful, and generating meaningful insights. The student also writes a 50-70 page thesis text, reporting on the current state of the art, the thesis process and results, and the research contribution. To pass the thesis (a requirement to finish the Master program) the student has to successfully defend the work in front of a jury of academics.

About our research group:
http://augment.cs.kuleuven.be

About our university:
https://www.kuleuven.be/english/

About me:
http://svencharleer.com/blog/about/

My PhD Acknowledgements <3

Leave a comment

Four point five years. Summarised in 131 pages and 18 scientific publications. That and a new title.

While the new title does sound cool (how else can one be taken seriously when announcing world domination), the biggest result is the little book: the dissertation, in my case, on Learning Dashboards. But while I hope the pages reach the intended audience (the Learning Analytics community), I wonder if anyone beyond that will read it.

Probably not. And that’s fine.

But more than just research and writing went into this achievement. A lot of people, friends, family, and colleagues, were part of the process. Pulling and pushing me along the path that led me here. So even if the main contribution doesn’t reach a large audience, I hope the acknowledgements in my book do. Without these people, I would not be where I am today, nor would this small advancement in Learning Analytics research.

So I could be ignorant and assume you’ll download and read the book ;) But I’ll just leave you with the first few pages, the important ones about.. the important ones.

Here we go…

Twenty years ago, I figured, let’s do Physics. I loved all things space and dinosaur related, and had an awesome Physics teacher. However, that same teacher told me studying Physics would most likely land me in Finance. So, I followed my other passion, Computer Science.

 

But after completing my degree and spending years in the private sector, I wondered if I had made the right decision. While I had a passion for programming, the lack of creativity that comes with a software engineering job (beyond the code) was killing me. Moving to Nottingham to pursue a career in games did not improve the situation either: a developer just develops, it seems. Side projects (indie game development and art academy) were an attempt to bring some creativity back into my life, but I needed a serious change where I spent most of my time: the day job.

 

The PhD was an unexpected opportunity that presented itself in my mailbox. Two weeks it sat there until I finally decided to reply. What followed was life changing. Research meant I could explore the unknown, build things no one had before, and join the user in their experience with our new creations. I got paid to create visualisations, play with new technology, and spend time thinking of all the crazy things we could accomplish with it. Both the nerd and the artist in me were satisfied. This might just be where I belong. It only took me 36 years to figure it out…

 

Thirty-six years is a long time. I owe where I am today to a lot of amazing people: for the opportunities, the support, the patience, and the listening.

 

I would like to thank Erik. In 2013, I somehow convinced him I would be the right guy for the job. When times got tough, he would keep convincing me I was. “I wasn’t that smart either and look where I am now”, a pep talk I will never forget. From all the “bosses” I’ve had, he was one of the few who genuinely cared about his people, at a personal, family, and career level. Thanks for letting me get to know you and your amazing family. Your awesome ideas will live on in our work, we will all make sure of that.

 

If it wasn’t for Bert, I would have never even considered a PhD. But it was his better half, Katrien, who got me in the room with Erik. I owe a lot to her. She stepped in just as I was close to jumping ship. She was the motivating force I needed, and pushed me across the finish line. Thanks to both Katrien and Tinne, this last year and a half of the PhD has been amazing. We’ve published great papers, made a name in the community, and put our stamp on student advice at the university.

 

Joris. What was it Jose said, thanks for the coffee? I’ll do one better: bourbon at Harvard, such an amazing trip! He never once doubted me, and I will never forget his endless “het komt goed” (it will be all right). And who’d have thought, it did! (I guess this calls for another round of drinks in Boston!).

 

I would also like to thank Andrew, Bieke, Yolande, and Martin, for taking the time to read my dissertation, providing valuable feedback, and a memorable private defence.

 

But my biggest thanks goes to Elke. I could not have achieved this without the love of my life (15 years this year, 10 as my gorgeous wife). Always supporting my crazy decisions. She quit a promising career to follow me abroad and let me pursue my game developer dream back in 2008. And in 2013 she supported me again in my biggest career change, when I gave up a well-paid, secure job to become a student once again. We were not expecting the PhD to be such a roller-coaster. I experienced moments of joy and despair, feelings that would affect her as much as it did me. But she always had my back, endured the after-work rants, and supported me in every way possible. Without her, I would never have managed.

 

Hazel joined us (in the womb) at the start of the PhD. Kids do not make things easier. But they do give you a reason to keep going. During dark periods of the PhD, she was always there to put a smile on my face (or add to the misery with sleepless nights. She’s a little monster like that). Hazel, if you read this when you are older, we love you and we will make sure you get to follow your dreams just as we could.

 

My parents, Marinette and Guy: they have always been there for me, supported me, and believed in me (and also provided me with all the nerdy hardware a kid needs to keep his technology interest going). And my grandparents, Meke, Vake, Peter Wieke, and Bobonne. Meke is not here to see this, but if someone believed I could pull this off, it was her.

 

My parents-in-law, Monique and Daniel, and the “Moekes”, for treating me like one of their own. Monique, the things you have missed out on, it is not fair. We miss you, words cannot describe.

 

Kurt, thanks for showing up at the defence (if you didn’t, this is going to be awkward). You’ve been that one true best friend. Always there in time of needs. And always making me look good at the board game table.

 

Franky, the bastard who pulled me out of the Flemish, secure mindset, and lured me to England. I ended up working long unpaid hours in Nottingham and lived amongst criminals and drunks. But I regret nothing!

 

José, for his unique perspectives on things, telling it like it is, and your attempt at keeping me sane through the PhD (it did not work).

 

Thomas for the babysitting and being Hazel’s coolest uncle. And “Tantan”, for taking care of Hazel all those Mondays, and for all the things you have done for Elke.

 

Kris, for bringing that new addiction into my life. The Nets won’t run themselves! Lies, for being Hazel’s awesome godmother. Jim, thanks for letting me win sometimes. Wait, no.

 

Sean, Jenna, Greg, and Johnny. One day Rad Lab Games will rise again!

 

François and Denis, we will make that dinner happen and bathe in Brasschaat’s sushi! Until then we will just shoot people online.

 

NorthgateArinso: Fred, Schtroumpf, Maarten, Karo, and Tom. My first and fourth job (thanks Samir), and also my last job before I ran off to academia (I am not implying anything!).

 

Everyone at Monumental Games and iChoosr, even though the stops were short, they were life changing.

 

My current, former, and visiting colleagues at the coolest lab of the Computer Science department: Yucheng, Karsten, Francisco, Gonzalo, Robin, Tom, Sam, Bruno, Victor, Gayane, Chen, Sten, Till, Frans, Samara, and Oana. And all the amazing people of the weSPOT, eCloud, and ABLE projects.

 

The Blade Runner soundtrack, for getting me through numerous paper deadlines.

 

And Bert. For getting me into this mess in the first place.

 

Dude, sucking at something is the first step towards being sorta good at something.

– Jake the Dog, Adventure Time

I love you, Pumpkin.

– Honey Bunny, Pulp Fiction

PhD in CS. Release Date: July!

Leave a comment

It has been a tough 4.5 years, but the end is finally near! Last Tuesday I had my private defence, or “preliminary defence” (or viva for the Brits, I hear). In Belgium, this part is pretty much the scariest moment of your doctoral career: it’s the proper, final, oral exam. Sometimes it is an enjoyable chat where they (4 professors) treat you as peers for the first time, sometimes it’s a soul-crushing, head-ache inducing, exhausting emotional rollercoaster. Mine jumped around between these two extremes, with some of this:

Still, the experience was a good one. They do a good effort in making you doubt your work, and I walked away feeling more confident and proud of what we have accomplished these long years.

The public defence, where I get to bore the audience with 45 minutes of presentation, is planned on 6 July 2017. If you are in any way interested in Learning Dashboards, Learning Analytics, or just like free drinks and snacks, I suggest you come watch! If interested, do shoot me a message for the details (it’ll be in Leuven, Belgium)! Official invites will be sent in a few weeks…

Meanwhile, enjoy the slides of the private defence: