Real-Time Dashboards to Support eSports Spectating

This must have been my favorite project during my time at academia.

This article is also available on Medium.

The League of Legends and Counter-strike: Global Offensive dashboards in action! Read on to learn more.

My PhD was funded through multiple (European and smaller) projects, on topics such as Learning Analytics, Digital Humanities, and Unemployment. And they were interesting: I designed data visualizations on tabletops, desktop, and mobile, I researched the user’s perception and experience… all fun and games.

But no, not games.

I’ve always had a soft spot for games. I moved abroad for two years (and had my wife give up a great job) just so I could live the dream and be a game developer (a story for another time perhaps). And combining my research with games was something I had dreamed of for quite a while.

Alas, no funding to support this research. But life finds a way, right? ;)

Students to the Rescue

Students! Of course! Every year we had new Master in Engineering students working on both fun (and sometimes boring) Master thesis topics. So I introduced a new one: eSports dashboards to support spectators. The high paced action of different players across large battlefields is not easy to follow. So how could we help spectators get better insights in what was going on, real-time, during these hectic games played by professional gamers.

A huge success among students, but sadly enough spots were limited, and two students “won the lottery” (we have no say in the matter. It’s quite random): Hans Cauwenbergh and Bram Luycx.

And we got lucky. This turned out to be one of the best collaborations with students ever.

The Dashboards

Hans jumped on League of Legends, while Bram took on Counter-strike: Global Offensive. We designed and set up a couple of surveys on Reddit in order to get insights in spectators’ motivations and perceived usefulness for different statistics provided by the respective games to inform the design of the dashboards. And then, they were off!

Viewer preferences (167 responses for League of Legends, 596 for Counter-strike: Global Offensive)

League of Legends

a) Gold distribution, b) Damage dealt, c) Vulnerability.

The dashboard is the result of five iterations: two digital non-interactive designs, two interactive prototypes, and the final fully functioning prototype developed with React.js and D3.js. Iteration one to four were developed using the interface design tool Figma. 22 participants took part in the evaluations of the iterations of the prototypes.

The main view is split in three modules: Gold, Damage, and Vulnerability. All modules are continuously updated in real-time. Gold and Damage are quite obvious. Let me explain Vulnerability.

Vulnerability provides an indication of how vulnerable each player is. A simple linear equation combines the position disadvantage of the player to the rest of the team and/or enemies, current health and mana points, defensive statistics and summoner spells availability. These values are combined and visualized around the player icon as a summary indicator. This view lets spectators make predictions regarding the state of the game.

Hans also added an interesting Jungler visualisation:

Counter-strike: Global Offensive

a) Round progression, b) Economy button, c) Map button, d) Scoreboard, e) Money Distribution, f) Average Damage per Round

This dashboard is the result of three iterations: a paper prototype, a non-interactive digital prototype designed using Figma, and a fully functioning prototype developed with React.js and D3.js. 14 participants took part in the evaluations of the iterations. The dashboard consists of Round Progression, Economy, Map, Scoreboard, Money Distribution, and Average Damage per Round.

Results

Through semi-structured interviews and eye-tracking we managed to get further insights into the perception and experience of participants spectating these eSports games while using our dashboards. It resulted in an interesting list of design goals for spectator dashboards that we published and presented at CHI PLAY 2018 in Melbourne, Australia.

LoL and CS:GO eye tracking results from each participant after the evaluation. The attention heatmap indicates the areas of interest where participants focused more during the whole match.

I’ll refrain from turning this digestible blog post into a long scientific article. But if you are interested in the guidelines, you can find the article here. Or go through the slides below!

If there is enough interest, I’ll post a summary of the guidelines on Medium ;) Just let me know in the comments!

Hans (full-time) and I (freelance part-time) haven’t stopped here though and are working for GRID eSports Gmbh where a lot of eSports data magic is happening!

Next to that, I’m a Freelance Data Visualisation Experience Designer/Adviser. I’m for hire, so if you have any dataviz needs, send me a message!

Wait. Invoicing has language requirements?

Photo by Conor Luddy

This story is also available on Medium

As a freelancer, you kind of have to do and think of everything. My accountant is a great help, but it’s not always easy to ask the right questions. Certainly not when you’ve just made the jump into the world of freelancing (like me).

Invoicing scared me a little. The laws around it are quite strict, invoices MUST contain specific fields and words or they’re just not valid. Sure, if your customer still pays you, that’s great. But the tax man can issue fines if specific information is missing or incorrect.

As paranoid as I am, and wanting to make sure I didn’t make any huge mistakes during my first few months of freelancing, I checked and double checked all the requirements. And as I have both local and foreign customers, I ran into this annoying rule:

Your invoice must be written in the language of the area where your business resides.

However, for customers abroad, laws have recently changed, and English is acceptable (disclaimer: check your government website for the correct rules).

An example

When I am sending an invoice to Germany, I can use English. Great. But in Belgium (where I live and my business resides) we speak 3 languages: Dutch, French, and German. You’d be tempted to send an invoice in English to your customers in the French and German speaking parts, right? Well, no, to make sure your invoice is legal, it must be in Dutch. Sure, you can add a French copy, but the Dutch one is valid, the French isn’t.

Are you a British freelancer with a legal address in France
(or any other country, but do check local laws), doing business locally? . Make sure those invoices are in the local language, and not in English.

Invoicing apps

There are plenty of apps out there that take care of your invoicing. But with all these legal limitations, I decided I’d need a locally developed tool, as that would most likely guarantee the legal correctness of my invoices.

Teamleader looked promising, with lots of functionality, multi-language invoices, and even built-in support for correct taxing and phrasing (regarding VAT). But at 50 Euro per month (to get the invoicing, and a bunch of tools I would not use) I decided to look elsewhere (for now).

My accountant uses Yuki. It seems to be a pretty standard tool that everyone uses (in Belgium and Netherlands?), although I can’t say I’m a huge fan. Its visual and user experience design give me headaches (it’s been a while since I got vocally angry at an application). But it’s amazing feature-wise and does pretty much everything you need. Yep, also invoicing.

But what it generates… well. Your clients’ first reaction will be that an invoice from the 90s got lost and decades later reached their inbox.

My current love: Harvest

No feature bloat. Just time tracking and invoicing. Good looking invoices even! I first ignored Harvest due to its language limitations. Sure, it has a translation option that allows you to translate/rename every single label on your invoice, but there is no way to switch back and forth without needing to refill the complete Translations page on the Harvest website every time you need to switch languages.

But I found a way around that: this free Chrome plugin I threw together to solve my problems. Fill in your custom labels, save it. And repeat. As many versions of the labels as you like. It’s still not perfect. Between every invoice you send out you need to head back to the settings, load the labels you want, save, head back to invoices. But it beats having to type it all in again. And the biggest bonus: I can use Harvest for all my time tracking and invoicing needs <3

Yes, I’m aware other solutions exist. I sometimes just like to create things. The bonus of this plugin is that it’s not limited to specific languages, gives you more control, and different wording can even be used across languages.

It’s a temporary solution. I’ve already asked Harvest about this and they’ve jotted my feedback down. Fingers crossed! But in the mean time, if you’re a Harvest user, feel free to use the plugin.

Hope this information helps some of you out there. Do share your invoicing woes in the comments below! And .. happy invoicing!

LISSA: a student-facing Learning Analytics dashboard based on readily available higher education data

This article is based on the paper “Learning Analytics Dashboards to Support Adviser-Student Dialogue”, published in IEEE Transactions on Learning Technologies in June 2017. I presented this work recently at the Dataviz Belgium Spring Meetup 2019, Mechelen, Belgium. It was originally published on Medium, but deserves a spot on my own blog…

During my PhD I mainly focused on Learning Dashboards. Most of these the research on these dashboards focuses on teachers and researchers as users, but my goal was to empower the students, by giving them insights into their own process through data visualisation. I designed and evaluated a number of dashboards, but one stood out: LISSA, or “Learning dashboard for Insights and Support during Study Advice”. LISSA supports the conversation between study advisers and students, and has been deployed across campuses of KU Leuven.

We involved 17 study advisers in our study. These study advisers are responsible for both the study advice and content-related support for first-year students in a particular program. They are experts in both the content of the first-year courses, the current organisation of the program, and the regulation, both program-specific as university-wide.

The study advisers helped us get insights into how they work, and what their requirements were during these advising sessions. These sessions are private conversations with a student (occasionally with parents) taking place in an office environment. These students typically do not have a flawless study career: they have trouble studying, would benefit from a personalised program plan, did not achieve enough credits through the year, or simply wish to re-orientate towards a new program.

The study advisers have multiple tools and websites to their disposal. But combining and interpreting these multiple channels of information for each specific student requires effort and time, and is error-prone. In addition, data is often incomplete.

The dashboard

Using a user-centred, rapid-prototyping design approach, we started of with Sketch for our initial designs and D3.js / Meteor to create our final interactive prototype. Here are two screenshots of the final design (in the meantime current PhD students redesigned the interface):

Year Overview

LISSA provides an overview of every key moment in chronological order up until the period in which the advising sessions are held: the grades of the positioning test (a type of entry-exam without consequence), mid-term tests, January exams, and June exams. A general trend of performance is visualised at the top: the student path consists of histograms showing the position of the student among their peers per key moment.

Every course is represented by its name and grade (out of 20). A green, orange, and red colour coding represents successful exams, tolerable grades (students can request to pass a limited number of 8–9/20 grades) and failed courses. The course is accompanied by a histogram visualising the performance of peers and the position of the student among them (black highlight).

Planning

For June sessions, it is important to plan the re-sits during September. Too few exams result in a credit threshold issue, while too many will most likely result in failure. The check-boxes next to the failed exams re-sit planning let adviser and student select several courses. The “re-sit exam success rate graph” uses historical data to provide insights into the number of students succeeding the selected number of exams in the past.

Prediction

The stacked prediction bar provides historical data of students with a similar profile (based on the number of exam passed or failed) to the student: it shows the distribution of the duration of the bachelor program (three-four-five years or drop-out/ “NIET”) with similar September re-sits.

Data Sources

To visualise the key moments, data regarding student grades is required. This includes all first-year students of the current year to populate the courses and course histograms, the student path, and course histograms. All grades regarding the January, June and September periods are available in the KU Leuven data warehouse.

The stacked prediction bar is based on the first-year student grades of previous years. This provides the data needed to predict the three, four, five, or more years length of a Bachelor degree.

We created a data process pipeline using Python scripts to convert the different files and formats into a simple representation that is imported into a MongoDB.

Lessons Learnt

The role of the Learning Analytics data

LA dashboards are often developed for specific institutions with certain data requirements. The Learning Analytics data necessary to deploy LISSA is very basic: grades of students across key moments and data regarding student success (derived from historical grade data). This data is usually available in most higher education institutions, but limited to staff. Yet, we have shown that this data placed in a student advising context, can help support students, provide insights into their progress and help plan their future.

LISSA is based on factual data. Exam success rate and bachelor duration show what has happened historically as facts and provide no calculated estimations. This reliable way of visualizing the data provides reassurance among both study advisers and students about the advice they are giving and receiving.

Personal background data regarding socio-economic status, parents’ education, gender, and high school achievement can provide further insights and help the study adviser understand the student’s situation better. However, this unalterable data does not provide the students with actionable insights. It is therefore important to investigate how to integrate this data in an ethical manner.

The role of the Study Adviser

LISSA facilitates insights at multiple levels, but these insights benefit from guidance by the study adviser. Even though the data is objective, there is still a need for critical and reflective interpretation by domain experts . Overconfident students might interpret an overall negative result as a surmountable problem, whereas the study adviser could advice and plan a more achievable program, preventing the student from wasting years on incorrect choices. LISSA can portray a student in a negative way, while a discussion with the student might reveal problems that are easily resolved, e.g. a change in study method, a new program, or a change in attitude. Without the study adviser’s guidance, such students might choose not to continue their Bachelor program.

LISSA still leaves room for personal opinions and tacit experience, as they still play an important role during advising sessions by allowing study advisers to e.g. emphasise certain results to push them on the correct path. Many external factors, such as information gathered through discussion and previous study adviser experiences with students, impact the decision to deviate from the factual data or interpret it differently.

Transparency

During the semi-structured interviews and workshops, ethical issues arose regarding confronting the students with the data.

Some study advisers did not show LISSA to students with a very high number of failed courses. While some students might benefit from an “eye-opener”, study advisers prefer to use LISSA as a motivational tool.

An important role of LISSA is the ability to position a student among peers. In general, the use of histograms was considered very useful and positioning had positive effects such as motivating the student when a bad grade is still good among peers, or a course failure rate is very high. Some study advisers worried about demotivation: a student on the low end of the histogram might see succeeding the course as an unachievable goal.

There is however a demand by students to gain access to LISSA outside the sessions. But study advisers fear that the data visualised can be greatly misinterpreted without their guidance: students with fear of failure or over-confident students might interpret the data incorrectly. Parents can play a negative role into either pushing too hard, or interpreting mediocre results as insurmountable. The lack of knowledge about higher education with parents without a degree might stop a student pursuing an achievable degree. These problems might result in wrong decisions regarding exam and study career planning. Study advisers do see potential in providing reduced information, but what this reduction entails must be further discussed and evaluated.

For more details regarding the process, evaluation and findings, check out the original paper. Need help visualising your data? Send me an email, I’m for hire!

Why Deepfakes are a good thing

[Also on Medium]

If you should believe the media, Deepfakes equals the end of days. Fake news, fake evidence, this technology will create a world of disbelief, blackmail,… trust goes out the window, we’re all doomed. Period.

The media likes to scare us, as, well, scary stories sell. And for once, it might actually help get the positive message across.There’s a way to use Deepfakes for good, and I’m not just talking about my wife on the Tonight show.

We believe everything

Whether it’s the news, a magazine, a book, gossip, it’s all just words, spread around, by friends or “reliable” sources. And we (should) all know to take everything we hear and read with a grain of salt. Yet, we let them lead our thoughts and our beliefs, which influence important life choices such as who we vote for, who we love, who we hate. Words are the most easily manipulated form of message (why not buy a book on how to manipulate people). And still, millions of people live by words that cannot be proven to be true, on a daily basis, and would even give their lives for them.

Roswell’s Flying Saucer 1947

We used to consider photographs real. Then Photoshop happened.

Trump & Saruman

Video is real. Even though we know 90% of what we see in Hollywood movies is fake, every other video is real right?

We must become more aware

Fake news is a big topic. While we know in our hearts never to trust politicians and news outlets, we still do, we still believe every word they say but go crazy when we finally find out it’s all lies. We don’ blame ourselves, though.

Deepfakes just added another level of worries. Video can be faked?.. Is anything we see and hear real anymore?

No. Nothing is. Just as you shouldn’t believe every single word on tv, in the newspaper or [insert your favorite religious book here], video is just another medium ready to be manipulated.

But don’t blame Deepfakes. Don’t demonise the technology. Thank Deepfakes!

Think about this: If someone was able to create this technology in their bedroom throwing together a bunch of existing tools, someone with a bigger budget must have pulled it off a long time ago. There’s no doubt large organisations with massive resources haven’t explored these techniques. Who knows, maybe we’ve seen some of their work, on the news, without knowing it.

So thank Deepfakes, for making us aware of this, making us realise once again that we can’t take everything we see and hear for granted. For creating a problem for us to solve, early on, before it becomes so big, and has influenced so many of us incorrectly, that it’s too late.

It will take time. A new skill we must all learn. So doubt that next video you see on the Internet. Hell, doubt everything you see, read, or hear. Be more critical! Think for yourself.

Family fun with deepfakes. Or how I got my wife onto the Tonight Show

[Also on Medium]

[ Update 3 Feb 2018: added two new creations at the bottom of this post. Last one turned out really well ]

I’ve first heard of deepfakes a good week ago. Thanks Twitter. Thanks Tim Soret.

Yes, it’s pretty damn cyberpunk. But from a superficial point of view, /r/deepfakes (extremely NSFW! You have been warned) consists of people using an app created by user “deepfakes” to create fake celebrity porn.

This has caused a shitstorm on the Internet, media discussing the legality of it all, websites taking down the deepfake creations, and people panicking as they realise AI is going to screw us all up (newsflash: it’s already been happening in much less obvious ways). And meanwhile, Nicolas Cage is taking over Hollywood.

While everyone’s debating whether this is good or bad, I just had to find out more. First thing that came into my mind? How can I apply this to everyone I know (in a non-porn way, in case you wondered).

How does it work?

The deepfakes app is a deep learning algorithm that learns how to reconstruct faces. Give it a bunch of pictures, let it run a few hours, and it spits out fuzzy copies of those images. Do note, it doesn’t create a copy. It learns what a face looks like, in different expressions, and is able to output that face solely based on that. There’s a detailed explanation on Reddit but let me try and dumb it down.

Think of it like this: imagine if you could stare at someone for 12 hours straight, observing all their expressions and absorbing that into your brain. Then that person asks you to sketch his face on paper, smiling, crying, any expression you’ve observed. What do you do? You immediately generate a photographic quality sketch on paper, from the mind! (using a pencil)

It’s insane!

While that’s pretty cool, it only gets better. See that “encoder” part? The FakeApp uses one encoder for all faces. It’s the decoder that’s kept face specific. And here comes the really cool part. Let it learn two faces, and things become more interesting.

Right, now see how this works. The encoder takes an image of a face, let’s it run through its “brain”, and the decoder spits it out again. In the example above, it can do so with faces of Anne Hathaway, and Elke, my wife. Ok, so far so good. But now let’s feed it a picture of Anne, but use the decoder that generates Elke’s face!

You just created a new photo of Elke. A photo that never existed, in the same angle, the same expression, as Anne! Amazing!

Family fun

Sure, putting celebrities’ faces on your favorite porn stars is an interesting use case. But we can leverage these celebrities for other things, such as inserting your friends and family into blockbuster movies and shows!

For the best result, you must first find an actor/actress that has a similar head shape as the person you wish to insert. In case of Elke (my wife) I accidentally noticed, while watching the Dark Knight Rises, that Anne Hathaway might be a good fit. I guess you know Anne, so here’s Elke:

All I needed was about 850 photos of Elke, a few 1000 of Anne, a lot of computing time, et voila: Elke’s on the Tonight Show starring Jimmy Fallon.

Bonus effect: now we know what Elke looks like with short hair :D

Here’s a little comparison gif:

I personally think it’s fun, can be innocent, and even makes for a nice surprise/gift. Remember, any tool can be used for evil. And as long as we’re not banning guns, this should not be a high priority, amirite?

There’s so much you can do with this technology. You know those dumb emails people send around where they replaced dancing elves heads with their own, or even worse, yours? Well, now you can put your best friend into his favourite movie: have her dance with Patrick Swayze and have the time of her life, or have an alien burst out of his stomach. It’s all within your reach now!

Beyond just pure fun, I can only imagine how people will start turning this tech into business ideas. Fashion will be huge (what would I look like with this kind of hair, this kind of dress…), fitness could be interesting (do I look good with muscles, will I really look better skinny), travel (this is you standing on a beach is going to be quite convincing). It’ll bring advertising to a whole new level. No need to imagine what if, they’ll tell you what your “better” life will look like! And it’ll be hard to get that picture out of your head…

Update: in the mean time, I’ve created two more. Elke’s a huge fan of Steve Carell, and I suddenly realized Anne Hathaway co-starts with him in Get Smart. First attempt was okay:

Then I wanted to try this one (original video):

And I think it turned out great: