Reflection with Interactive Tabletops: an Evaluation


Last Tuesday Jose and I had the opportunity to experiment on a couple of students during a thesis poster presentation taking place just before the last #chikul13 class. While Jose was focussing on making students more aware through ambient displays, I did my first test-run of the Navi Tabletop Application. As you may remember, we’ve been awarding students with activity and progress badges. This app visualises these achievements and gives students a collaborative tool to reflect upon their work.


The rapid prototyping with my custom HTML5/CSS3/Javascript framework works! I managed to get a fully functional tabletop prototype with actual badge-student relationship visualisations working in a weekend. Here’s a quick before-after screenshot comparison.


(please ignore the English in the screenshots, I have no control over the badge descriptions sent back to me by the data repository ;))

The bezier curves are courtesy of Paper.js. While I moved away from using a canvas element to draw my interface, it’s great as an extra layer to draw more demanding objects, like the relationship lines.

As I explained before, I want to create a tool to support collaborative reflection between students and teacher. The teacher would then guide this process. However during my testing on Tuesday I decided to let them freely explore the app by themselves and see what happens.

Navi Tabletop App

Let’s start with a quick explanation of how the app works.

navi_v2_step1 navi_v2_step2

Badges can be dragged onto the playfield (the top 70% of the screen). Students who have been awarded this specific badge will light up (in blue) in the student container. The opposite is also true: dragging a student onto the playfield will light up the badges in the badge container that this specific student has earned.

navi_v2_step3 navi_v2_step4

The highlighted items together show the union of the relationships with the items in the playfield. Touching an item in the playfield will show its relationship to other items in the playfield through connected lines. This will also limit the highlighted items (students or badges) to the ones related to the touched object.

Badges are awarded every 2 weeks. The user can cycle through these time periods in the badge container.

Test Chamber 1

I decided to use the think-aloud protocol, give no scenario and see whether the students could make sense of the tool and data by themselves. I filmed the whole process using a camera on a tripod and asked them to fill in a survey afterwards. For students who weren’t part of the #chikul13 course I gave a bit of background information on the badge system.

During the one hour poster presentation session, 14 students tested the application. 2 groups of 3 people and 2 groups of 2 people used the table simultaneously. The remaining 4 people used the application by themselves.

Almost everyone figured out how to use the app without guidance. Users started dragging items onto the playfield and started seeing and understanding the relationships between names and badges.

The most interesting events occurred when students of the #chikul13 course grouped up with their actual #chikul13 group members (they work in groups of 3 during the course): At one point they became interested in seeing their group badges per period. They dragged out their names and started cycling through the time periods. When noticing a low amount of awarded badges, they started reflecting about that period and found reasons for the lack of badges. They also saw which periods what person was more active and had an awkward but funny moment when 2 of them had a negative badge for not commenting at all during a 2 week period.


That group I just mentioned was a perfect example of our ideal target audience. It was interesting to see them behave the way they did: discussing and reflecting over their progress in the #chikul13 course even though the amount of data they were given was limited.

As for people using the tabletop app by themselves, results were less interesting. They were struggling to continue exploring and needed a little push now and then to have ideas on what to do. But as I am aiming for a collaborative tool, this isn’t a big issue.

The survey showed similar results. Students would like to use the tool with others, however there was more interest in using the tabletop with other students than with a teacher. The word “fun” was used a few times when asked if the collaboration added anything to the experience. They also admitted that using the tool together invites for more communication and quicker understanding of the data. Almost everyone agreed that more information would certainly improve the application as the actual comments, posts and tweets (on which the awards are based) aren’t yet available through the tabletop.

It was clear that the UI wasn’t flawless. People tried double tapping and wished the lines would remain visible when letting go of objects. Due to some issues with the time period selection and a lack of a time period spanning the entire course timeframe, people weren’t convinced the tabletop app helped them get a good total overview of the class’s progression

The survey also contained a SUS questionnaire. This resulted in a SUS score of 71 counting all students, 76 counting only the #chikul13 students. An above average score which is not too bad for a feature-limited prototype with a very specific purpose.

What’s Next

I am pretty happy with these results and it was exciting to see the students play around with the Navi Tabletop App. But there is a lot of room for improvement.

Period selection

First up is the time period issue. While I could fix the button’s responsiveness and provide the user with better visual feedback, I believe cycling through the periods is not necessary at this level of the interface. I will remove it and bring it back at badge level (including a default full course timeframe selection).


Items light up when related items are dragged onto the playfield, but this wasn’t obvious enough to some users. A more visually striking highlighting (e.g. animation) that catches users’ attention might solve this problem.

Double Tap

Quite some users attempted double tapping, something not very common on touch screens. I believe the lack of actions available on items might be the cause of this double tapping reaction. Some users expected the lines connecting related items to remain visible while the “correct” action was holding a finger down on the item. Adding action buttons to items in the playfield to support these kind of features could indirectly solve this behaviour.

Badge Information

People had to drag the badge all the way onto the playfield to see its description.  A simple touch should already show that information.

Lack of Information

The big next step is showing the actual blog/comment/tweet data. Most people agreed this would make the application more useful. Once Jose’s StepUp! backend can provide me that information, I’ll be adding that functionality to the tabletop. This will hopefully bring the reflection process to a whole other level.

Another Time Perhaps

Great, you got this far! I bet you were hoping on a video and… there isn’t one. But fear not, one will be posted soon! Screenshots and text can only say so much right?

Feedback, comments and ideas more than welcome as always, so feel free to fill that comment box below!

Reflection in Learning: Tabletops & Beyond

Last Wednesday our HCI group had the opportunity to meet with the folks of CUO, the Centre for User Experience Research, at the Social Science department of our university for a little brainstorm session. I was asked to present a bit of my work with which I threw a few ideas into the group and which provided me with some nice feedback. The slides of my presentation are available here, but they are pretty visual so won’t make much sense without me yapping along to them. The first half is much in line with what I’ve posted here and here though. As for the second half, let me give you an update.


We want to promote reflection and think our badge data is a good starting point. The badge data is an abstract, high level representation of the activity and achievements and can be a central access point from which we can further drill down into the raw data. While a student can figure things out by himself, input and guidance from a teacher through this reflection process can greatly improve the result. Putting students and teacher around a table for discussion seems like a good idea, so why not use a digital tabletop?

Tabletop Reflection

Let me explain how the first few steps of this process would work:

We put the teacher and a couple of students around the tabletop. The teacher drags out the names of the present students onto the tabletop. These expand and become the students’ personal consoles.

The teacher then pulls up the badges. As the tabletop app now knows which students are standing around the tabletop, it can adjust its badge visualisation and limit it to data relevant to these students. Dragging a badge onto the tabletop will expand the badge, provide more options such as showing its relation to the students and the class and provide a drilldown point to more information, in our case: tweets, blogs, comments, etc.

This provides the teacher and students with the basic elements required to facilitate discussion and reflection. The teacher has an immediate overview of the progress and the data of the students. He/she can give feedback, discuss the artefacts and guide the students in their learning process with the data available on the tabletop. The students can study and discuss the artefacts and compare their own work to that of their peers.

A couple of screenshots of the rough tabletop prototype:

Fullscreen_07_05_13_11_52 Fullscreen_07_05_13_11_51-4


Let’s do a quick tech intermezzo, before we lose half of the audience ;) While the goal is to create a tabletop reflection tool, we also want the technology to support multiple platforms. I walked away from the pure Paper.js idea I mentioned previously though and have now created a custom Javascript framework to suit my specific needs. The framework logic is built on a mix of Object-Oriented with Component-based design, the visual design is all HTML5 and CSS3. This allows me to quickly create classes that support multi-touch, animations (CSS3 and custom) and event triggers and can easily be hooked up to the DOM elements. No rocket science, just very helpful for rapid prototyping ;) As an example, the above prototype was developed in a couple of days.


The features required in our process aren’t all available yet in our prototype but will be developed soon. The next step (after some evaluations) will probably be about how we can expand this to collaboration. We want the students to collaborate around the tabletop but also collaborate with people out in the field. Imagine students outside or even out of the country being guided by the tabletop users and in turn providing the class with realtime data from these locations. Google Glass as a direct stream to what’s happening there? Why not!

We think there is potential in not just taking learning outside and the outside into the classroom, but also bringing these learning artefacts and analytics into the real world. We have more interesting ideas on this and how to involve the parents into the process, but more on that later. If I tell you everything now there’s no reason to come back right? ;)

Open Badges & Motivation

Last time I mentioned how the students’ online presence helps both the students and the teachers keep track of their progress. By having them use Twitter, blogs and comments to report on their progress, we create an open system that helps awareness of the class activity for both teachers and students.

Thanks to Jose Luis Santos‘ trackers, we have detailed statistics on every student, showing us the amount of blog posts, tweets, comments but also the interaction between students and people outside the course. Indirectly, we can get an estimation of the quality of e.g. a blog post through the amount of comments it has received from others. We can also quickly discover inactive students.

Apart from just raising awareness, we also want to promote activity, quality and results. We try to do this through gamifying the progress of the course. This gamification concept comes from videogames where certain milestones and actions are rewarded and shared publicly with all your friends. These rewards are more commonly known as achievements.

In the learning environment we have set up, we can visualize the progress of our students through graphs and raw data through applications such as StepUp! (Jose Luis Santos) and the Activity Stream (Gonzalo Parra) but we can also limit this visualisation to the important milestones and actions reached during the course. And for this, we use badges (following the Open Badges standards).


Badges give the students an immediate overview of their achievements and allow them to compare these accomplishments to their peers. Through a personal badge dashboard (see image) they know at a glance the state of the class’s progression. It quickly becomes clear which achievements are easy to get, what the student is lagging behind on compared to the rest of the class and what work remains. This information should again raise awareness.

But there is also an immediate feedback which would otherwise be difficult to give outside of class. When a student shows a lot of activity on e.g Twitter, a badge is instantly awarded. Other badges are given after a 2-week period: No activity is not acceptable so a negative badge is awarded; a lot of comments on your blog post means it has provoked a lot of discussion and can be assumed qualitative, which deserves a positive badge. Deliverables are also awarded with badges. A good evaluation on their developed prototypes also deserves a badge.

With this feedback through rewards we hope to achieve a change in bad (inactive) behaviour, promote positive behaviour like activity and quality, and improve awareness in and outside of class. We also hope the dashboard will help the students reflect on their activities. By being aware of their peers’ progress and being rewarded (or not), they can conclude whether or not they are on the right track.

The exact effect on motivation, awareness and self-reflection is still under evaluation. But while we wait for further results, feel free to head over to Navi, the badge dashboard used for #chikul13 and share with us your thoughts!

“Big things have small beginnings ..”

– David, Prometheus –

A new blog, a new job! 10 years after graduating from KU Leuven, I was lucky enough to get the opportunity to head back to university and work as a doctoral researcher for Prof. Erik Duval at the Human-Computer Interaction Group at KU Leuven.

It’s quite a different world from what I’m used to. I’d like to say things are more vague but that’s just another wording for “having lots of freedom”. You have plenty. And that means your creativity plays a very important role. It takes some getting used to and for now I (think I) am coping just fine… I have a vague idea (there it is again, vague) of what I’ll be doing… For now at least ;)

Working as doctoral researcher means I have a PhD to finish within the next few years. The subject of this PhD is Learning Analytics & Dashboards. One way to explain it would be:

We try to figure out what the important traces are that learners leave behind, how we can visualize those and how we can create reflection, awareness and collaboration tools to improve their learning. [Duval]

In short, we want to improve the learning process. And while money rewards and candy do work, bribing is not our main focus ;)

That’s still vague (again!) for an outsider but let my overactive imagination explain what I would like to see in the following 10 years: Augmented Students ..

Through the use of devices such as ambient displays, table tops, mobile and wearable devices, enhancing the students’ learning process by creating a context-aware, always-on digital presence that enables easy collaboration, total awareness of their peers’ progress and instant feedback on their learning activities resulting in constant self-reflection.

Sounds a bit sci-fi? You probably guessed the source of my inspiration then ;) But technology is advancing so fast and with devices such as Google Glass and Leap Motion ideas like these aren’t too far fetched. But we have to start small. And I’ll get more into detail on our current research in a later post.

Apart from a PhD student, I’m also a teacher assistant for a user interface course called CHI (yes, it’s totally related to the #chikul13 updates on my Twitter). We have 26 students (mostly engineering students in their 4th year) who are currently developing a recommendation app. I’ll be sure to post the links to their blogs, as they are really worth checking out and we do hope to see some external input on their work.

So here we are. Taking part in an attempt to revolutionise the way we, our kids and grandkids will learn.. This sure is going to be an interesting ride. I hope you’ll tag along!