Unified UX/UI across tabletops & mobiles

Let’s get a bit more technical today. I’ve been talking a lot about the pedagogical side of things but have been neglecting the technical aspects. Since I’ve spent the last 10 years as a software engineer, not a teacher, a “tech” blogpost seems in order..

If you’ve read my previous post you know I’ve developed Navi, a badge dashboard, which is a small (on-going) Java project running on Google App Engine. This dashboard allows students and teacher to access class progression data through their mobile phone and desktop browsers.

We want to take this a step further and go beyond desktops and mobile devices by exploring the possibilities of tabletops and ambient screens in class rooms and create collaboration and reflection tools using the learning analytics data we possess (Step Up!). Our goal is to create a unified, always-present digital environment that spans across all these devices and delivers a homogeneous UX and UI through its unique look and feel.

The requirements

Visual Uniformity
While a uniform look might help brand identity, our main reason to create a visual uniformity is to create a similar UX across all platforms. Moving from one device to another should be a seamless experience and require little extra effort from the user.

The learning analytics data we have needs to be visualised. To engage students we will require something more than tables and forms, certainly when we consider deploying on tabletops and creating interesting ambient displays.

Cross Platform
Creating a unified experience across this big range of platforms is tricky. Using each native environment would be very time consuming. While multi-platform tools do exist, they would not allow for the flexibility we require. (we have a couple of students comparing such tools for their master thesis here and here)

Multi-touch Interactivity
Multi-touch devices such as tablets, tabletops and (specific) ambient displays allow for rich multi- and single-user experiences. Therefore we need a solution that will not limit us to simple single touch interaction.

The technology

While web applications have always been a good solution to go cross platform, it’s only since the arrival of HTML5 that web development has become really viable to create proper applications. With the new Canvas element and CSS3 we can create much more interesting visualisations and UIs and aren’t limited anymore to the basic objects provided by the HTML framework. Throw some Javascript into the mix and you have yourself a very potent, interactive, animated, vector-based canvas to work on that runs across any device (with a proper web browser anyway).

There are quite a few libraries for visualisations on the web. Examples are D3.js, Processing.js and Paper.js. As the UI will be an important part of my development, my preference went to Paper.js, a vector graphics framework supporting scene graphs, bezier curves, interactive animations and more. As it is more focused on pure vector drawing (instead of data visualisations focus of D3.js), it seemed more suited for the job. With my game development background, it also felt a bit more familiar ;)

To create the unified touch experience, Javascript and HTML5’s touch events are a great starting point. We immediately have support for mobile devices and can easily support tabletops using the npTuioClient browser plugin and MagicTouch.js script which allows TUIO connectivity.

For my backend I continue using Java, building on the current Navi project by exposing its data through a RESTful webservice.

The framework

Getting this technology to work is quite straightforward, but I need an abstraction layer so I can focus on the UX and visualisations without having to worry about the underlying libraries. Therefore I’m first developing a simple UI framework (currently “cleverly” named Navi Framework) around this technology that will let me easily create the UI elements and interactions.

The explanation of the actual design of this UI framework deserves its own blogpost and since we’re busy hosting the #lak13 conference in Leuven it will have to wait another week. But if you have any thoughts on the technology I am using or have any interesting suggestions, feel free to throw your comments my way!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.