An Open Source Solution for Usability testing software

When looking for usability testing software at my new position, I naturally turned to the usability software companies that I know and have used in the past. However, I was faced with unique challenges that made finding the right software difficult. I was not testing web-based apps, so I was not looking to products such as UserZoom etc. I don't have a Mac at work so, I wasn't going to be using Silverback. I need to support two cameras as well as to record the desktop, so I could not use Morae. And finally, I was going to be testing on Linux which left me with, um..nothing.

But, hey, Linux is Open Source, right? So I was thinking that there must be an Open Source Usability Testing software out there. A quick search on Google and found links to people like me searching for open source usability software and people who were testing the usability of open source software, but no open source usability software to be found.

So, of course, I decided to code my own from scratch. Just kidding. I am a UXer, not a coder. I also consider myself what one of my professors referred to as a digital bricoleur. Bricolage "is the construction or creation of a work from a diverse range of things that happen to be available, or a work created by such a process. 1" - in other words, a tinkerer, a creator of works from "found objects". I use what is available to get the job done.

So as a good UXer, I start with requirements gathering. I needed:

1) Software that could handle multiple cameras so that I could record the (non-standard, proprietary) keyboard and touch motions as well as the user's expressions.
2) The ability to record the screen.
3) The ability to record audio.
4) The ability to add timestamps and events. (Task started, task ended, etc.)
5) The ability to allow others to observe the testing (nice to have)
6) Decent frame rates and video/audio recording quality.
7) The ability to access the test device from my computer (needed) and remote access (nice to have)
8) The ability to run the testing software under Linux.
9) Cheap or free.

It would also have been nice to have things like support for eye-tracking, mouse trails, visualizing clicks, and software that automatically creates pretty reports of my results like the big (and expensive) usability packages can do. That is rather overkill for me, however, I would be more than happy with the above requirements at the moment.

So, after much trial and error, I came up with the following solution: I am running Open Broadcaster Studio to record as many cameras as I please as well as the recording the desktop. I run the plugin Infowriter to record timestamps and have up to five hotkeys to record events. You can either just have Infowriter write the timestamps to a file, or you add a text field to OBS which is set to "chat log mode" and the source file you just created for Infowriter which will show the last six entries on the OBS screen which will also be recorded.

If you are working on Windows you can also record keyboard strokes with NohBoard on the test computer, hiding it off screen, and then choosing record window on OBS. NohBoard also lets you define you own keyboard which would have been awesome for people like me who are working with companies that use proprietary keyboards. Unfortunately there is no NohBoard version for Linux (yet?) so this was not an option for me.

Once I have my test machine set up and running OBS and I have OBS set up the way I want it, then I start a VNC connection to my laptop and I can retreat to my "control room" which is preferably not the same room, and then use the conference phone in the room to communicate with the user. Or, as is often the case when I can find only one room available, I am sitting in the same room with the user. Since OBS is primarily software for streaming podcasts and gaming, I can also stream the test to my dev team if I so choose.

This is a somewhat clunky solution and I think there is room for improvement in how to get everything set up. I suppose I should also use Jitsi or the like for communication with the test subject, but I don't see the need for that given the fact that all of our conference rooms have perfectly good conference phones and I am a bit afraid what it might do to the bandwidth.

The resulting recording are excellent in quality and meets the needs of non-academic, non-laboratory testing.

Since I don't work at Google or Microsoft, this works just fine for me. And you can't beat the price.