alexandtheweb / blog

Mobile user testing: the rigs and the challenges

As of two weeks ago I was still a complete noob when it came to testing mobile products with real users. Figuring out the best testing approach within the time, budget, setting and product constraints of the project was… well, kind of fun, actually.

The aim of this post is to catalogue some of the mobile testing rigs I discovered or played with along the way. It’s also to make note of various considerations to the user testing protocol which I think are unique to the mobile setting. As is usually the case with these things, I probably didn’t learn as much as I could have, nor did I arrive at some ideal scenario: everything will depend on the product, target audience, device tail being tested.

The setup I finally ended up with (or had to settle on) worked reasonably well. The caveats particular to my situation were as follows: we needed to test the product on-site and we had no real budget for procuring a fancy kit.

Mobile user testing rigs:

I got to play with one of these at the HCID interaction lab a couple of weeks ago. This all-in-one mobile rig is meant to be paired with one of Tobii’s eye tracking units and is really rather nice and really rather expensive – it’s probably what you want if you have deep pockets and want to look like a very serious researcher.

This little webcam was recommended to me by Tracy, who documented its application in a usability study here . It’s also what the folks at MailChimp have been using. At a measly £40, this guy is definitely a budget-friendly option and has the advantage of being easily adjusted and repositioned. I ended up buying one – in a sexy black edition – but sadly found the video footage was not of the quality we were after.

Not a video capture system, but a lovely, modular rig which you can fit onto the back of the testing device and equip with a camera. Mr. Tappy bas the wonderful advantage of allowing users to pick up and carry the test device. I really hoped to get my hands on a Mr. Tappy (ahem) but alas, Nick ran out of stock and I ran out of time. I would have been curious to see if the weight of the rig and its angling over the device interfered with the testing. Hat tip to Harry for the recommendation.

These familiar staples of lecture theatres are the surprising solution for mobile user testing an and were suggested to me by Patrick from Makemedia. They suit the budget-conscious, starting at a few hundred pounds.

  • Copy stand fitted with an HD video camera

In the end our own home-grown solution was a copy stand mounted with an HD camera. Copy stands (also known as rostrum cameras or animation stands) such as this one are usually equipped with a standard screw-in camera mount and are adjustable in height. The video footage was captured and edited with Morae.

Certainly that’s not all that’s out there. I’ve seen people do some crazy things with Skype.

Physical and ergonomic considerations:

OK, a whole flurry of things may impact how your tests will pan out. Here is the handful I made note of as future learnings:

  • Will the quality of the testing be affected if the user isn’t able to freely move the target device or assume a familiar, comfortable position? My little testing adventure wasn’t particularly affected by this, but a few users did have trouble with legibility of smaller UI elements because of the artificial placement of the device. Also, some talented people do think this is a big deal
  • If the product being tested requires the user to pick up the device and move it in order to trigger some form of interaction (think gyroscopic stuff like shaking), is your filming rig set up to capture the breadth of this type of interaction?
  • Will the quality of the testing be affected if your users are unable to use their personal device or its equivalent?
  • If you’re asking users to use their personal device, can you supply an equivalent on the day? Can you provide an equivalent with appropriate device preferences such as screen brightness, orientation or text size display?
  • Consider left-handed users and large-handed users. Also, some people who are perfectly happy having their face recorded will get seriously self-conscious about having their hands filmed. True story.

Phew. Hope this is helpful. We ended up with some good insights and a very happy client so we couldn’t have fluffed up that much. But I still want a Mr. Tappy.