BunnyfootBunnyfootBunnyfootBunnyfoot
  • Our services
  • About us
  • UX training
  • Our clients
  • Blog
  • Contact us
    Home Content: White Papers, Research Reports and More Effective mobile usability testing – the rig matters

    Effective mobile usability testing – the rig matters

    By Eleanor Holmes | Content: White Papers, Research Reports and More | 0 comment | 14 April, 2011 | 0

    Technology’s moving fast, isn’t it?  Too fast for natural and effective mobile usability testing solutions to be on the market for the increase in apps and mobile sites we are seeing. Taking inspiration as we go, and getting as close to natural behaviour as we can, we are building the Dream Mobile Testing Rig. This blog is about that journey and will help you to understand the pros and cons of what is out there when choosing mobile testing solutions.

    There are many types of mobile usability testing rig, how do you choose?

    When you are comparing mobile usability testing solutions, here are a few things to look for to achieve the best results:

    • Is it natural? Can it be held in one hand, for long times, without an obscured view, etc
    • Is it adaptable? Does it allow for different mobile types – Smartphones, Blackberries, Keypads, Sliding, etc
    • Is it personal? Can you use the participant’s own technology so the handset usability doesn’t interfere with results?
    • Is it the full experience? Are you recording screen, face, fingers and comments to get a complete picture?
    • Is it viewable? Can it be recorded and even watched live?

    Our first iteration: “Look, but don’t touch”

    Verdict 1/10: Quick and simple to test the initial stages of layout and language, but not good beyond this stage.

    When mobile testing started, we used an emulator on screen to track designs just as we would a website. This is cheap, simple and effective for eye tracking and giving proof of information architecture and labelling. It can be recorded, viewed remotely and even eye tracked. However, it just isn’t natural. You’re not getting an idea of how the handset itself will affect the usability and not capturing the experience of trying to complete the task on a small screen with fingers and thumbs.

    Second iteration: Getting heavy

    Final score: 4/10: Lighter materials and more grip on the clip and this setup would be a winner.

    For our first truly mobile rig, we clipped a soft-clip poseable camera to the top of any phone, focused on the screen. This allowed people to use their own phones and hold them naturally as they move around.  The weight of the clipped rig was the only issue – making it heavy to hold the phone and would make the whole rig shift occasionally during testing, losing sight of what is happening.  Participants can be perturbed by you attaching a big, heavy clip to their precious, delicate technology!

    Third Iteration: The glasses cam

    Final Score: 7/10 Participants loved it, but caused extreme motion sickness amongst viewers.

    We had a project come in for the iPad, which forced us to rethink the solution. We went for a camera attached to a pair of glasses, which meant we could track exactly where the participant was looking on the screen and what they touched plus a fixed camera on the desk to record their facial expressions. It worked really well for the participant, but left viewers feeling motion sick.

    Fourth iteration: Our most recent baby

    Final score: 8/10 Well adopted and natural to hold, but could still be lighter and should remove tray sides.

    We took the clip set up from our first design, knocked some weight off, added some stability and attached it to a plastic tray that the phone just sits in to avoid slip. We added a second camera to the face and an audio clip to capture reactions.  The cameras fed into  a picture-in-picture control box and converted to 1024×768 VGA, big enough to view remotely on screen and be recorded. The tray accommodated all types of modern phones and helped the participants adopt it nicely, but the tray sides stop people holding the phone as they normally would.

    Fourth Iteration: The Dream Rig

    We’re planning to take the sides off the tray to make it more of a smooth sledge with little interruption to the natural feel of the phone. We’re happy with a cameras and an audio clip, but we need to make the device arm as lighter, so we’re constantly sourcing and testing new equipment. The facial expressions camera being on the table captures more, let’s them move more and captures both body and face, so that’s here to stay. Making the connection wireless would mean we could test in the field, allowing people to use their phones as naturally as possible in all situations.

    Of course, a design that was virtually invisible would be the dream – but this is probably a long way off…

    The options we dismissed and why


    Screen Recording Applications: These you just download onto a phone to record the interaction

    There are actually a few screen recording applications out there but they all have the same disadvantages. There is no app which will support all the different mobile platforms and devices. The results they deliver aren’t great; the video files do often have lagging issues and just a few mobile screen recorders actually record the touch points on smart phone screen.

    Generally, people were upset enough about us clipping a camera onto their mobile, I can’t imagine they would react kindly to us wanting to download software onto their phone.


    A static document camera: Fixed on the table, records interaction from above

    This is lovely for getting a clear view of the interaction, but in that case the phone must be held at a flat angle or lay on a desk and the participants must keep within the camera range, which is not a natural way for people to interact with their mobile. Take a look at the image of how Google have been testing on their mobile – the document camera is quite intrusive when you have to lean over to see the screen as it is.

    We’re aiming to bring as natural a feel as possible to testing, so we are always aiming to make solutions that allow us to test mobile websites or mock-ups in the urban field, their natural habitat. So, in the future, static cameras are also not our weapon of choice.

    Further reading:

    This presentation includes all the testing rigs that are out there – it’s more for people looking to build their own rig, but can give you some good insight as someone looking to test about what to look for in your mobile usability solution.

    http://www.slideshare.net/beleniq/do-it-yourself-mobile-usabilitytesting-at-euroia-2010

    Here’s some really nice, lightweight looking trays that can be made for a fiver. They’re still not perfect, we prefer the poseable camera, but they look smart and are a great way of cheaply making your own.

    http://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
    http://www.90percentofeverything.com/2010/11/15/more-mobile-usability-testing-sleds/

    Methodology, Mobile, Usability testing

    Eleanor Holmes

    More posts by Eleanor Holmes

    Related Posts

    • Bunnyfoot airport research

      Nov 27, 2015

      As part of our Internet of Things (IoT) consultancy service we recently researched customer experiences at airports, specifically the departure journey. In the IoT you are ultimately trying to design service solutions, and in order to do so you must... Read more

    • User Testing is Key to Understanding People

      May 12, 2014

      In 2012 we carried out research investigating the effectiveness of Google AdWords. Our previous research indicated that 41% of people tested didn’t know that Google AdWords were ads. This year Google AdWords rolled out a new design which we tested, and... Read more

    • Misleading Ads Mean Huge Budget Wastage for Leading Brands

      Jul 7, 2004

      Many leading brand advertisers could be wasting up to 90% of their online advertising budget - and causing mistrust and frustration amongst users - because of misleading adverts that make the user click-through by accident, according to new research from... Read more

    Leave a Comment

    Cancel reply

    Your email address will not be published.

    NextPrevious

    Recent Posts

    • ‘AR’-commerce – a novel and functional shopping experience January 26, 2021
    • Webinar: UX & Sustainable Service Design January 12, 2021
    • AR isn’t mainstream… or is it? January 7, 2021
    • What makes an effective UX Consultant? Bunnyfoot speaks to UX Usability Podcast December 11, 2020
    • What’s in a name? User Experience and UX Design vs. Design Thinking November 24, 2020

    Categories

    • Brain Bites: 2 Minute Insights
    • Brain Feasts: Longer Reads
    • Brain Licks: 10 Second Wisdom
    • Bunnyfoot Updates
    • Content: White Papers, Research Reports and More
    • Events
    • News and Announcements
    • Talks and Presentations
    • Tools and Resources
    • Uncategorised

    LONDON

    4th Floor
    54 St John’s Square
    Farringdon
    London
    EC1V 4JL

    Tel: 0207 608 1670

    SHEFFIELD

    The Old Chapel
    8 Mortimer Street
    Sheffield
    South Yorkshire
    S1 4SF

    Tel: 0114 478 2950

    OXFORD

    Boston House
    134 Downsview Rd
    Wantage
    Oxfordshire
    OX12 9FF

    Tel: 01235 606 140

    Cyber Essentials (PLUS) Logo
    • Copyright Statement
    • Terms & Conditions
    • Training Course Terms & Conditions
    • Privacy Policy
    • Jobs
    © Bunnyfoot Ltd 2020 | Bunnyfoot is part of the Sideshow Group
    • Our services
    • About us
    • UX training
    • Our clients
    • Blog
    • Contact us
    Bunnyfoot
    We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
    Privacy PolicyREJECTACCEPT
    Privacy & Cookies Policy

    Privacy Overview

    This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are as essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
    Necessary
    Always Enabled

    Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.

    Non-necessary

    Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.