In 2005, here at Bunnyfoot, we carried out an eye tracking usability study; it showed that 79% of people were able to find the 2003 UK gross domestic product using Google.
We carried out a similar eyetracking study in May 2009 using Bunnyfoot’s Mass User Testing approach and found that this had dropped to 37%.
We also compared the performance of Google to the new WolframAlpha search engine where 100% of people got the correct answer. This result is worrying for Google for two reasons:
Google’s algorithms have got better in the intervening years; despite there being significantly more pages indexed on Google in 2009 compared to 2005 Google returns fewer results for the same search string; “gross domestic product UK 2003”. Given more pages to return results from and better algorithms it ‘should’ be easier to find information, not harder.
The general level of people’s Internet experience and expertise has increased since the original study – people ‘should’ be more successful, not less.
WolframAlpha also outperforms Google on three key measures of usability; effectiveness and efficiency and satisfaction. However, the strength of the Google brand dominated WolframAlpha with 100% of users saying that they would recommend using Google to a friend with only 77% saying they would recommend WolframAlpha.
The study is by no means comprehensive; it is based on a single search query and one that favours WolframAlpha’s approach to knowledge management/search, but is does pose an interesting question:
Can Google’s search dominance be beaten by better results and usability or is the brand so strong that people will stay loyal no matter how good the competition gets?
We developed ‘mass user testing’ in response to the real world needs of commercial clients and to combat the deficiencies inherent in the most widely used traditional usability testing methods (we have actually been doing this for about 4 years but formalised it last year).
The key to mass user testing is using large numbers of people rapidly and cost effectively – this is achieved through recruiting people ‘off street’ with the lure of some cash (or other incentive – we are quite creative in this regard) for about 15 minutes of their time.
The BBC news item below shows a report on the UK’s first Internet enabled car – produced and invented by Bunnyfoot in 2000. The car was intended as a demonstration of the essential importance of usability and accessibility … our message got somehow lost in translation in the newspapers and TV shows that ran the story, but it taught us a lot about different communication methods and to always look to the future.
Since then (is it really 10 years ago?) we have produced hundreds of video demonstrations showing usability testing, eyetracking and accessibility in action, our customer experience presentations at seminars and conferences etc, and many will be appearing here in the next few months – but this BBC one was one of our first … and is still a firm favourite.
What is perhaps surprising is that this type of technology and other ‘alternative interfaces’ haven’t really come on that far in the last 10 years– it seemed then (in 2000) that things like sophisticated voice interfaces for all sorts of devices and uses were bubbling just under the surface. In 2009 though you are likely to be annoyed at best, but most probably bemused, by the majority of telephone interfaces (has anyone tried Egg’s?), never mind anything more ambitious. It seems like it should be simple but this type of interface requires just as much research and careful design (perhaps more) than seemingly more complex visual interfaces. I’ll return to discuss this in more detail in a future post.
The point of the Bunnymobile video?
It was meant to demonstrate that usability and accessibility are vital for the interfaces of the future:
the car used software that blind people use to translate web sites into voice = accessibility
and needed to be simple enough so distraction didn’t cause you to crash (amongst other things) = usability
It seems we were right, and they still are important … lots more challenging and interesting work to do though.
After many years of fudging responses to the common question “when are the new global accessibility guidelines coming out?” – I can finally give the the answer – it is today (11th December 2008) see the Press Release and WCAG 2.0 Introduction for more info.
The new standard is called the Web Content Accessibility Guidelines 2.0 (or WCAG 2.0), it replaces WCAG 1.0 – the previously recognised global standard which had been in place since 1997 (that’s eons in ‘Internet time’)
What does the new accessibility standard mean for you?
I am often asked how the eyetrackers work (second only to why the name Bunnyfoot?) – so here it is – in essence it is really simple – a digital camera videos your pupils (the holes that let light into your eye) and a computer works out where you are looking based on the video images.
Well there is a little bit more to it than that (not much though):
The infrared diodes shine light on the person in front of the eyetracker (it’s 14 times less strong than that emitted from a TV remote – so doesn’t burn their eyes out).
‘Sheffield?!’ I can almost hear you say,’ The grim Yorkshire city where they set The Full Monty?’. Yes the very same one and it’s come a long way since the coal mines and steel industry closed down for good; Sheffield has successfully reinvented itself a digital hub with a future the traditional industries could have never provided.
Sheffield has however not lost its love for Steel as can be seen by our new slide in reception ; )
How do you measure the effectiveness of your in-game ad investment?
Do you need to know accurate performance and brand engagement metrics?
It is not just about brand awareness or brand recall anymore, the new era of digital innovation provides us with an array of rich media to communicate with the increasingly cynical consumer. Games offer a huge untapped market with a broader profile than typically assumed. 59% of the UK population (26.5million) are gamers and 45% of those are women! Playing games is not just a nerdy boy thing anymore.