Recently you've seen how you can use Verify for some unorthodox purposes, like investigating watch-wearing trends or seeing whether people will switch to Verizon for the iPhone, or stay with AT&T. Today we thought we'd show you how you can use Verify for a more topical purpose: evaluating the usability difference in two competing services.
The Setup
For the test we wanted to see how usable Craigslist and eBay Classifieds were when a user needed to find a specific category. In this case we went with motorcycles, something we were reasonably sure would be represented using similar terms on each service, and that most people wouldn't be visiting on a frequent basis.
Using Verify we created a Linked Test, which is two or more tests strung together into one flow for test takers. In that linked test we created two click tests, one for each service asking users to click on where they'd go to view motorcycle listings.
The Analysis
Click tests obviously show you where people click - what they also do is show you how long it took the user to click. For this test we wanted to see not only an overall view of whether people found the category but also how quickly they did so.
To accomplish that we exported each tests' raw data to CSV format, which gave us the precise time it took to click. We could then use a simple spreadsheet tool (in our cases Google Docs) to average the time it took for each test.
The Results
On average, it took Craigslist users 31.5 seconds to click on the Motorcycles category. In contrast it took those same users on eBay Classifieds 14.8 seconds. That's with the clear outliers removed in favor of Craigslist. To find an uncommon category like Motorcycles, it takes users on eBay roughly half the time it takes users of Craigslist.
Also of interest was the actual click map. While time was important for this test, it's also useful to accompany that with the click data.
As you can see, testers looking at eBay Classifieds overwhelmingly clicked in the same place. While the incidence of clicking the correct category was also high on Craigslist there was a great deal more uncertainty - people indicated they would try and search and check under automotive services, as well as many who clearly didn't want to try and find it, so clicked anywhere else on the page.
The Conclusion
With about 80 respondents there's certainly a trend indicated by these results that eBay Classifieds does a better job of more usably displaying the breadth of its categories and services. This despite the fact that Craigslist is much more popular - we could assume people are more familiar with Craigslist, and still find eBay Classifieds easier and faster to use.
Would you consider using eBay Classifieds based on its usability? Or does Craigslist's dominance mean you'll head there, no matter how annoying it is to find something? Let us know in the comment!
