It's Official: Captchas Are Bad for Business by Mark
August 07, 2009 in Business and Startups with 23 Comments
Nearly a year ago, we told you captchas had seemed to become a necessary evil for websites. Between the spam and the abuse, web forms just aren't secure, and it felt like the only way to combat it was with those nasty captchas.
Captchas have always felt like a cop-out. As a business, you're passing your own business problem onto users. Moreover, the way in which the problem is passed onto users is half-assed and can cause loads of confusion.
Fundamentally, that's just wrong.
About the Study
I hate to say I told you so—wait, no I don't—but, according to a recent study by SEOmoz, captchas could be costing you conversions. Here's how their case study broke down:
- It was performed across 50 websites, each ranging from one to five years old.
- The study was done over six months, splitting it evenly with captchas on for three and off for the other three.
- Each form was used to collect common contact information (name, address, etc).
- Every successful, failed, and spam form submission was recorded and tallied for the numbers you see in the study.
The Results Are In
The results, shown in the graph below, weren't that shocking to us considering we've all been in the same position with captchas before and had at least one failed attempt in our lifetimes. See for yourself.
Results of three months of testing with captchas on across fifty sites.
In the graph, blue shows successful conversions, green indicates a failed submission, and red shows spam submissions. At first glance, the results are striking, but note the scale and the y-axis—we start at 600.
Still, we're looking at 7.3% lost opportunity because of a single element on your page. And while some spam got through without the captcha, we can't help but wonder if a powerful spam filter like Akismet could help decrease those occurrences. We use it on our own blog here and have had wild success with it.
It's worth noting that in a similar situation, a single change to a web form button brought in an additional $300,000,000. What's this mean for us and captchas? Well, in other words, that 7.3% of failed conversions could be costing you thousands of lost conversions and who knows how much lost revenue.
So, What Now?
We had plenty of feedback in the comments of our last post. The consensus has been that we have captchas, and they keep out the spam, but they are really a pain for users and not the best solution—but what else do we have?
The best solutions so far has been using what's called a "honeypot," a form field hidden to users like you and I, but visible to the typical spam bot. They're called honeypots because, like bees to honey, spam bots flock to any form field. There are numerous variations out there, and the premise seems quite sound. But can we count on this holding up in the future?
Not likely. We're sad to say that spam is inevitably here to stay. However, designers, developers, and businesses can make huge differences by solving this problem in better ways that don't create bad experiences for users. The business reasons alone should make such solutions even more appealling.
Our suggestion? Take notice, don't (mis)place problems on your visitors, and don't forget to use design to creatively and effectively solve your business problems.
23 Comments
uidesigner says:
It really is a doube edged sword. We want to keep out spam, but yet we want to present a positive and quick user experience. I'm sure some creative genius will come up with the next best approach.
Maybe, using a unique identifier contact open id or something so you can bypass captcha if you are a verified person? Of course, this would not account for the general population coming to your web site, but it's a start.
Mark says:
@uidesigner: Indeed, it's all about the tradeoff in the end. We'll see what comes out in the next few years to combat spam. It's actually quite sad and pathetic that such a thing exists—who has time to create these things!?
And as for the unique identifier, we're already there in many case. Requiring registration on blogs and other services is actually the first step to that. Looking forward to what's next!
Mike says:
We've debated it and have put captcha on hold as while spam may be a pain to trash, it is irrelevant to our potential customers... Like UIdesigner says, waiting for the uber elegant way to handle it :)
Good read and facts!
Oliver Nassar says:
I think the JS approach is pretty powerful. For me, I open a session when a user visits (cookie based), and then before ANY form is submitted, I capture the event on the client side, make an ajax call which generates a unique token for the session. The response is then thrown into a hidden field, and then when I process the form on the server side, make sure that the the submitted token matches the one in the session.
With this approach, I've never had a problem with a spammer getting past it. Namely, it relies on cookie based sessions (weeding out bots that don't manage cookies), ajax/js to make a call and store the token, and then to submit the form. It requires some up front dev. costs, but long term saves you tons of time once you automate the process.
Mark says:
@Mike: Glad to hear others take up the some position!
@Oliver: Yeah, it sounds like you've got a sweet setup for weeding out bots on a few levels there. When can we hear more? :)
And to your point on costs, I can't help but smile when I hear about people and businesses taking the initiative up front to spend a little to make an experience better by leaps and bounds. We need more of that!
Scott says:
If my users have to wade through spam to find worthwhile user-generated content on my site, a captcha helps users solve their own problem. I'm not passing my business problem on to my users.
I'd say 7.3% lost conversions is just a cost of doing business. However, I would imagine there's a tipping point where your business is large enough that 7.3% turns into enough money to pay a human to deal with it. Just expect high turnover as those employees burn out. =)
Grafton Falvey says:
In the article, they state in a couple of places that the lost conversion rate is probably closer to 3%. That misleading SEOmoz graph doesn't help.
In the first case, without captchas, about 4.1% of all submissions were spam. If the spam rate is constant between the two data sets, that means about 95 of the total submissions in the second captcha-equipped data set were spam. Subtract the 11 known spam bits off the top, and you're left with 84 spammers out of the 159 failed conversions. That's about 3.4% of probable lost human conversions, or less than half the total failure rate.
Alexander Bickov says:
This post paid by spamers :)
@ithoughts_de says:
How do these Spam-bots work? Usually they shouldn't be able to "understand" JavaScript. If they just download the site, fill in all the form fields and submit the form, there are many possible ways to stop them ... like load the whole form with an AJAX-call, so the bots will never see them. Oliver Nassar's solution seems little more complicated but pretty foolproof.
Mark says:
@Scott: Absolutely, it all depends on the situation. There's always a tipping point to go from being protective to over the top protective.
@Grafton: You're right about the stats, but at what point does 3.4% become worthwhile to pursue? While the study doesn't mention what kind of sites these are, I'd be curious to see how even larger sites deal with the lack of captchas.
Wordpress.com, perhaps the largest blogging service out there, is backed by the spam filter Akismet and they seem to do fine with just that. We use Akismet here, too, and while we get a certain amount of spam, nowadays you our readers never see it.
As vested people in this blog, we take on that responsibility. And it's worth it all. There's always a tradeoff, and in many situations, it's worth it for whatever option you choose.
@Alexander: You mean spam-blockers :).
@ithoughts_de: From what we've heard, javascript solutions do seem like some of the most popular. I'm not sure exactly how they work. From our experience though, overdoing it with javascript can lead to problems of its own. While not that likely, what happens when javascript fails? Still, it sounds like a great idea :).
Chris Korhonen says:
The reports findings seem spot on to me - sophisticated Captchas may help limit spammers, but what you are doing is passing the problem on to your customers.
We recently removed ReCaptcha from our registration form, replacing it with spam prevention techniques on the front-end and back-end. The result? Around a 10-15% increase in the sign-up form's conversion rate and over the past 6 months, zero spam accounts that we have encountered.
Thats not to say Captcha's are not useful - in many cases when dealing with user generated content, they are necessary in order to protect users and your own brand, but in cases such as registration they are often a very lazy and intrusive way of solving the problem.
Mark says:
@Chris: Awesome to hear! Captchas seem to have their place as a proven defense, and while flawed in its own way, a captcha does keep spam out. Unfortunately, it can keep people out, too.
Robert Rawlins says:
I can totally believe these results, it's nice to see a few decent alternatives to captcha that are totally invisible and protect dependant on how the users enters the form, rather than having them validate their request.
Adam says:
CAPTCHAs are a valid use for denying spam and false form submissions. IP/doman blocking, 1x1px images, ect... are good but well formed CAPTCHAs are the best defense currently.
Justin says:
One of the best ways to circumvent honeypots and captchas is to use login authentication like Facebook Connect or Google Login. The problem is really already passed to them, and spam bots (so far) have not used them at all. The majority of users (depending on a savvy demographic) use FB connect to login to their favorite sites.
Mark says:
@Adam: I don't think anyone disagrees with you, but it'd be nice if we could :p. They work, but they confuse and they frustrate.
Find me a captcha that enhances the experience—one that truly makes visiting that site a better experience—on any web form and I'll toot my horn to another beat. Until then, while they work, they suck :).
@Justin: That's true! Using a third party system to take the brunt of the work in this situation sounds like a great idea, especially ones as large and secure (I hope) as Facebook and Google.
These solutions don't weigh on the visitor experience too much (extra steps suck, but only if it's that confusing), but they could cost you conversions should they ever go down or stop working. Savvy demographics should be set :).
I mention security only in passing as Twitter, certainly a large site, might not make for the best authentication given that they have their own problems with spam accounts and security.
palgrave says:
There's a good article here http://www.sitepoint.com/blogs/2009/05/14/captcha-alternatives/ that gives alternatives to captchas.
I have applied most of the methods described in a custom form, with success to date.
Mark says:
@palgrave: That's a good read for developers and designers. They advocate analyzing your (possible) spam problem in several ways, as well as listing out tools to help prevent it in the future. Good policy—analyze first, react second.
Martijn says:
Those 3D graphs are hard to read, you know... also bad for business. The first ar of the first graph appears to raise beyond 820, while the numbers add up to 817. 2D graphs are better for usability here... Sorry, but I had to say.
Otherwise good article. I personally hate graphical captcha's and I'd be glad I didn't have to build them anymore in our sites. My solution would be simple and elegant. Ask the user a question that any of them is able to answer. Something that relates to the topic of the website, or something general. But it must also be something a computer cannot answer. Example: ask the user to fill out the answer to "5+2" and you're done. Problem solved.
Mark says:
@Martijn: Happy to be called out on the graph—we admit, that particular graph is misleading in terms of the ratio of conversions.
We've seen a few instances of the "ask-a-question" method. Khoi Vinh over at Subtraction asks users to fill in a letter, which we experimented with, too. In the end, we nixed it and have just fought spam with Akismet ever since. Haven't had a problem :).
Web Host Right says:
I must say, ive never felt 100% ok about using captchas on some of my sites, as mentioned, it does seem like passing the problem onto the visitor/potential customer and it's the sort of thing slowing them down or preventing them altogether from completing an action.
A few times ive left a site after becoming frustrated with a captcha.
Elizabeth K. Barone says:
I never entirely liked Captchas on my sites, mainly because they annoy the hell out of me, so who's to say they don't annoy my users? (Ever try to put a shortened URL in a Facebook status update? Gah!) I was able to take reCAPTCHA off my WP sites because Akismet seems to do the job. Unfortunately, email contact forms are still hard to protect without annoying my users. And 70+ spam emails at any given time in my inbox (and my clients' inboxes) are no fun.
I'm going to have to check out those honeypots!
Mark says:
@Web Host Right: We feel the same way. It gets in the way. I have abandoned an entire sign up flow just because I couldn't get through a captcha. That kind of problem just makes users frustrated with the service and themselves, too.
@Elizabeth: Let us know how it goes with the honeypot solution! Comment spam is an easier thing to get past, but custom forms like that are the key to getting around the spam attacks.