"We're seeing a huge spike in traffic! Our marketing must be working!"
Some website admins may get excited about a sudden increase in visitors to your site. Not only does it help your business, but when you can show that your site actually has a following, it's easier to attract paid advertisers, increasing revenue. Yet not all traffic is created equal, and you might be surprised to learn that a good portion of your site traffic -- in some cases, well over half -- isn't even human, but bot traffic.
Image Credit: Flickr, Creative Commons, Lisk Bot
Let's Start at the Beginning: What Are Bots?
In the simplest terms, bots are software applications that are written to perform specific tasks on the Internet, usually repetitive tasks that would be impossible or difficult for humans to perform with great speed. Not all bots are malicious; online chat sites that automatically scan for profanity or inappropriate content use bots.
However, malicious bots are more common. Hackers develop them to disrupt normal Internet traffic, steal information, or commit fraud. Malicious bots usually fall into one of four categories:
Impersonators. These are the most dangerous bots. They are designed to disrupt the functioning of the site and steal information; they might cause a DDOS attack, inject spyware onto your site, or appear to be a fake search engine, among other things.
Hackers. Usually targeted at CMS sites, these bots are primarily written to steal information or hijack the site.
Scrapers. These hit e-commerce and content sites most often, stealing content to post elsewhere, or using the information gathered to reverse engineer a business model.
Spammers. As previously mentioned, these are the most common bots, posting spam in comments, spreading phishing scams, and engaging in negative SEO against competitors.
Why Are Bots So Bad?
Depending on the type of bot that's visiting your site, it might seem that the problem is nothing more than an annoyance; after all, it's relatively easy to delete or block spam comments. If you use a quality website builder, your site should be protected against hackers and their malware, and the bots will have less of a probability to be able to attack your site successfully.
However, even if they don't successfully hack into your site and cause damage or steal data, bots create problems in terms of analytics, particularly when it comes to advertising. For starters, sites set rates based on traffic -- the more traffic, the higher the rates they can command for space. If most of the traffic isn't human (this study showed 61.5% of traffic is not), then those rates may be artificially inflated.
And as an advertiser paying for advertising, you want to be sure that the traffic you're paying for is legitimate.
Beyond advertising, bot traffic skews analytical data in other ways. For example, sites conducting A/B testing may have inaccurate results due to bot traffic, or make other changes based on what the false data shows. Bots can also slow down the speed of your site, making it difficult for legitimate users to get what they need. They might also fill out Web forms to subscribe to newsletters or other updates, affecting your CRM efforts.
Finding the Bots You Are Looking For
There are usually signs that your site is being overrun with bots, and you need to be aware of them. Some of the more common signs include:
Problems with site performance. If it becomes slow or crashes frequently, bots could be the problem.
You see your site content showing up elsewhere online. Bots aren't concerned about plagiarism, and will blatantly steal your content.
Your bounce rate and new session rate is excessively high. On the other end of the spectrum, very low pages per session and session duration rates are also clues that there could be a bot issue.
The traffic source seems "off." For example, if you have a page in English, but most of your traffic comes from non-English speaking countries, it's likely bot traffic.
Vastly different analytics if you filter for bot traffic.
So what can you do if you have a bot problem? It's practically impossible to block bot traffic, but given the rise in the problem -- which affects even Alexa MVP sites that receive more than 1 million visitors per day -- Google and other companies are finding new ways to filter bots.
When performing analytical analyses, you can filter the traffic from known bots from your results and get a more accurate picture. And of course, protecting your site and users from malicious bots is key; installing the most up-to-date protections and security tools is your best defense against these potentially destructive scripts.