How to Effectively Protect Your Website from Bots & Spammers?
Are you in a war against bots, fake accounts and gambling advertisers? Let’s discuss how to protect yourself from them as a simple developer.
In this article, you will find a list of captcha services you can use to block content from bots. Which methods do bot owners use? And what can we do for them? I’m a web developer running a website with over 50K users. And these bots are everywhere, now on our website abp.io.
Our main goal here is to detect whether the request is from an actual person or a bot. And the most common method is to use challenge-answer test; Captchas.
As you know, all big systems like Instagram, Twitter and Facebook are having problems with these fake/bot users. They take some serious precautions to prevent this abuse. But for simple websites, how can we protect ourselves from these bots? I’ll explain all the details about this topic here…
Current Situation
In this article, I’m writing about my own experiences on our website abp.io. So first, I spotted which email providers are being used by bots. The below picture shows that gmail.com and qq.com are mostly being used by users.
Which Domain Services Are Being Used by Bots?
When I checked, the user list of those who are trying to post advertisements is mostly from the gmail.com and qq.com domains.
In the below screenshot, you can see the bot accounts. These are posting advertisements on our platform.
Why Are Bots Registering to a Software Development Website?
Our website is not a world-wide famous content system. That’s why it may be weird to see the bots on this platform. But wait! We are popular on .NET developers and our website shares the users’ information publicly. This is my profile URL https://abp.io/community/members/alper
As you see from my profile, there’s a “Biography” field where I can write some free text. This field is enough for bots to share their advertisements.
Bots write their advertisements using the name, surname, biography, and website URL fields. In this way, they leave a backlink to their website. I took a screenshot of some fake user accounts on our platform. As you see, they are here to advertise for online gambling websites.
How can We Protect Our Website from Bots?
There are several ways to ban the bots. The most common way is using a CAPTCHA service. While this is the most effective way, at the end of this article, I’ll share other methods that you can combine with Captcha.
Which Captcha Services are Used by Popular Sites?
Before we start using something, we should always look at what popular systems are doing. Let’s see what GitHub, Twitter, LinkedIn, Amazon and others are doing!
GitHub.com
GitHub uses FunCaptcha by ArkoseLabs. When I checked this service, I saw that many big systems use it—for example; DropBox, Adobe, Expedia, Hotels.com, Blizzard, OpenAI, Roblox and Microsoft.
It uses its own service, ReCaptcha, with different difficulty levels. For example, when you use a VPN to access Google services, it shows the following challenge to understand whether you are human or not.
Twitter first accepts users with a valid email or phone number. This verification eliminates invalid users before using a captcha.
After successful verification, Twitter shows some challenges from FunCaptcha by ArkoseLabs.
LinkedIn also uses FunCaptcha by ArkoseLabs.
Let’s see the other popular websites and what they used for captcha solutions:
- Amazon is using Arkose Labs FunCaptcha.
- Roblox is using Arkose Labs FunCaptcha.
- Netflix is using Google Recaptcha Enterprise V3.
- Adobe is using Google Recaptcha Enterprise V3.
- Binance is using Geetest Captcha.
In summary, the popular websites generally use Arkose Labs FunCaptcha. You can test FunCaptcha’s challenge on the Amazon.com website from the following link https://www.amazon.com/aaut/verify/flex-offers/challenge?challengeType=ARKOSE_LEVEL_1&returnTo=https://www.amazon.com&headerFooter=false
Best Captcha Services to Block Spammers
Here’s a list of popular CAPTCHA services:
1-) FunCaptcha by Arkose Labs
Known for using interactive tasks as CAPTCHA challenges like game-like challenges, FunCaptcha aims to provide an engaging experience. Platforms often choose it to reduce friction for legitimate users. Many popular websites prefer this service.
- Pricing: Not free!
- Website: https://www.arkoselabs.com/
2-) hCaptcha
It’s the second most popular captcha service after FunCaptcha. It provides similar bot-detection features and emphasizes data privacy.
- Pricing: It’s free but has premium plans with more features.
- Website: https://www.hcaptcha.com/
3-) Google reCAPTCHA
Google’s reCAPTCHA is widely used and offers various versions, including “invisible” CAPTCHA (v3), which identifies bots without user interaction. It’s popular for its ease of integration and accuracy. But many bots are already solving reCAPTCHA’s challanges.
- Pricing: It’s free up to 1 million calls per month.
- Website: https://developers.google.com/recaptcha/intro
4-) BotDetect CAPTCHA
BotDetect offers a customizable CAPTCHA solution without tracking users, which is suitable for privacy-conscious websites. It includes options for audio and image CAPTCHAs, adaptable to different user needs. Simple usage for simple websites.
- Pricing: Free and paid plans.
- Website: https://captcha.com/
Open-Source Captcha Libraries
If you are willing to create your own Captcha service, here’s a list of some libraries that you can use:
- DNTCaptcha.Core: Captcha generator for .NET.
- SixLaborsCaptcha: Captcha generator with SixLabors libraries for .NET.
- Tesseract: OCR library in C++ for Python.
- MotionCAPTCHA: A jQuery plugin for drawing shapes.
- Negative-captcha: Creates negative captcha in Rails.
- Django-simple-captcha — Adds captcha images to any Django forms.
- Securimage: Captcha for PHP.
- Captcha_solver — Univeral API to captcha-solving services.
Are Captcha Services Really Good Solution?
While we focus on implementing a captcha solution on our website, we should also check attackers… Which methods do they use to bypass our guards? Are these captcha services really good? Can attackers easily bypass these protections? The answer is YES! I will list you the popular anti-captcha services. These guys are giving a paid service to bypass your captchas. So you should also care about their API’s. Maybe you need to monitor which IP addresses you are getting too many requests. And then ban these addresses.
How to Bypass Captcha — From the Attackers Perspective?
There are many paid services to solve your application’s captcha. They even use humans to solve the captchas.
Which Services Are Bots Using?
I listed the most popular services that bots are using to solve your website's captcha challenges.
2Captcha
https://2captcha.com/ is an automatic captcha bypass service. I took this screenshot from their website. As you see, they solve almost all popular captcha protections.
AntiCaptcha
They solve the following captcha providers. Their website is https://anti-captcha.com.
CapSolver
This is another popular captcha solver. The website is https://www.capsolver.com. And it solves popular captcha services.
Death By Captcha
This service solves 20 different captcha services. Their website is https://deathbycaptcha.com
There are more captcha solvers than captcha generators!
SOLUTIONS 💁
As a Simple Developer, How Can I Make It Harder for Bots?
As you can see from the Captcha solvers, these people are wasting a lot of time defeating all these captcha services. They even run a business on top of it. Here’s a list of what you can do for bots/fake users/attackers etc…
1. Rate-Limiting
You can use an IP-based rate limiter. For example, if you get repeating user registrations from the same IP. For example, Hotmail allows 3 accounts per day per IP address as stated here. For more info, you can read the Rate Limiting Best Practises guide. For ASP.NET Core, there’s a built-in rate-limiting middleware that you can use. See the documentation at this link.
2. JavaScript Load Tokens
This approach generates a unique token only after certain JavaScript code is executed on the client side, which most bots cannot handle. When a human loads the page, the JavaScript runs, creates a token, and sends it along with the request. Since bots generally don’t execute JavaScript, they fail to send this token. This method works well as an additional verification layer.
3. SMS Verification
SMS verification requires users to confirm their identity via their mobile phone. Many bots can’t complete this. You can use services like Twilio.
4. Random Captcha Providers
Bots can solve almost all the popular captchas. However, most of these can solve one captcha service for a website. You can use different captcha services randomly to confuse the bots. By doing so, bot operators will waste more time to make a specific integration for your website.
5. IP Trustworthiness
There are APIs that evaluate IP addresses based on trustworthiness, checking if they are flagged for bot activity. For example IpQualityScore.com is a service to detect if IP is already marked as bot. I checked one of the bot IP addresses that I found on abp.io and it successfully identified as a bot in this example. Or whatismyipaddress.com is another alternative.
6. Hidden Checkbox Honeypot
Adding an invisible checkbox in your HTML form is a simple but effective trap. Bots like checking all checkboxes in a form. You can ban the IP addresses that check this checkbox.
7. Attack Pattern Monitoring
Check if the attacker is using a different user-agent or adding a custom header. If so, you can easily ban these requests based on a specific pattern.
8. Allow Only a List of Countries
If your business targets specific countries, then block other countries. It will simply filter most of the attacks.
9. Document Verification
For high-security applications, document verification is a powerful method. In this solution, people send pictures of their passports or local ID cards. For example, LinkedIn uses a system called Persona.
By implementing these methods, you can build a layered defense system against bot traffic. Protect your site from bots and maintain a secure, user-friendly environment.
🧑🏽💻 I’m Alper Ebicoglu
ABP Framework Core Team Member.
Follow me for the latest news about .NET and software development:
🌐 twitter.com/alperebicoglu
🌐 github.com/ebicoglu
🌐 linkedin.com/in/ebicoglu
🌐 medium.com/@alperonline