Cantech Knowledge Base

Your Go-To Hosting Resource

  1. cPanel
  2. Learn to Block A User Agent in An .htaccess File

Learn to Block A User Agent in An .htaccess File

Have you ever caught a pestering bot crawling your website and stealing your website content, and overloading your server with requests? Don’t get me wrong, some bots are useful (like Googlebot indexing your web pages), but other bots can be harmful, which could cause spam on your site, botnet or other security issues, or possible excessive bandwidth use for “bad bots” scraping your content. It’s best just to block user agents.

One solution is at the .htaccess level by blocking certain user agents. In this blog, I am going to help you learn to block user agents. Again, to help protect your website and improve your website’s actual performance and minimize performance in the first place. So let’s get started!

What is an .htaccess File?

Typically, your .htaccess file will be located in the root directory of your website. If you’re not able to see it, ensure that your file manager, cPanel or Plesk (if available), are set up to show hidden files. You can also access the directory using an FTP client such as FileZilla.

You can easily edit your .htaccess file and filter out unwanted visitors or block certain user agents (bots, scrapers, malicious crawlers), with which you can protect your website, avoid spam, and improve performance.

Why Block a User Agent?

Blocking certain user agents can:

  • Protect your website from spam bots that attempt to post spam comments or malicious content.
  • Prevent content scraping to safeguard your original content from unauthorized copying.
  • Reduce unnecessary server load and enhance performance.
  • Improve security by restricting access from harmful bots.

How to Block a User Agent in .htaccess

Blocking a user agent in your .htaccess file is simple. Just follow these steps:

Step 1: Locate Your .htaccess File

Your .htaccess file can usually be found in your website’s root directory. If you are not able to see it, ensure that hidden files are visible in your file manager. Or, you can use an FTP client (like FileZilla).

Step 2: Add User-Agent Blocking Rules

Open the .htaccess file using a text editor and insert the following code:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} "BadBot" [NC]
RewriteRule .* - [F,L]

What this does:

  • Blocks any bot using “BadBot” as its user agent.
  • [NC] makes it case insensitive.
  • [F,L] means forbidden and stops further rules.

If you want to block several bad bots, use this format:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} "(BadBot|EvilScraper|FakeCrawler)" [NC]
RewriteRule .* - [F,L]

This method blocks multiple bots (using BadBot, EvilScraper, FakeCrawler) efficiently in one line.

Some bots hide their identity by sending blank user agents. To block them, add this rule:

RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteRule ^ - [F]

This blocks any request with an empty or missing user agent.

Step 3: Save and Upload the File

Once you’ve added the rule, save the .htaccess file and upload it back to your server if necessary.

Step 4: Test Your Website

After implementing the changes, visit your website and ensure everything functions correctly and normal visitors can still access it. You can use online tools to test whether the blocked user agents can still access your site. If something breaks, remove or adjust the rules.

Need Help?

If you’re using one of Cantech’s services like Cloud Hosting, Managed VPS Hosting, or Dedicated Server Hosting, our team of professionals can help you set up your .htaccess file to improve your site security. Combine our superior servers with our 24/7 support to get the best security and reliability.

Final Thoughts

Blocking user-agents in .htaccess is an easy and effective way to protect your site. You can follow these steps to keep a secure and optimized website.

Want more expert tips on website security and performance? Stay tuned to our blogs and explore our web hosting solutions tailored for your business!

Have questions? Contact Us today!

FAQs

What happens if I block the wrong user agent?

Blocking an internet bot that is a legitimate user agent (like Googlebot), could mean your site won’t be indexed properly. Always double-check user agents before blocking every bot that is visiting your website.

Can I unblock a user agent later?

Yes! Simply remove or modify the blocking rule in your .htaccess file and save the changes.

Will blocking bots affect my website performance?

If you take the time to prevent and block harmful bots from a website, the performance of your web website will actually be improved due to lower traffic, memory, and data consumption.

How do I find out what bots are visiting my website?

You can see your server logs, or you can use a search engine console like Google, or an analytics package to see what user agents are visiting your site.

Does Cantech Networks offer managed security solutions?

Yes! If you are using Managed Hosting or Dedicated Servers from Cantech Networks, we can help you, by configuring .htaccess rules to improve your websites security.

June 21, 2025