What is a web crawler? | How web spiders work
A web crawler, or spider, is a bot typically operated by search engines like Google and Bing. Their purpose is to index the content of websites all across the Internet…
How CAPTCHAs work | What does CAPTCHA mean?
CAPTCHAs and reCAPTCHAs determine if a user is a bot. While these tests can help stop malicious bot activity, they are far from foolproof. What is a CAPTCHA? A CAPTCHA…
What is a spam bot? | How spam comments and spam messages spread
A spam bot is a computer program that helps to spread spam across the Internet. Spam bots often scrape contact information, create fake user accounts, or operate stolen social media…
What is a reverse proxy? | Proxy servers explained
A reverse proxy protects web servers from attacks and can provide performance and reliability benefits. Learn more about forward and reverse proxies. What is a proxy server? A forward proxy,…
Sample robots.txt
# .__________________________. # | .___________________. |==| # | | ................. | | | # | | ::[ Dear robot ]: | | | # | | ::::[ be nice ]::…
What is click fraud? | How click bots work
Click fraud fakes clicks target pay-per-click ads, boosting webpage search rankings or artificially inflating the popularity of a post. Click bots are often responsible for click fraud. What is click…
What is a bot? | Bot definition
A bot is a software program that operates on the Internet and performs repetitive tasks. While some bot traffic is from good bots, bad bots can negatively impact a website…
What is rate limiting? | Rate limiting and bots
Rate limiting blocks users, bots, or applications from overusing or abusing a web property. Rate limiting can stop certain kinds of bot attacks. What is rate limiting? Rate limiting is…
What is personal information? | Personal data
Personal information is any information that can identify a person, from someone’s name and address to their device identifier and account number. What is personal information or personal data? Personal…
What are the Fair Information Practices? | FIPPs
The Fair Information Practice Principles, or FIPPs, are a set of principles for data privacy that many organizations follow today. What are the Fair Information Practice Principles (FIPPs)? Fair Information…