AWS WAF Bot Control and Carrier IP Tests

AWS WAF Bot Control is the bot layer many teams switch on in front of CloudFront, an Application Load Balancer or API Gateway because it is already in their account. It is also refreshingly documented: you can read the rule names, see the labels it attaches to each request, and write your own logic on top. That makes it the best vendor on this list for learning how bot detection weighs an IP. This page walks through the model, what a dedicated NeonProxies carrier line changes in it, and how to use lines to test your own AWS rules.

Common versus Targeted inspection

Bot Control is a managed rule group with two inspection levels. Common identifies self-declared and verifiable bots, such as search engine crawlers, monitoring services, social media preview fetchers and HTTP libraries announcing themselves in the user agent, and labels them by category. Verified bots, whose identity AWS can confirm, get a separate label so you can allow them without allowing imitators. Common is mostly about who a client says it is.

Targeted goes further for bots that do not identify themselves. It adds a client-side JavaScript integration or mobile SDK that interrogates the browser and issues a token in the aws-waf-token cookie, then applies browser fingerprinting, request volume per client session, and machine learning over traffic patterns. It can respond with the CAPTCHA or Challenge actions, where Challenge is a silent browser check and CAPTCHA asks a person to solve a puzzle.

Where IP data enters the picture

AWS keeps address-based signals mostly in separate managed rule groups. The Anonymous IP list flags traffic from hosting providers and cloud platforms, and the IP reputation list flags addresses associated with bots and threats. Teams commonly enable these next to Bot Control, which is why a scraper running on an EC2 instance in another account can be labeled as hosting-provider traffic before Bot Control even weighs in.

A NeonProxies line exits from an AT&T, T-Mobile or Verizon range, so it will not carry a hosting-provider label, and because each line serves one customer it does not pick up reputation from others. Bot Control's own rules still see the user agent, the token, the fingerprint and the request volume. A carrier address with a library user agent is still labeled as a library, and a missing or expired token is still a missing token, whatever network delivered the request.

Testing your rules with real carrier traffic

This is where a line earns its keep for AWS teams. Before moving a rule from Count to Block, you want to know how genuine mobile users are labeled. Browse your application through a carrier line from a normal browser profile, then pull the matching entries from your WAF logs and read the labels: did the request pick up a bot category, did the token validate, which rule matched. Repeat from a cloud instance as a control. The difference shows how much of your protection depends on the Anonymous IP list and how much on Bot Control proper.

Keep the tests realistic. Walk the journeys customers take, at the pace they take them, and include the mobile network latency that makes token acquisition slower on a phone than on an office connection. If your Challenge action times out on a carrier line, it will time out for customers too.

Being a good client of someone else's AWS app

When the protected application belongs to someone else, the labels are theirs to act on. Identify your job honestly in the user agent if the site allows declared crawlers, keep to modest rates, and stop when a CAPTCHA appears instead of attempting it. Rate-based rules in AWS WAF count requests per address over a short window, so a single line pushing parallel workers will trip them long before any bot model does. One worker per line, with a sensible delay, keeps you under those counters and makes your traffic easy to reason about.

Setting up a AWS WAF Bot Control proxy on NeonProxies

1. Enable Bot Control in Count mode on your own web ACL and turn on WAF logging.
2. Buy a line in the metro where most of your users are.
3. Browse your key flows through the line in a normal browser profile.
4. Repeat the same flows from a cloud instance as a control.
5. Compare labels, token validation and matched rules in the logs.
6. Adjust rules, then move them from Count to Block when mobile users pass cleanly.

AWS WAF Bot Control proxy questions