Our crawler reads only public pages: the same responses a browser or search crawler would get. It identifies itself, follows robots.txt, and never logs in.
What the crawler requests
A check reads what any visitor could read without an account. That means:
- Your public pages, fetched as served and again after running JavaScript
- robots.txt, your sitemap and other files at standard locations, such as llms.txt
- The markup of forms and buttons, which we read but never submit
How it identifies itself
Every request carries the user agent Code4XBot/1.0, so you can find our visits in your logs.
How to block or slow it down
The crawler follows robots.txt and Crawl-delay. To block it across your whole site, add this to robots.txt:
User-agent: Code4XBot
Disallow: /If your site blocks the crawler, the check stops and says so. It never works around the rule.
Live agent testing
Live agent testing is different: a real agent completes tasks on your site, such as filling in a form. It runs only inside an agreed engagement, on flows you choose.
What we store, and for how long
- Results. The website, the date and what each check found. A free check result stays at its link for 30 days.
- Server logs. The IP address, user agent and time of requests to code4x.com, kept for a short period for security.
- Nothing about your visitors. The crawler reads your pages. It never reads analytics, cookies or anything behind a login.
Who can see a result
Each result lives at its own link with an ID no one can guess. The website’s address never appears in the link, and the link never shows who asked for the check.
Report a problem
If our crawler caused trouble on your site, tell us and we’ll stop checking it. Write to us through the Talk to an engineer form and say what your message is about. We reply by email.