noindex / robots Meta Tag Tester
Why isn't Google indexing your page? Three layers control indexability: robots.txt (path-level), X-Robots-Tag HTTP header (any), and <meta name="robots"> (HTML page only). This tester analyzes inputs from all three and tells you the effective indexability.
How to use the noindex / robots Meta Tag Tester
Enter the page's robots <meta> tag, any X-Robots-Tag HTTP header value, the path being tested, and your robots.txt content. The tool weighs all three layers together and reports whether the page is effectively indexable, explaining which rule wins.
Why three layers decide indexability
When Google won't index a page, the cause is usually one of three controls fighting the others: robots.txt decides whether the crawler may fetch the URL at all, the X-Robots-Tag HTTP header can set noindex on any response, and the <meta name="robots"> tag does the same inside an HTML page. A conflict between them produces surprising results — a blocked page can't even read your noindex.
This tester takes all three inputs and reports the effective indexability, naming the directive that wins so you know exactly what to change. To then generate a correct crawl file, use the robots.txt generator.
Common use cases
- Indexing diagnosis — find why a page isn't appearing in search.
- Conflict spotting — see when robots.txt blocks a noindex from being read.
- Pre-launch checks — confirm staging rules won't deindex production.
- Header vs meta — compare X-Robots-Tag against the meta tag.
- SEO audits — validate indexability across a set of templates.
Frequently asked questions
Which layers does it evaluate?
robots.txt path rules, the X-Robots-Tag header, and the <meta name="robots"> tag — and how they interact.Why can a noindex tag be ignored?
robots.txt blocks the URL, the crawler never fetches the page and so never sees the noindex. The block must be lifted for the directive to take effect.