The Anatomy of a Bulk Testing Report
When you execute an evaluation using the SEO Free Bulk Robots.txt Tester, the tool returns a comprehensive diagnostic report analyzing up to 100 paths simultaneously. To the untrained eye, a large table of status badges can feel overwhelming. However, every column provides essential technical evidence regarding how search crawlers will handle your URLs.
In this guide, we break down each metric and demonstrate how to turn diagnostic findings into actionable engineering tickets.
1. Summary KPI Metric Cards
At the top of the results panel, four summary cards provide an immediate high-level snapshot:
- Total Evaluated: The count of unique, valid URLs processed in the batch.
- Allowed: Paths that search crawlers are permitted to request.
- Blocked: Paths that match an active Disallow directive and will be rejected by crawlers.
- Warnings / Scope Errors: URLs that were malformed, belonged to a different domain origin, or encountered syntax issues.
2. The Detailed Results Table Breakdown
Every evaluated row contains six key technical data points:
| Field | Technical Significance | What to Look For |
|---|---|---|
| Target URL / Path | The specific address tested against rules. | Confirm path casing, query strings, and trailing slashes. |
| Status Badge | The binary crawl outcome: ALLOWED or BLOCKED. | Ensure top organic landing pages are never marked Blocked. |
| Crawler Token | The user-agent evaluated (e.g., Googlebot, *). | Verify whether rules came from a dedicated group or wildcard fallback. |
| Matched Group | The User-agent header owning the winning rule. | Confirms which block of directives governed the decision. |
| Decisive Rule & Line | The exact directive pattern and file line number that won. | The core evidence: shows which rule beat all other competing lines. |
| Plain-English Explanation | Technical rationale detailing octet length and tie resolution. | Explains why this specific rule superseded conflicting directives. |
3. What Is a "Decisive Rule"?
Under RFC 9309, multiple rules may match a given URL. The decisive rule is the single directive that possessed the longest matching character length (highest octet specificity). If two rules tied in length, the decisive rule is the Allow directive.
Identifying the decisive rule eliminates guesswork. If a URL was unexpectedly blocked, the results table will pinpoint the exact line in your robots.txt file causing the restriction.
4. Filtering and Safe CSV Exporting
To streamline large audits:
- Filter Tabs: Click "Blocked" to isolate only disallowed URLs for immediate troubleshooting.
- Live Search: Filter results in real time by typing a keyword (e.g.,
/category/or.pdf) into the filter search box. - Export to CSV: Download the complete dataset. Our export engine automatically escapes formula control characters (
=,+,-,@) to prevent spreadsheet security vulnerabilities.
Frequently Asked Questions
Why does a row say "Allowed by Default"?
If no rule in the applicable crawler group matched the requested URL, RFC 9309 dictates that all non-matching paths are permitted by default.
Why were duplicate URLs removed from my batch?
Our evaluator automatically deduplicates identical inputs to ensure clean, accurate reporting and conserve computational resources.
What does "Out of Scope" mean?
If you enter a website origin of https://example.com, but paste a URL from https://otherdomain.com, the tool flags it as out of origin scope.
