Robots.txt
textareaRequired input
Noderan MarketplaceContent & Marketing
Demo ReadyLive E2E verifiedDeterministic~5s
Robots.txt Inspector
Summarizes crawl rules, sitemap references, and blocking risk.
Inspect pasted robots.txt directives for crawlability signals.
Workflow summary
Inspect pasted robots.txt directives for crawlability signals.
What is Robots.txt Inspector?
Robots.txt Inspector is a ready-to-run Noderan marketplace workflow for content & marketing teams. It helps users move from a manual process to a repeatable automation with visible credit cost and app-based execution.
- Who is it for?
- Marketing and technical SEO teams
- What problem does it solve?
- Inspect pasted robots.txt directives for crawlability signals.
- How does it work?
- Inspect pasted robots.txt directives for crawlability signals.
- How does pricing work?
- 1 credit per run.
- What is the next action?
- Open the app marketplace to activate this workflow, or review credit pricing.
Robots.txt Inspector FAQ
Short answers for activation, pricing, inputs, and execution review.
Summarizes crawl rules, sitemap references, and blocking risk.
Expected inputs
Demo input
Robots Txt
User-agent: * Disallow: /admin Sitemap: https://noderan.com/sitemap.xmlView raw JSON
{
"robotsTxt": "User-agent: *\nDisallow: /admin\nSitemap: https://noderan.com/sitemap.xml"
}Output highlights
- user-agent groups
- disallow rules
- crawlability
Example result
User Agent Count
1Disallow Rules
/admin
Sitemap Urls
https://noderan.com/sitemap.xml
Warnings
No blocking rule for all content detected
Crawlability
partially_restrictedView raw JSON
{
"userAgentCount": 1,
"disallowRules": [
"/admin"
],
"sitemapUrls": [
"https://noderan.com/sitemap.xml"
],
"warnings": [
"No blocking rule for all content detected"
],
"crawlability": "partially_restricted"
}Use cases
- Review robots.txt before launch or SEO audits.
How it works
- 1
Review the workflow and expected credit cost.
- 2
Connect the tools or inputs required for execution.
- 3
Run the workflow and inspect outputs in the app history.