The ultimate open developer suite to audit your website's AI readiness. Generate both llms.txt maps and llms-full.txt content corpuses from XML sitemaps in seconds.
Paste the contents of your llms.txt file below to perform real-time linting, syntax checking, link verification, and density parsing.
Enter any domain name or website URL to check if they serve llms.txt or llms-full.txt files.
Fill out the metadata fields below to dynamically compile both a root map index (llms.txt) and a comprehensive corpus template (llms-full.txt).
Paste your domain's XML sitemap URL below to parse and filter URLs. Export a structured index (llms.txt) and a comprehensive content template (llms-full.txt).
Deploy dynamic, crawler-ready llms.txt configurations in your existing web architecture instantly. Select your setup below:
loading template...Providing structured descriptor files acts like a table-of-contents catalog for crawler heuristics.
Autonomous scraping agents from OpenAI (O1-bot), Anthropic (ClaudeBot), and Gemini scan for `/llms.txt` at the domain root before scraping page trees, reducing redundant request rates.
By providing direct descriptive labels and links to your API references, software libraries, and price guides, you control target sources cited in AI answers.
Avoid server timeouts. Direct crawlers to pre-cleaned Markdown files (like `llms-full.txt`), preventing indexers from executing heavy database queries or tracking scripts.
In the model-driven internet of 2026, search patterns have evolved from simple click queries to synthesis engines. Structuring content for Large Language Models is now a vital development practice.
Originally proposed by Jeremy Howard and the team at Answer.ai, llms.txt is a plain text Markdown file hosted in the root directory of a web host. While robots.txt blocks file directories, llms.txt helps indexers catalog key resources and retrieve target documentation immediately.
The core philosophy is efficiency. LLM prompts operate under strict token budget ceilings. By serving clean Markdown lists without layout blocks, tracking scripts, and styling rules, you help AI models digest content with 100% fidelity.
To pass parser validation scripts, files must comply with the following structural rules:
Below is a standard representation showing sub-sections and descriptive link summaries:
# My Platform Name
> Clean deployment and API infrastructure details.
My Platform is a serverless operations tool optimized for static frameworks.
## Technical Guides
- [Quick Start Guide](https://mysite.com/docs/start): Onboard, build, and configure systems.
- [REST Reference](https://mysite.com/docs/api): Complete endpoint parameters and routes.
## Full Content Database
- [llms-full.txt](https://mysite.com/llms-full.txt): Integrated corpus containing full post texts.An llms.txt file is a markdown-formatted file placed at the root of a domain to help AI agents and Large Language Models (LLMs) effectively discover and ingest relevant website content. You can read our complete guide to the llms.txt standard for more details.
You can use our llms.txt Generator to manually enter project details and links, or use the Sitemap.xml to LLMs.txt converter to automatically generate files from your existing XML sitemap index.
As search shifts toward generative AI (GEO), providing structured content for crawlers like OpenAI-bot and Gemini is crucial for being cited correctly in AI-generated answers. Explore the SEO benefits of llms.txt for a deeper dive.
The file must be placed in your website's root directory, accessible at https://yourdomain.com/llms.txt. This convention is similar to how robots.txt is served across the web.
While robots.txt is an access control system (exclusion rules), llms.txt is a navigation index (inclusion guide). It highlights the most important pages AI models should index first. Read more on the differences between llms.txt and robots.txt.
llms.txt serves as a concise roadmap index of links and summaries. llms-full.txt is an optional companion containing the actual textual content of those pages in Markdown, permitting deep ingestion in a single pass. Check out the ingestion specification for technical details.
Use our llms.txt Validator tool. It performs real-time syntax checking, link verification, and structure analysis to ensure your file meets the official Answer.ai specification.
Yes, you can implement it dynamically. We recommend reading our guide on how to add llms.txt to WordPress or checking out our comparison of WordPress SEO plugins for llms.txt support.
Absolutely. Modern frameworks like Next.js can serve these files using dynamic route handlers in the App Router. Follow our Next.js integration guide for full implementation steps.
Even with an llms.txt file, you can control crawler access via robots.txt for specific bots. View our curated list of AI crawler user-agents and IP ranges to keep your blocking rules up to date.
No. llms.txt is a public file intended for discovery. You should strictly avoid placing private, sensitive, or paywalled information in it. For more on best practices, see llms.txt security and privacy.
Yes. Our Sitemap.xml to LLMs.txt converter allows you to fetch any remote XML sitemap, filter out URLs you wish to exclude (like transactional or admin pages), and compile the final text files automatically.