llms.txt Examples, Format, and How to Create One for Your Docs

Q: What is the difference between llms.txt and llms-full.txt?

`llms.txt` is a curated index — a short Markdown file listing your most important pages with one-sentence descriptions. `llms-full.txt` contains the complete text of those pages concatenated together, allowing an AI agent to ingest your full documentation in a single request without following individual links.

Q: Does Google use llms.txt for SEO?

Google has stated it does not use `llms.txt` for traditional web search indexing. The file is designed for LLMs operating at inference time — AI agents, chatbots, and retrieval-augmented systems — rather than for search crawlers. Implementing it does not replace standard SEO practices like proper sitemaps and structured data markup.

Search engines get a robots.txt. AI crawlers and agents get an llms.txt.

The llms.txt standard gives Large Language Models a fast, low-noise path to the content that matters most on your site. It is a plain Markdown file served at the root of your domain — no heavy HTML, no navigation menus, no ads. Just a curated index of your most valuable pages and the context an LLM needs to understand and cite your content accurately.

If you are building documentation-heavy developer tools, RAG pipelines, or any product where AI discoverability matters, an llms.txt file is a practical step worth adding to your deployment checklist. file2markdown.ai helps you convert the source documents behind that content — PDFs, Word files, HTML exports — into the clean Markdown that llms.txt is built on.

What Is an llms.txt File?

An llms.txt file is a plain Markdown document, served at https://yourdomain.com/llms.txt. The spec (first published at llmstxt.org) was designed to let LLMs quickly find your most authoritative content without crawling your full site.

It differs from robots.txt and sitemap.xml in a key way: where those files contain machine directives, llms.txt is written for an LLM audience. Instead of crawl rules or URL lists, you write a short natural-language summary of your site and a curated list of your most valuable pages with one-sentence descriptions.

The standard also defines an optional companion file, llms-full.txt, which concatenates the full text of those linked pages. AI agents that need to ingest your complete documentation in a single request can fetch this file instead of following each link individually.

For a broader look at why Markdown outperforms raw PDFs and Word files for AI consumption, see the guide on why LLMs prefer Markdown.

The llms.txt File Format

The specification has three required structural elements:

H1 title — the name of your site or project.
Blockquote summary — one to three sentences describing what the site does and who it serves.
H2 sections — categorized lists of Markdown links, each followed by a one-sentence description.

An optional ## Optional section at the bottom can hold lower-priority content (blog archives, changelogs) that an LLM may skip when context is limited.

# Your Product Name
> One-paragraph summary of what your product does and who the intended audience is.

## Getting Started
- [Introduction](https://yoursite.com/docs/intro): What the product is and who it is for.
- [Quickstart](https://yoursite.com/docs/quickstart): Get running in under five minutes.

## API Reference
- [Authentication](https://yoursite.com/docs/auth): How to authenticate API requests.
- [Endpoints](https://yoursite.com/docs/endpoints): Full endpoint reference with parameters and examples.

## Optional
- [Blog](https://yoursite.com/blog): Tutorials and product updates.
- [Changelog](https://yoursite.com/changelog): Release notes for recent versions.

Keep the file under a few hundred lines. LLMs prioritize content above the ## Optional boundary, so place your most authoritative pages first.

Real llms.txt Examples

Example: Developer Conversion Tool

# file2markdown.ai
> A free online and API tool that converts PDFs, DOCX, XLSX, PPTX, images,
> and more into clean Markdown for AI pipelines, RAG workflows, and Obsidian.

## Core Converter
- [Convert any file to Markdown](https://www.file2markdown.ai/convert): Drag-and-drop converter supporting 10+ file formats.
- [PDF to Markdown](https://www.file2markdown.ai/convert/pdf-to-markdown): Preserves headings, tables, and lists from PDF files.
- [DOCX to Markdown](https://www.file2markdown.ai/convert/docx-to-markdown): Converts Word documents with full structure intact.
- [XLSX to Markdown](https://www.file2markdown.ai/convert/xlsx-to-markdown): Extracts spreadsheet data into Markdown tables.

## API
- [API Documentation](https://www.file2markdown.ai/api): REST API reference for automated document conversion.
- [Pricing](https://www.file2markdown.ai/pricing): Free tier and paid API plans.

## Optional
- [Blog](https://www.file2markdown.ai/blog): Guides on Markdown for RAG, AI agents, and document workflows.

Example: SaaS Analytics Platform

# Acme Analytics
> B2B analytics for e-commerce teams. Tracks revenue attribution,
> cohort retention, and LTV across sales channels.

## Product
- [Dashboard Overview](https://acme.example.com/docs/dashboard): How to read the main analytics dashboard.
- [Revenue Attribution](https://acme.example.com/docs/attribution): Methodology, data model, and channel definitions.

## Integrations
- [Shopify Integration](https://acme.example.com/docs/shopify): Connect your store in five minutes.
- [API Reference](https://acme.example.com/docs/api): REST API endpoints for custom data pulls.

## Optional
- [Help Center](https://acme.example.com/help): FAQs and troubleshooting guides.

How to Create Your llms.txt File from Existing Documentation

Most teams already have their documentation scattered across PDFs, Word files, or HTML pages. Here is a practical workflow for turning that content into a valid llms.txt file.

Step 1: Identify your highest-value pages

List the 10–20 pages that best represent your product: the introduction, quickstart guide, core feature explanations, and API reference. Skip thin content, duplicate pages, and purely promotional material.

Step 2: Convert existing source documents to Markdown

If your documentation lives in PDFs, internal DOCX files, or HTML exports, convert it to clean Markdown before embedding it in llms.txt (and especially before concatenating it into llms-full.txt).

Use the PDF to Markdown converter for research reports, whitepapers, and data sheets. Use the DOCX to Markdown converter for internal spec documents and guides. For documentation stored in spreadsheets, the XLSX to Markdown converter extracts tables into proper Markdown syntax.

This step is critical for llms-full.txt. Feeding an LLM raw PDF text means garbled tables, broken paragraph breaks, and embedded page numbers. Clean Markdown means the model reads accurate, structured content. See the guide to preparing documents for RAG pipelines for more detail on why Markdown quality matters at ingestion time.

Step 3: Write the llms.txt shell

Create a file named llms.txt in the public root of your project. Add the required H1 title, blockquote summary, and H2 sections. Each link entry should follow this pattern:

- [Page Title](https://yourdomain.com/path): One sentence describing what the page covers.

Use section headers that reflect how your product is organized — not generic labels like "Docs" but specific ones like "API Reference" or "Getting Started."

Step 4: Optionally build llms-full.txt

For developer documentation and API references, llms-full.txt gives AI agents access to your complete content in one fetch. Concatenate the full Markdown body of each page listed in your llms.txt, separated by --- dividers, and serve the result at /llms-full.txt.

For large document sets, the batch file conversion workflow lets you convert entire folder structures of PDFs and DOCX files to Markdown in one pass, making it straightforward to assemble a complete llms-full.txt.

Step 5: Serve and validate

Place llms.txt at your domain root so it loads at https://yourdomain.com/llms.txt as plain text. Verify that all URLs in the file resolve to live pages. Update the file whenever you add, remove, or significantly revise a core page — treat it like sitemap.xml, updated as part of your documentation deployment pipeline.

Frequently Asked Questions

What is the difference between llms.txt and llms-full.txt?

llms.txt is a curated index — a short Markdown file listing your most important pages with one-sentence descriptions. llms-full.txt contains the complete text of those pages concatenated together, allowing an AI agent to ingest your full documentation in a single request without following individual links.

Does Google use llms.txt for SEO?

Google has stated it does not use llms.txt for traditional web search indexing. The file is designed for LLMs operating at inference time — AI agents, chatbots, and retrieval-augmented systems — rather than for search crawlers. Implementing it does not replace standard SEO practices like proper sitemaps and structured data markup.

How often should I update my llms.txt file?

Update the file whenever you add, remove, or substantially change a key page. Many teams trigger a regeneration step as part of their documentation build so the file always reflects the current state of the site. If you are also maintaining llms-full.txt, that file will need to be regenerated whenever any source page changes.

Should I include every blog post in my llms.txt file?

Include only cornerstone posts with genuine reference value — a deep-dive on your API auth model belongs in llms.txt, while a product launch announcement does not. Use the ## Optional section for blog content so the LLM knows these pages are lower priority and can skip them when its context window is limited.