Help AI tools and agents better understand your website by adding an llms.txt file to your project. llms.txt is a machine-readable Markdown file placed at the root of your website. It gives Large Language Models (LLMs) and AI agents a concise overview of your website, its purpose, and its most important content. This can make it easier for AI systems to discover and interpret the information you want to highlight.
Follow the steps below to create, add, and validate your llms.txt file.
Generate your llms.txt content
We recommend using the LLMrefs llms.txt Generator to create the initial Markdown content for your website. Open the tool, enter your website URL, and click Generate.
In the example below, we’re using https://www.rishikeshwalks.in/ as the reference website. If your website contains a large number of pages, the generation process may take a little longer.
LLMrefs will crawl your website and generate the llms.txt content. You’ll see a preview of the generated content in the left column and the raw Markdown version in the right column.
Review the generated content
Review it carefully to make sure it accurately represents your website.
You can:
- Add important pages that may have been missed.
- Improve or refine page descriptions.
- Remove irrelevant content.
- Reorganize sections to make your website easier for AI systems to understand.
- Make sure important information about your brand, products, or services is accurately represented.
This review is important because the llms.txt file becomes a concise reference point for AI systems discovering your website.
Copy the generated content
Once you’re satisfied with the generated content, click "Copy" at the bottom of the raw Markdown panel. The complete llms.txt content will be copied to your clipboard and is ready to add to your website project.
Add the llms.txt content to your project
Open your website project and navigate to: CMS → Project Settings → SEO. Locate the Llms.txt file field and paste the Markdown content you copied in the previous step. Click "Save Changes" to publish the file. Your website will now make the llms.txt file available at the root of your domain.
Validate your llms.txt file
Verify that the file is accessible at the root of your website. For example, the llms.txt file for our reference website is available at: https://www.rishikeshwalks.in/llms.txt.
Use a validation tool such as llmstxtchecker.net to check the file. Open the validator, enter the URL of your llms.txt file, and click "Validate".
Review the audit results to identify any errors or recommendations. If the validator reports an issue, return to Step 1, update the generated content as needed, and repeat the process until the file passes validation.
Your website now has an llms.txt file that provides AI systems with a structured overview of your website and helps them discover your key content more efficiently.