Make your site easy for AI search to find and cite
AI search visibility has four parts: a crawler can reach your page, the page is eligible to appear, an answer cites it, and someone follows the link.
GET /robots.txtUser-agent: *Allow: /User-agent: GPTBotDisallow: /Your robots.txt blocks some AI crawlersGET /llms.txtHTTP/2 404No usable llms.txt summary found for AI toolsThe free scan covers the setup side: your home page's markup, robots.txt and llms.txt. It doesn't measure mentions, citations or visits from AI answers.
What a scan can flag, and what AI tools decide
You control access and page setup; you can't choose the sources an answer cites.
A scan can flag
- Whether crawlers can reach your pagesThe free scan reads your robots.txt and llms.txt.
- How your home page is set upIt reads the markup crawlers get.
What AI tools decide
- Which sources an answer citesYou control access and page setup, not the sources an answer chooses.
- Mentions, citations and visitsThe free scan doesn't measure them. Google's Generative AI performance report shows its own.
Explore AI search access and page setup
Help AI search tools read your website
5 guides
Choose which AI crawlers may use your pages for training, search or user requests.
How to fix
- Open your site's robots.txt file and find the AI crawlers we list, the automated visitors AI companies use to read websites.
- Check each company's own documentation, because some of these crawlers feed search results while others collect content to train AI.
- Unblock only the ones you want reading your site, and keep blocks on uses you don't agree to, such as training.
- Then run the check again.
AI visibility
Your robots.txt blocks some AI crawlers
Your robots rules block AI crawlers from reading the site, which can keep it out of AI-driven search and citations.
Fix with your agent
# SiteCMD Fix Brief: Your robots.txt blocks some AI crawlers Site: https://yoursite.com ## What is wrong Your robots rules block AI crawlers from reading the site, which can keep it out of AI-driven search and citations. ## How to fix Review every surfaced user-agent token against the provider's current official documentation, separating search/discovery indexing, user-triggered retrieval, model training and data-use controls, and advertising validation, since similarly named tokens can have different effects. Allow only products the organization intentionally supports and preserve blocks that match legal, licensing, privacy, and commercial policy; do not bulk-enable all AI-labeled crawlers merely to clear the finding, and remember a robots rule is public crawl guidance, not authentication. ## Acceptance criteria After you finish, SiteCMD will re-run the check against https://yoursite.com. The fix is only accepted if that check passes. Make the smallest change that satisfies it.
Learn what goes in an llms.txt file and which AI tools read one.
How to fix
- Open your home page address followed by /llms.txt to see what's there, since some AI tools read this optional short summary of your site.
- If being found and cited by AI tools matters to you, write your site's name, what it covers, links to your most important pages, and how you'd like to be cited.
- Put that text in the file, or remove an empty file if you don't want one.
- Then run the check again.
AI visibility
No usable llms.txt summary found for AI tools
Your site's optional llms.txt file is empty, so it gives AI tools no summary to read.
Fix with your agent
# SiteCMD Fix Brief: No usable llms.txt summary found for AI tools Site: https://yoursite.com ## What is wrong Your site's optional llms.txt file is empty, so it gives AI tools no summary to read. ## How to fix Confirm that `/llms.txt` returns an empty text body; this check skips a missing file or an HTML rewrite rather than treating absence as a defect. If a supported client needs the endpoint, fill it with concise, accurate public information and links kept in sync with the site; otherwise remove the empty route. Re-fetch the deployed endpoint and test the client you target; file presence does not establish proposal conformance or client support. ## Acceptance criteria After you finish, SiteCMD will re-run the check against https://yoursite.com. The fix is only accepted if that check passes. Make the smallest change that satisfies it.
See what Google needs before a page can appear in AI answers, and what makes it worth citing.
How to fix
- Add something readers can learn from your experienceShare what you've found or done yourself, such as a test result or a worked example. Make clear which claims you've tested and which come from another source.
- Keep the answer readable and accurateOrganize the page around the question it answers. Update facts when they change, and make sure important information is available as text.
Read structured-data errors and identify the publisher and authors behind a page.
How to fix
- Decide whether this page is something search engines may show in a special way, like a product, event or article.
- If not, you can leave it as is, since this extra information, called structured data, is optional.
- If so, add structured data that matches what the page shows.
- Then run the check again.
AI visibility
No structured data for rich search results
This page carries no structured data, so it cannot be considered for rich search features that a similar page might get.
Fix with your agent
# SiteCMD Fix Brief: No structured data for rich search results Site: https://yoursite.com ## What is wrong This page carries no structured data, so it cannot be considered for rich search features that a similar page might get. ## How to fix First identify a current consumer or search feature that is useful for this exact page; structured data is optional, and generic markup added solely to clear the issue creates noise. If a supported use exists, add JSON-LD that accurately describes visible content, keep every value truthful and synchronized with the production page, and validate with Schema.org tooling. Valid markup enables consideration; it does not guarantee a search treatment. ## Acceptance criteria After you finish, SiteCMD will re-run the check against https://yoursite.com. The fix is only accepted if that check passes. Make the smallest change that satisfies it.
Find an empty app shell and review whether the server sends useful page content.
How to fix
- If this page is part of an app people log in to, rather than a public page you want found in search, it may not need changing.
- This needs a change to your site's code.
- Make the server send the page's main content ready to read, instead of building it only in the browser.
- Then run the check again.
AI visibility
Page may look nearly empty to search engines
The content that visitors see only appears after JavaScript runs, so the page search engines first fetch can look nearly empty.
Fix with your agent
# SiteCMD Fix Brief: Page may look nearly empty to search engines Site: https://yoursite.com ## What is wrong The content that visitors see only appears after JavaScript runs, so the page search engines first fetch can look nearly empty. ## How to fix Fetch the deployed URL while logged out and compare the raw response HTML with the rendered DOM; a mount container and a small source-text estimate do not prove the page is empty, and an intentionally client-only application may not need changes. If important public content is genuinely absent from the response, use the installed framework's current server-rendering, static-generation, or targeted prerendering path for that route, then re-fetch with JavaScript disabled to confirm content and metadata appear and hydration still behaves. ## Acceptance criteria After you finish, SiteCMD will re-run the check against https://yoursite.com. The fix is only accepted if that check passes. Make the smallest change that satisfies it.
Frequently asked questions
Does Google Search need llms.txt or special AI markup?
No. Google Search doesn't use llms.txt or special AI markup, so neither is required for its AI features. Google's AI optimization guide still recommends structured data for rich results.
How do I check my website's AI search visibility in Google?
Use the Generative AI performance report in Search Console. It shows how often links to your pages appeared in AI Overviews and AI Mode, and which pages appeared most.