--- title: "Free AI Site Audit: Can AI Crawlers Read Your Site? – Probeli AI" description: "18 checks of how easily AI crawlers and agents can reach, read and use your website, plus technical and site-type checks. Free in every workspace." url: "https://probeli.ai/free-ai-site-audit" --- AI site audit · Free # Find what AI crawlers *can't read on your site.* 18 checks of how easily AI crawlers and agents can reach, read and use your website. *Free in every workspace, and it keeps working after the trial.* [Run your free audit](https://app.probeli.ai/signup) [Explore demo](https://app.probeli.ai/demo) The AI site audit for webflow.com as the example site: a score of 94 out of 100, rated Excellent and up 6 since the last audit, with 16 of 18 checks passed and nothing urgent; AI crawlers are allowed, there are no bot challenges and content is visible in the HTML, while structured data is the one high-priority fix. The checks ## Three groups of checks, *one score.* The 18 scored checks cover what AI crawlers meet on your site: can they reach it, read it and use it. *Each failed check comes with why it matters and how to fix it.* [Run your free audit](https://app.probeli.ai/signup) The AI site audit for webflow.com as the example site: a score of 94 out of 100, rated Excellent and up 6 since the last audit, with 16 of 18 checks passed and nothing urgent; AI crawlers are allowed, there are no bot challenges and content is visible in the HTML, while structured data is the one high-priority fix. ### Every check, pass or fail A score out of 100, with its change since the last audit. The AI site audit for webflow.com as the example site: a score of 94 out of 100, rated Excellent and up 6 since the last audit, with 16 of 18 checks passed and nothing urgent; AI crawlers are allowed, there are no bot challenges and content is visible in the HTML, while structured data is the one high-priority fix. ### Fixes, ranked In the order that helps your kind of site most, from urgent to low. - ### Content and structure Semantic HTML, headings, titles and descriptions, canonical links, alt text and structured data. - ### Discovery and crawlability AI crawler access in robots.txt, sitemaps and llms.txt, bot challenges and content that needs JavaScript. - ### Agent readiness Docs for machines, an OpenAPI file when you offer an API, and forms that agents can fill in. The crawler ## Your site, read the way *AI crawlers read it.* The audit reads up to 40 pages, homepage included, plus robots.txt, sitemaps and llms.txt. *It reads the HTML as a crawler gets it, so content that only appears with JavaScript is flagged.* [Meet ProbeliAIBot](https://probeli.ai/bot) - robots.txt - Sitemaps - llms.txt - Structured data - Canonical links - ### 7 technical checks Page weight, render-blocking files, images without set dimensions, the mobile viewport, hreflang, redirect chains and conflicting canonicals. - ### Checks for your kind of site A shop, a local business, software or a publisher, worked out from your site. You can correct it. - ### Sensitive topics For money, health or the law: expert authors, cited sources and disclaimers. When it runs ## Runs after setup, *again when you ask.* The first audit runs by itself once your workspace is set up. *Owners and admins can run it again right after a fix, then once a day, and everyone on the team sees the results.* - ### After setup Add your website and the first audit runs without you starting it. - ### Re-run after a fix One re-run right away, then one every 24 hours, so you can check what changed. - ### Shared with your team Every member sees the score, its change since the last audit and the fixes. ## Every part of Probeli AI - ### [AI visibility](https://probeli.ai/features/ai-visibility) Your prompts asked on ChatGPT, Gemini, Google AI Mode and Perplexity every day, every answer read. See AI visibility - ### [Keyword research](https://probeli.ai/features/keyword-research) What you and your competitors rank for, grouped into topics. See keyword research - ### [Calendar and Autopilot](https://probeli.ai/features/autopilot) Topics planned on days, written as drafts on their dates. See Autopilot - ### [AI articles](https://probeli.ai/features/ai-articles) Researched, sourced drafts that wait for your review. See AI articles - ### [Your pages](https://probeli.ai/features/your-pages) A report on each page you have, and suggestions to fix it. See Your pages - ### [Traffic](https://probeli.ai/features/traffic) Google clicks and AI visits next to each article's citations. See Traffic - ### AI site audit 18 checks of whether AI crawlers can read your site. Free. ## Questions about *the audit.* **What does the AI site audit check?** The AI site audit runs 18 checks of how easily AI search engines and agents can crawl, read and use your website. The checks come in three groups: content and structure (semantic HTML, titles, structured data), discovery and crawlability (robots.txt access for crawlers like OAI-SearchBot and PerplexityBot, sitemaps and llms.txt, bot challenges, JavaScript-only content) and developer and agent readiness. The audit also works out what kind of site you have (an online shop, a local business, a SaaS product, a publisher or blog) and whether it covers a sensitive topic such as money, health or the law, from your business brief and the pages it reads. It then adds the checks that matter for that kind of site, such as product markup and visible prices for a shop, your address, phone and opening hours for a local business, a pricing page and docs for software, named authors and dates for a publisher, and expert authors, sources and disclaimers for sensitive topics. Technical checks look at page weight and render-blocking files, the mobile viewport, language versions (hreflang), redirect chains and conflicting canonical links. An owner or admin can correct the detected category at any time. You get a score out of 100 for the core checks, and one list of fixes ranked by how much each helps your kind of site, each with why it matters and step-by-step instructions. The extra checks read only the pages the audit already fetches, so they don't add requests to your site. **Is the AI site audit free?** Yes. The audit is included in every workspace at no extra cost, and it keeps working after your trial ends, even without a plan. It needs a Probeli AI account and checks your workspace's own website. Owners and admins can run it: the first audit any time, one re-run right after, then one every 24 hours. An audit usually takes a few minutes, and everyone on your team can see the results. **Does Probeli AI crawl my website?** Yes, lightly, and only public pages. Probeli AI reads a site in four cases: its homepage when you add it as your website or as a competitor (and when you ask to read your own again in its settings, which also shows whether that read worked and why not), up to 40 of your pages (plus robots.txt, sitemaps and llms.txt) when you run the AI site audit, a few pages of your own website for its business brief, and pages that AI answers cite, to find the brands they name. For the brief, Probeli AI reads your homepage and up to 4 key pages, such as about, products or pricing, one at a time, and one workspace's brief doesn't read your site again within 10 minutes. Every request identifies itself as ProbeliAIBot. The audit, the brief and the cited-page reader follow robots.txt (the token is `probeliaibot`) for every request, sitemaps, llms.txt and redirects included, and limit how fast they send requests to each site. Only robots.txt itself and the homepage read of your own website when you enter it or ask to read it again aren't subject to it. If your robots.txt blocks ProbeliAIBot, the AI site audit runs as a limited audit. Our [ProbeliAIBot page](https://probeli.ai/bot) lists those requests and has the details. [See all questions](https://probeli.ai/faq#audit) ## Run your free audit, *then see what AI says.* Add your website and the audit runs once your workspace is set up. It stays free after the trial. [Run your free audit](https://app.probeli.ai/signup) [Explore demo](https://app.probeli.ai/demo) 7-day free trial, card required · Cancel anytime