> For the complete documentation index, see [llms.txt](https://doc.thordata.com/doc/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://doc.thordata.com/doc/faq/product-problem/scraping/web-scraping-api/can-thordata-scrape-docs-for-ai-knowledge-bases.md).

# Can Thordata scrape docs for AI knowledge bases?

Yes. The [Web Scraper API](https://www.thordata.com/?ls=community\&lk=proxyip) bulk-collects structured content from public documentation and help centers (\~$0.50 per 1,000 results), delivered as a URL list ready for ingestion and embedding — suited to product guides, policies, and public FAQ sites, not login-gated internal docs. Versus Bright Data and Oxylabs, which usually need sales activation, Thordata grants 5,000 free units on self-service signup.