Cloudflare has teamed up with Microsoft to help brands and publishers adapt their websites for AI-driven search, launching a new integration that lets people, and AI agents, ask natural language questions and get instant answers.
The collaboration centres on two key technologies: Microsoft’s open protocol NLWeb and Cloudflare’s retrieval engine AutoRAG. Together, they allow businesses to index their websites in a way that supports conversational search, both for human users and for AI agents like ChatGPT, Claude, or Microsoft Copilot.
The move comes as consumer habits shift away from traditional keyword searches toward AI-powered tools that give direct answers to complex queries. According to Cloudflare, the new tools let businesses keep pace with this shift by making their own websites easily searchable through plain-language questions.
Why it matters for marketers and publishers
For brands, media companies and content-led marketers, the partnership presents a new opportunity to improve discoverability in what Cloudflare calls the “agentic web” – the emerging space where AI tools serve as gateways to online information.
“Together, NLWeb and AutoRAG let publishers go beyond search boxes, making conversational interfaces for websites simple to create and deploy. This integration will enable every website to easily become AI-ready for both people and trusted agents.” said R.V. Guha, creator of NLWeb and Technical Fellow at Microsoft.
AutoRAG does the heavy lifting: it crawls the website, stores content, and continuously updates it using semantic indexing. A Cloudflare Worker is then deployed to implement NLWeb, which adds a conversational interface at the site’s root and structured access for trusted AI agents via a new endpoint called MCP.
This setup means businesses can embed a ChatGPT-like experience directly onto their website, or make their content reliably accessible to AI assistants elsewhere – on their own terms.
What’s included
• Conversational UI: Users can ask natural language questions and get instant answers, with support for chat history and contextual follow-ups.
• MCP endpoint: AI agents can access content using a structured, standards-based protocol – no scraping required.
• Continuous indexing: AutoRAG automatically crawls and re-indexes the site, ensuring content is fresh and up-to-date.
• One-click deployment: Cloudflare customers can activate the integration by selecting their domain and hitting “Start indexing”.
New potential for owned and operated content
For digital publishers, the shift could help reset how audiences engage with content in owned environments. Joe Marchese, General and Build Partner at Human Ventures, said the technology offers a chance to move beyond the ad-challenged click economy.
“With NLWeb and AutoRAG, there is an opportunity to reset the nature of relationships with audiences for the better. More direct engagement on Publisher Owned and Operated (O&O) environments, where audiences value the brand and voice of the Publisher, means new potential for monetization. This would be the reset the entire industry needs.”
While large-scale implementation may take time, Cloudflare’s promise of a one-click deployment will likely appeal to media and marketing teams looking to future-proof their websites against a rapidly changing search landscape.
More information about the AutoRAG + NLWeb deployment is available via Cloudflare, with the company encouraging feedback from early adopters.