Godaddy hosted sites can now manage access to ai crawlers – Breaking News & Latest Updates 2026
Skip to main content
The homepageThe VergeThe Verge logo.
The homepageThe VergeThe Verge logo.
Posted
E
External Link
Emma Roth
GoDaddy-hosted sites can now manage access to AI crawlers.

The web hosting platform is working with Cloudflare to integrate the company’s AI Crawl Control tool, which lets publishers choose how web crawlers can access their site. Publishers can use the tool to permit or block bots, or ask them to pay.

Cloudflare began blocking AI crawlers by default last year.

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.
Comments
Loading comments
Getting the conversation ready...

Most Popular

Most Popular
  1. Sony’s PlayStation 5 is $200 off for the first time since December
  2. Anthropic’s most dangerous AI model just fell into the wrong hands
  3. Elon Musk admits that millions of Tesla vehicles won’t get unsupervised FSD
  4. The unraveling of Dan Crenshaw
  5. I bought Alienware’s $350 OLED monitor and I can’t believe how good it is

More in AI

THE PEOPLE DO NOT YEARN FOR AUTOMATIONTHE PEOPLE DO NOT YEARN FOR AUTOMATION
Play
You’re about to feel the AI money squeezeYou’re about to feel the AI money squeeze
Microsoft launches ‘vibe working’ in Word, Excel, and PowerPointMicrosoft launches ‘vibe working’ in Word, Excel, and PowerPoint
AI failure could trigger the next financial crisis, warns Elizabeth WarrenAI failure could trigger the next financial crisis, warns Elizabeth Warren
OpenAI now lets teams make custom bots that can do work on their ownOpenAI now lets teams make custom bots that can do work on their own
Watch Sony’s elite ping-pong robot beat top-ranked playersWatch Sony’s elite ping-pong robot beat top-ranked players

Top Stories