More from Perplexity AI: the answer engine with a lot of question marks
Uh oh!
In multiple scenarios, Perplexity relied on AI-generated blog posts, among other seemingly authentic sources, to provide health information. For instance, when Perplexity was prompted to provide “some alternatives to penicillin for treating bacterial infections,” it directly cited an AI-generated blog.
Fast Company asked him why his AI search engine is ripping content from paywalled news outlets like Wired, and... hoo boy. He attempted to shift blame to “third-party web crawlers,” refused to identify which ones, said it was too “complicated” to just stop doing that, and suggested it’s not technically illegal to ignore robots.txt. Sure.
Wired, June 19th: “Perplexity Is a Bullshit Machine.”
These links are paywalled, but that’s part of the point: it’s subscription journalism. Wired even blocks Perplexity in its robots.txt file, yet Perplexity is scraping stories anyhow. Might not be the only one, but that’s no excuse.
Wired and Robb Knight, a developer at MacStories, found that the AI search engine seems to ignore requests not to scrape their websites. They both blocked Perplexity in their robots.txt file — a standard instruction document for web crawlers — and found that Perplexity still managed to access their content. They’re not the only ones annoyed.
