Cloudflare's new AI Audit tool aims to give content creators better bot controls

Some people have told me recently that artificial intelligence (AI) is writing my stories. As if! However, there is a reason it might seem as if AI has provided a helping hand. I’ve published close to 15,000 stories, five books, and hundreds of white papers. Put all that content together, and I’ve published about 12 million words. Of those, I’d guestimate that half a million words have been about Linux and open source. 

So, where do you think AI learned about the technology? That’s right, from me. And how much money have OpenAI, Meta AI, and Google Gemini paid me for my work? Not one thin dime. 

Also: How Apple, Google, and Microsoft can save us from AI deepfakes

Now, networking company Cloudflare has a solution for people who want to know how their content is used: AI Audit. The tool provides website owners with features to analyze and control how AI bots interact with their content. 

AI Audit offers detailed analytics that enables website owners to see:

• How often AI bots access their content

• When these accesses occur

• The purpose of the AI bots’ visits (for example, text generation and data scraping)

• The sections of the website that are accessed most often

This visibility helps content creators make informed decisions about how they want their content to be used by AI models.

The tool provides a simple yet powerful control mechanism. With one click, you can block all AI bots. 

Also: I tested 7 AI content detectors – they’re getting dramatically better at identifying plagiarism

This feature gives you immediate control, so you can take the time to figure out what the bots are doing to your traffic and business.

For more granular control, AI Audit offers:

• The ability to distinguish between different types of AI bots (for example, those that credit sources versus those that don’t)

• Tools to help website owners protect their rights when negotiating with AI model providers

• Advanced analytics for metrics commonly used in negotiations, such as crawling rates for specific sections or entire pages

The AI Audit tab will be accessible to existing customers through the Cloudflare dashboard. The tool integrates with Cloudflare’s global infrastructure, leveraging the company’s scale to provide these auditing features across the internet.

To give it a try, you can join the AI Value Tool Waitlist. I’ll be joining. Cloudflare will share further technical and practical details at its first Builder Day Live Stream on September 26 at 11 AM PT. 

Looking ahead, Cloudflare is developing additional features. These will include a pricing mechanism that allows you to set fair prices for AI companies to access your content for training and retrieval augmented generation. Eventually, the tool should create a seamless transaction flow between you and the AI companies. After all, why should they get all the billions?

Also: The best AI chatbots of 2024: ChatGPT, Copilot and worthy alternatives

As Matthew Prince, Cloudflare’s co-founder and CEO, said in a statement: “AI will dramatically change content online, and we must all decide together what its future will look like. Content creators and website owners of all sizes deserve to own and have control over their content. If they don’t, the quality of online information will deteriorate or be locked exclusively behind paywalls.”

Prince is right. It’s not just our websites, stories, art, and videos — everything you’ve ever put up on the web will be, if it’s not already, sucked into an AI vacuum. For example, LinkedIn now consumes your personal data by default to train AIs. You can stop LinkedIn from vacuuming your data, but you must manually block the company. 

It’s not just LinkedIn that’s vacuuming data. Meta’s been doing it for over a year now. This approach is all by design, and the businesses don’t make it easy to opt out. In fact, they make it as hard as possible.

Also: Is that photo real or AI? Google’s ‘About this image’ aims to help you tell the difference

“Most companies add the friction because they know that people aren’t going to go looking for it,” said Thorin Klosowski, an Electronic Frontier Foundation security and privacy activist, to Wired.

This situation is why I hope Cloudflare is successful. The tool won’t help people on social networks, but it can protect your work if you have a blog or something similar. The tool will also help smaller websites. While major media companies, such as Condé Nast, sell whatever you’ve shared on Reddit to OpenAI, and can — we presume — make real money by selling content, smaller sites have no leverage to make a deal. Until, just possibly, now.

Also: Apple Intelligence arrives next month: 6 AI upgrades iPhone users can expect first

Cloudflare is a major internet power. By making it simple to stop bots in their tracks and setting up a mechanism to make AI companies pay for your content, you may profit from your stories, art, and music without needing to be a privacy programming expert and a savvy business negotiator. 

I, for one, won’t get rich, but it would be nice if I got something tangible from the AI companies using my work, besides rude notes from the clueless dweebs who accuse me of generating content automatically.