Why I decided to no longer allow Pangram/Substack to scan my posts for AI

The Twelve Inch

The simple answer?

Because it doesn’t work.

Whenever a platform I’m active on introduces a new tech feature, I want to try it. I want to know how it works. So when Substack introduced the Pangram AI detection feature, I was curious. Of course I wanted to see what my own “report card” would look like.

I’ve always been very open about my use of AI.

I’m not a native English speaker. AI chatbots have been a wonderful addition to my workflow because they allow me to make sure that what I publish is grammatically correct. I’ll go even further than that. The fact that these tools became available is one of the reasons I finally decided to take the leap and start writing and publishing on this platform.

Apart from grammar checks, ChatGPT or Claude will sometimes advise me on the structure of a post. Advice I sometimes follow and sometimes completely ignore. 😁

And even when I do follow it, I invariably end up changing things again because I want the text to sound and feel like me. It is, after all, my text. I have to be pleased with it, not the chatbot.

So when AI scanning became available, I tried it on the Marvin Gaye episode I had just published.

The result?

84% human. 16% AI generated.

Oumph.

I didn’t see that one coming.

But hey, the article was already published. I wasn’t going to rewrite it because an AI detector had decided that 16% of me apparently wasn’t me.

Then I scanned the B-Side.

48% human. 52% AI generated.

Now that was strange.

Because I wrote both articles exactly the same way I always do.

For the next episode, on the Pet Shop Boys, I decided to change a few details. I left out the emojis, for instance, and made some other small adjustments.

And guess what?

Hooray!

100% human.

Even the B-Side came back 100% human.

The nightmare was over. The pressure was gone.

The questions, however, remained.

How can changing a few details produce such wildly different results?

A text is either written by a human or generated by AI. And I’m pretty sure that in most cases you don’t need an AI machine to recognise AI slop anyway.

You see it.

You feel it.

But the moment you introduce a scanning system like this, something else happens. You put pressure on the writer.

Let’s try to avoid that sentence.

Maybe I shouldn’t structure it like this.

Perhaps I should change that paragraph.

In other words: let’s colour between the lines.

Because what will readers think when they see that 16% of your article has been marked as AI generated?

Let alone 52%.

Will they think I’m a fake? An imposter? Someone pretending to write something that was actually produced by a machine?

And that is where a supposedly helpful tool starts influencing the very thing it claims to protect.

Let me be clear about something.

I absolutely share the worries about AI slop and the way it is trying to take over large parts of the internet. I’ve worked in the music industry my entire professional life, so believe me, you don’t have to convince me that there is a serious problem.

But AI is not a subject that fits comfortably into a black-and-white discussion.

AI is multilayered.

You can use it as a helpful tool. You can use it to correct grammar, challenge your structure, translate something or help you think.

And yes, you can also use it to cheat.

The fact of the matter is that AI is here to stay, whatever scanning tools become available.

I’m a glass-half-full kind of person, so I’m convinced that the opportunities for people flooding platforms with AI slop will eventually become much smaller. You can already see this happening in the music industry. Uploading endless amounts of AI-generated music to DSPs is becoming more difficult. Rules are changing. AI-generated material is being treated differently by charts and platforms.

Take away the financial incentive and much of the industrial production of slop will disappear with it.

You don’t necessarily need a scanning device for that.

But there is something else about this whole discussion on Substack that sits even more uncomfortably with me.

And that is the way the platform has introduced this feature and is using it.

If you go into your settings, you have the possibility to:

“Tell AI tools not to train their models on your content.”

And there we find something interesting, because Substack also states:

“Substack never uses your content to train.”

And yet, here we are.

Because there isn’t a chatbot or AI detection system in existence that can analyse your text without first ingesting that text in some form. It has to process the content before it can tell you whether it believes that content was written by AI.

That is exactly what is happening here.

Even more importantly, my content is being passed to an external company whose service I never personally chose to use.

I don’t know that company.

I don’t know what happens to my content once it is processed.

And nobody ever asked me for permission.

The scanning option is enabled by default. I can deactivate it on a post-by-post basis, but that creates another problem.

Because what does switching it off suggest?

Why did he disable the AI scan?

Does he have something to hide?

So whichever way you turn it, Substack wins.

And then came this week’s B-Side.

When I finished writing it, I scanned it.

100% human.

As it should be.

Then I added an introduction, some pictures and a video.

I scanned it again.

Suddenly:

48% AI generated.

Wait... what?

So I removed the photos and the video and returned to the original version that Pangram had just declared 100% human.

I scanned exactly that version again.

54% AI generated.

Enough.

Seriously.

Writing is supposed to be fun.

It shouldn’t become a nerve-racking exercise in trying to satisfy an algorithm whose verdict can apparently change from one moment to the next.

And it certainly shouldn’t make writers wonder whether they need to change their own writing style simply to remain within some invisible lines drawn by another AI.

So starting with this week’s B-Side, the first episode in my Canadian disco series, I’m switching the AI scanning option off.

I realise I can’t completely stop platforms or automated systems from processing content once it is online. Sadly enough, that ship probably sailed a long time ago.

But I can stop participating in this particular game.

If a system can tell me that my own writing is 100% human, then 48% AI, then 100% human again and finally 54% AI, without me fundamentally changing the text, it isn’t providing useful information.

It’s creating noise.

Worse, it’s creating suspicion.

And I’m not going to change the way I write because an unreliable machine is trying to determine whether another machine wrote it.

I know who wrote it.

That should be enough.

---