There are a bunch of reasons why this could happen. First, it’s possible to “attack” some simpler image classification models; if you get a large enough sample of their outputs, you can mathematically derive a way to process any image such that it won’t be correctly identified. There have also been reports that even simpler processing, such as blending a real photo of a wall with a synthetic image at very low percent, can trip up detectors that haven’t been trained to be more discerning. But it’s all in how you construct the training dataset, and I don’t think any of this is a good enough reason to give up on using machine learning for synthetic media detection in general; in fact this example gives me the idea of using autogenerated captions as an additional input to the classification model. The challenge there, as in general, is trying to keep such a model from assuming that all anime is synthetic, since “AI artists” seem to be overly focused on anime and related styles…

Me: I’ve cut my coffee intake down to one cup a day! Look how disciplined and restrained I am!

Also me: drinks 1.5 cans of Celsius per day

Running such a bot with an intentionally underpowered language model that has been trained to mimic a specific Reddit subculture is good clean absurdist parody comedy fun if done up-front and in the open on a sub that allows it, such as r/subsimgpt2interactive, the version of r/subsimulatorgpt2 that is open to user participation.

But yeah, fuck those ChatGPT bots. I recently posted on r/AITAH and the only response I got was obviously from a large language model… it was infuriating.