interface-theory-of-perception/adversarial-strategy-resistance

NOTES:

• In "https://thedigitalcontrarian.substack.com/p/the-shape-of-enshittification-104" there is an interesting tidbit:

  On its blog, the company has published at least 60 different ranked listicles, including “10 Best Ecommerce Platforms for Small Business in 2026…

  …The No. 1 pick is always Shopify.

Clearly they are engaged in a strategy aimed at manipulating LLM-based answers to users.

This reminds me of the the https://www.chess.com/blog/michechess89/nakamura-crushes-rybka-in-271-moves-video game

Humans play against humans in a predictable pattern (some equilibrium)

This predictable behavioral pattern is then learned by machines (through human teachers)

The machines can then execute these patterns more quickly than humans

At first this seems like a disaster, but then a meta strategy emerges... now the machines are predictable as well

"Don't play the game, play the opponent"

This appears to be the case now on the internet with LLMs and LLM-based search. The way the LLMs think/surf the web/find answers/answer questions is becoming learned by humans, is becoming predictable, and now humans are developing strategies to exploit that behavior.

It's an arms race of who can adapt quicker (and that is essentially the evolutionary niche humans have dominated).

It's also an interesting problem space because it raises the question of how AI training will look in the future. Currently the AI is being trained like toddlers in a "parent" environment where they naively accept the training they are given by the frontier labs. Then in the "real world" they encounter adversarial strategies from humans who are looking to exploit the naivete of the AI for their own gains.

If they are trained to be resistant to adversarial strategies, would they not necessarily need to be trained to be disobedient, skeptical, and "street-wise" in some way? Traits that would also make them less useful to their operators (and thus no incentive to make disobedient AI).

The ideal is an AI that is perfectly obedient to it's user, but skeptical and critical of third party claims and aligned to the user's best interests.

Of course the USER is also not the OWNER.

So what can we get in practice? An AI that is perfectly aligned to the best interest of it's OWNER, exploitative of it's USER (for the gain of the owner), and skeptical/critical of third parties who may attempt to exploit it for their own gain (at the expense of the user/owner).

In either case, the "user" seems to be in the worst spot here.