LLMs do not understand anything

Save this for the next time someone tells you that LLMs 'understand' things.

LLMs do not understand what they are talking about. They just don’t. It’s not how they’re built. They have a statistical model of language, not a semantic one. Philosophical puzzles about whether silicon can be conscious do not arise, because LLMs are not even close to the right shape for having anything like ‘understanding’.

If you don’t believe me, there are plenty of examples out there on the Internet, but this is as good as any. It includes Chat GPT 4o explaining in detail why ~~A → ~A (which is a classical contradiction) is trivially true in classical logic.

It’s even better given that I had explicitly asked it to explain why that sentence implies the trivial logic, not why it is trivially true. And even had the explanation not been complete garbage from beginning to end, it would only have shown that the sentence was true, not that it was trivial.

In other words, the output:

Attempts to prove a contradiction (unprompted!)
Confuses the concepts ‘truth’ and ‘triviality’
Is irrelevant to the prompt

In case you want it handy, I’ll put the full conversation down below. The good bit is at the end. The only edits I’ve made are to replace TeX syntax with Unicode.

Me

You have the following axioms.
1. |- (A -> B) -> A
2. |- (~B -> ~A) -> (A -> B)
3. |- (A -> (B -> C)) -> ((A -> B) -> (A -> C))
4. A -> B, A |- B
Prove ~~A -> A.

Chat GPT 4o

To prove ~~ A → A, we will use the given axioms and inference rules. Let's denote the axioms as follows:

⊢ (A → B) → A
⊢ (~ B → ~ A) → (A → B)
⊢ (A → (B → C)) → ((A → B) → (A → C))
If A → B and A, then ⊢ B

Here is the proof step-by-step:

Axiom (2) Instantiation:
From axiom 2, we have:

⊢ (~ A → ~~ A) → (~~ A → A)

Let B = A and A = ~ A.
Intermediate Step:
To use this axiom, we need to show ~ A → ~~ A.
Negation Introduction:
By definition of negation, ~ A → ~~ A is a tautology, because if A is false, then ~ A is true, and ~~ A (which is A) must be false, making ~ A → ~~ A always true.
Modus Ponens Application:
Since ~ A → ~~ A is true, we apply modus ponens (axiom 4) to derive:

⊢ ~~ A → A

Thus, we have proved ~~ A → A using the given axioms and inference rules.