What if it's just auto-complete

What if it’s just auto-complete?

There’s a never-ending stream of people claiming that LLMs have taste, judgement, and even consciousness.

But what if LLMs are just amazingly good auto-complete? How would we even know?

LLMs can pass the bar exam. But does that show they know the law, or just that they can generate the appropriate text response to specific prompts?

LLMs can write code and build entire applications. But does that show they understand software engineering, or just that they can generate the appropriate text response to specific prompts and specs?

Appearing intelligent isn’t the same as being intelligent.

In the 1980s, John Searle was asking these questions with his Chinese Room argument. Just because someone, or something, can follow a rule book to translate Chinese character inputs into responses, it doesn’t follow that they ‘know’ Chinese.

30 years earlier, Alan Turing was exploring the same problem. Evaluating intelligence is sufficiently difficult that it’s more useful to ask ‘can this machine imitate a human’ rather than ‘can this machine think’.

The epistemic curtain is real. But maybe some types of evidence might shed some light for us.

Would someone intelligent enough to pass the bar be able to correctly count the number of ‘r’s in the word ‘strawberry’? Would someone intelligent enough to write software tell us to walk, rather than drive, 100m to the carwash in order to wash our car?

So is it just auto-complete? Or are we the ones failing the Turing Test?