PEG

Whilst cruising the interwebs I can across the follow, which nicely captures the limits of large language (LLMs) models.

In the persuasive practice of Derrida, Paul de Man and others, it took language not as a reminder of secret structure but as the home of a recurring crisis of meaning, a place where interpretation learned that it could never end. It did not hold, as many of its detractors thought it did, that there was no reality apart from language, and it’s wrong to translate Derrida’s famous ‘Il n’y a pas de hors-texte’ as ‘there is nothing outside the text.’ A hors-texte is an unnumbered page in a printed book. Derrida is saying that even the unnumbered pages count, just as an outlaw, in French an hors-la-loi, has everything to do with the law, since it makes him what he is. More crudely, we might say that interpretation is theoretically endless, but this claim itself needs interpretation. Endless is not the same as pointless; and what is endless in theory is often stopped easily enough in practice. We may think – I do think – that the reasons for stopping are usually more interesting than the empty possibility of going on for ever, although then it would be worth asking whether those reasons are practical or theoretical.
Wood, Michael. 2016. “We Do It All the Time.” London Review of Books, February 4. https://www.lrb.co.uk/the-paper/v38/n03/michael-wood/we-do-it-all-the-time.

Not a week seems to pass by without some surprising news concerning large-language models (LLMs). Most recently it was when an LLM trained for other purposes played chess at a reasonable level. This seemingly constant stream of surprising news has led to talk that LLMs are the next general-purpose technology—a technology that affects an entire economy—and will usher in new era of rapid productivity growth. They might even accelerate global economic growth by an order of magnitude, as the Industrial Revolution did, providing us with a Fifth Industrial Revolution.

The short answer is that AGI is a bit meaningless as a term as we don’t really know what ‘I’ is. The best approach is the assume that AGI means “human-like intelligence in machines” while accepting that “human-like intelligence” is something of an unknown.