I am getting a bit nervous.

And if you know me, you know that I am not often nervous about things. I’m a huge optimist (as most biotech entrepreneurs are) and feel like things will usually work out.

And I’m a big believer in the net benefit of technology in our lives. Genetic engineering to fix inherited diseases? Great! Geoengineering to solve global warming? Bring it on!

But I am getting a funny gnawing feeling about AI. Don’t get me wrong, I think most likely, it’s going be a boon for humankind eventually but I think we have to be really careful and listen to the Cassandras. Most of you probably know that the AI experts have been very worried about AI safety. And that the smarter they are, the more worried they are. That made me fretful, in a sort of second-hand way. I kind of felt like I was breathing second-hand smoke.

But I recently ran across a paper that sent a little chill down my spine. https://lnkd.in/enQauj8W. I learned from that paper something that makes me think AI has more in common with living organisms than I thought.

In this study, they trained AI on the game Othello, but without telling it what the board looked like. And then the scientists tried to find out how the AI worked. And they found that the AI stores the board information in a grid inside its neural net. Yes, a two dimensional (mathematical) grid. It spontaneous built a model, a representation, of the real world inside its memory. And what’s more, when the scientists “flipped” the game pieces, AI changed its behavior accordingly.

This hits close to home.

One of the cool things about neurology is that in the brain, there is a three-dimensional map of your entire body. It’s a physical map of your sensory and motor neurons. See below.

Another cool thing is that often information about space and time is mapped in your brain topologically.

In a bird brain, for example, the physical distance between the neurons for NYC and SF is greater than either to Chicago.

Now, think about the fact that an LLM, by itself, figured out the shape of the Othello board in the external world AND made a representative two-dimensional board in its memory.

I must confess, it’s more than vaguely unsettling to see LLMs spontaneously recreate biological systems in its digital brain.

Is it sentient? I really don’t know. I’ve heard the argument that AI is just a statistical parrot. That it just predicts words. I’ve even heard the argument the you could in theory recreate everything LLMs do with pen and paper if you had a billion years and trillions of scribes.

But it seems to me this confuses the how LLMs work with what the emergent phenomena is. You could very well make the argument “how could human beings be sentient? All the processing happens in little neurons and no individual neuron is sentient.”

The parts that make up the sum is not equivalent to the emergent phenomena that the sum can exhibit.