Rendered at 09:11:33 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
ChaitanyaSai 43 minutes ago [-]
Self Plug Alert!
For those interested in topic, our book Journey of the Mind would be a great introduction from a systems-thinking perspective. As you'll see on goodreads https://www.goodreads.com/en/book/show/60500189-journey-of-t... the reaction tends to be a bit bimodal, so at least some of you will find it absolutely fascinating.
We go from the simplest possible model of a mind (a single sensor and doer coupled together,separate from the world around them)to consciousness and beyond.
I am quoting a couple because I find it fascinating how readers have such widely differing reactions. (JoM actually does have an explanation for this: we are each a unique bundle of experiences cohering into an identity and a perspective)
"This book is both dense with biology and science as well as equally dense with a holistic philosophy that will genuinely shift your perspective on your place as a human, person and animal."
"Written in a cutesy, "fellow kids" tone that feels less like a serious exploration of the mind and more like someone trying to pander to what they imagine zoomers want. The style makes it almost unreadable and wastes your time. "
"We are all neurons in a (currently diseased) supermind. We are not just ourselves but collectively a Self, made of past and present (and artificial) intelligence. Collectively, a God that dictates the fate of other neurons, life forms, and ultimately, the universe at its whim. Without a doubt, this is one of the best books I have ever read."
cgio 2 hours ago [-]
My thinking, tangential to the temporal section, is an Implementation Trilemma, a CAP style theorem. An implementation map can be:
1. instantaneous and faithful, at the cost of depending upon substrate-specific persistence parameters;
2. faithful and substrate-neutral, at the cost of requiring historical information;
3. instantaneous and substrate-neutral, at the cost of losing faithfulness.
No implementation map can satisfy all three simultaneously.
To anyone who has ever built, designed or debugged physical hardware, the paper's "groundbreaking conclusion" is a mundane reality of physics that a state isn't practically a state unless it lasts long enough to be read.
cgio 43 minutes ago [-]
Exactly the intent of it! Does not claim inventing any physics, just relating it to implementation in philosophy. The graded theory is a bit more advanced on the physics front but not ready yet.
elendilm 6 minutes ago [-]
Curious about your paper (assuming you are the author). Clarify if I misunderstood.
Doesn't introducing separate time slices automatically forces you into a continuum of time and therefore the "Instantaneous" requirement "I" become meaningless.
One can't evaluate physical state transitions without continuous time or atleast some form of mathematical ordering.
But if you strip away continuous time and keep only the abstract "ordering", you are again back in the world of pure mathematics where states never decay and memory never fades.
timshell 8 hours ago [-]
This was the core premise behind my research during grad school and beyond. My favorite project — which ultimately became my dissertation — was a proposal to connect reinforcement learning algorithms, neuronal data, psychological behavior, and phenomenology. Specifically, the idea was the feeling of cognitive fatigue is a mathematical signal to ‘rest’, in which the value of rest is defined by offline mechanisms that contribute to decision-making. I proposed a connection to hippocampal replay, and was able to leverage fMRI and choice data to evaluate the hypothesis.
Zooming out of my specific work, the “best” success stories were connections across math/CS, neuro, and psych and the golden child is the dopamine model of reinforcement learning where firing patterns encode the TD prediction error
Is this a misunderstanding of what the double slit experiment actually means, or am I misinterpreting it? If this is based on the idea that reality changes strictly as a result of a conscious observer existing and it is somehow related to consciousness, etc, that is not accurate.
I learned this years ago from Philip Ball, one of the editors of Nature. There have been other scientists helping to clear up the confusion recently, though.
I am hopeful for a deterministic theory of the mind rather than the probabilistic inference in LLMs.
Then execute it as a program.
oleganza 2 hours ago [-]
It may very well turn out that probabilistic approximation would be 1000000x cheaper to compute than the deterministic precise theory.
Or even better, even formulating the deterministic theory would require 1000000x energy than approximation. And that's how'd we learn that biological minds do the same trick and skip the hard work :-)
elendilm 2 hours ago [-]
Or it may very well turn out to just be a next key lookup which is 100000x cheaper to compute than probabilistic inference.
The vast amounts of energy spends on generating text tokens while biology only consumes a fraction of it makes me hopeful.
My bet is on the deterministic model. When one appears, the whole Jenga tower of AI comes crashing down if everyone can do it dirt cheap.
dinunnob 2 hours ago [-]
"god does not play dice" was proven wrong a while ago ...
elendilm 2 hours ago [-]
Conflating biological mind with quantum mechanics is a category error.
dinunnob 2 hours ago [-]
* in so far as QM is natural, we evolved in its presence and so its probable that we are deeply integrated
Illegal character, no whimsy allowed, get his ass dang
cyanregiment 4 hours ago [-]
He's waiting til I hit 1k
He's gonna send Tom to do his dirty work
ohn03s 7 hours ago [-]
[dead]
randomImmigrant 6 hours ago [-]
I believe machines could be made that think. But not purely digital ones. The brain most certainly isn’t a thinking machine precisely because it doesn’t have the luxury of unlimited time. Its design is not based on executing best algorithms, but rather finding good enough solutions among many time constrained but interlinked signal cascades.
The brain and body instead construct time, a cohesive internal one,from the ground up, with nesting oscillators driving the sequence of all functions and behaviors, over long and short time closes.
What we call “thinking” is a filtered layer of it, a slice that the attention spotlight is focussed on, and in humans this process can be turned to cognition over multilayered abstractions, and using language or other symbolic communications to communicate some of the dynamics of this process, and even evoke them in others.
Turing machines, and modern digital computers with their Von Neumann bottleneck, and LLMs are outside of time. They do not experience time. They can check the clock. They can count steps. But they are not active and agentic in time. Till we change that we won’t make
machines that “think”. We can certainly make machines that simulate thinking, and that’s often good enough. But make the problem a real time one, with several conflicting verbal streams to parse and respond to serially and LLMs on their own fail catastrophically.
I think so long as the memory-processing divide remains, the simulation has to break apart processes for thinking in a way that’s alright for some cases, but not for others. And this is why we have the jaggedness of LLM intelligence I believe. They do not have internal timekeeping like almost all biological agents do.
hexer292 6 hours ago [-]
I'm probably way off the mark here, but my instinct tells me there's nothing intrinsic about the human condition that makes it the exclusive vessel of thought.
It is theoretically possible, after all, to simulate a human brain at a subatomic level. Sure, the complexity is absurd and interface required to measure it seems far fetched, but technically there's no restriction.
Because of this I believe what we know as "thought" or consciousness exists in the space between the physical world. It's just a way to represent the world around us, with the ability to reference the past and systems to predict the future.
LLMs to me are like a simplified version thats stretched across the domain of active memory. The "thought" process is the same, an active state (current token) guided by the past state (previous tokens).
I think the next step is to reduce what a token is. Right now they are vectors that are linked to words. If instead of words we fed it direct video and audio, and a way to translate the raw data into something resembling the information passed from our eyes and ears in real time, and train it like a child learns from its parents, it's hard to draw the line on where thought begins.
It's an interesting experiment, but I still consider it anti-human.
In the future, when we have "robot" rights for robots that identify as humans, you'll find me on the side of "they aren't a real human". We'll be called bigots, and I'm okay with that.
randomImmigrant 6 hours ago [-]
Made some updates to my post but glad you brought up simulation: today when we simulate the brain, at whatever resolution, we do so in digital systems that have the Von Neumann bottleneck. Processing and memory are separated.
In humans and all of biology, these things overlap instead. And that’s where simulations ultimately fail. They are out of step with real time in a way biology is not, because biology is attuned to the environment, and down to the molecular level, is actually enacting a physical process distributed across many cells that track the periodicities in various external signals, then anticipate them.
defrost 4 hours ago [-]
Does Danny Hillis' Connection Machine have a von Neumann bottleneck? Do transputer arrays separate processing and memory?
When custom silicon is made to speed up LLM behaviour doesn't that operate fully parallel with mixed memory and processing?
When real time remote sensing signal processing pipelines are built isn't it possible to parallel in the kinds of filters that "anticipate" the expected future (and highlight deviations from the expected).
randomImmigrant 4 hours ago [-]
Yes... when these things are done, we can talk. Which is why I said "I believe machines could be made that think". They can be. We just haven't done it yet.
defrost 4 hours ago [-]
> Yes.
To which question?
The Connection Machine CM-2 (1987) has a non von Neumann architecture and mixes memory and processing .. so that's been done, 40 years ago.
Anticipating the future in real time remote sensing signal processing pipelines has been about for at least [redacted] decades.
rustcleaner 5 hours ago [-]
>I'm probably way off the mark here, but my instinct tells me there's nothing intrinsic about the human condition that makes it the exclusive vessel of thought.
I'm in your boat. Molecular biology looks quite analogous to something like Conway's Game of Life, but a particle automaton with freedom in 3D space rather than cells in a fixed 2D aetheric grid.
Conway's Game of Life is Turing complete, the fundamental laws/algorithms are simple. Per Wolfram (NKS), you don't need complex algorithms in order to exhibit complex behavior. GoL is just four rules and a[n infinite] grid. Wolfram's automata are simpler.
I will go even further out 'there' and state that we are the equivalent to Greek "city-gods" which arise out of the polity/demos composing them, except our polities are little biological von Neumann machines acting in concert to emerge us in their aggregate. What most people think of as 'god' being some top-down omniscient/omnipotent creator entity, I think is wrong and reverses the true heirarchy of nature. The fabric of spacetime, the quantum field, or whatever emerges it from a lower level with its fixed simple rules, is the primary force driving the universe's [and our] existence. If there are gods, we are categorically much closer to them in form than the pious fanatics say!
>Its design is not based on executing best algorithms, but rather finding good enough solutions among many time constrained but interlinked signal cascades.
That's known as optimization algorithm, and genetic algorithm is known to produce good enough results fast https://en.wikipedia.org/wiki/Evolved_antenna
Computers are time constrained too and cut such corners just like humans.
> They do not experience time. They can check the clock.
Humans experience time because they check biological clock. And sometimes humans forget about time just fine, when they don't check the clock.
DiogenesKynikos 44 minutes ago [-]
Machines already do think.
Any reasonable definition of "thinking" includes what LLMs do. One would have to craft a rather arbitrary and tortured definition of "thinking" to include human thinking but exclude LLM thinking.
DiscourseFan 4 hours ago [-]
Alright Heidegger
RussianBot9580 4 hours ago [-]
This is basically Christian apologetics. We've collectively thought we were special for so long that must now go through increasingly complex mental gymnastics to prove to ourselves that we are somehow uniquely sentient, despite increasing evidence to the contrary.
randomImmigrant 4 hours ago [-]
This is neuroscience. I don't give a damn about Christianity. And nothing here says humans are special. Only that biology broadly is different than computing.
elendilm 3 hours ago [-]
Yet, evolution, the foundation of modern biology is easily simulated by any decent coder in 10 minutes - lookup weasel program.
The audacity to still claim biology is broadly different than computing is commendable.
ChrisGreenHeur 3 hours ago [-]
Evolution is very far from just being natural selection. It’s a quite complicated field with many mechanisms. And when simulated on a computer we have not gotten anything similar to real biology yet. So we have more to learn.
elendilm 2 hours ago [-]
No. Only 3 mechanics: replication, mutation and natural selection.
Write a weasel program and you will see for yourself.
current = "XXXXXXXXXXXXXXXXXXXXX"
target = "THE LORD OF THE RINGS"
loop {
replicas = replication(current)
replicas = mutation(replicas)
current = naturalSelection(replicas)
print("generation", i, current)
if (current == target)
break
}
You can thus make "Dog" turn into a "Cat". I rest my case :)
lioeters 14 minutes ago [-]
There have been experiments in "digital primordial soup", where primitive programs are cultivated under conditions with evolutionary pressure.
Computational Substrates: How Well-formed, Self-replicating Programs Emerge from Simple Interaction - https://arxiv.org/abs/2406.19108
Brainfuck, Forth, Subleq, and most recently Z80 assembly programs with mutations, interactions, and self-modification. Z80 has an LDIR instruction that copies a block of data from one location in memory to another, which allows the "organisms" to reach/discover reproduction quicker and more reliably in repeated experiments. As soon as that happens, it inevitably takes over the "world".
Hm, sounds more like "Lord of the Flies" than Rings.
DiscourseFan 4 hours ago [-]
I agree that it's a bit reductionist to boil this down entirely to "Christian Apologetics," I think you'd have to do some serious rationalizations to see how the notion of human finitude, time as the limit of cognition, is not originally a Christian notion, or at least is not informed by secular interpretations of theology such as the first place where this claim is made, Being and Time. Which is even more interesting because even Heidegger moved away from time as binding finitude and eventually came to the position that cybernetics would replace philosophy, that fundamental ontology would not withstand technological advancement.
firasd 9 hours ago [-]
Doesn’t the concept of a subconscious, or inner voice, invalidate any tight analogies to Turing Machines? What logical process is putting the Fur Elise into my head (earworms etc)
I think this article is right that definitions really matter here cause one could argue “CTM is trivial, because we can describe almost any physical system as executing computations.” So we have a dual problem here—not just the difficult discussion of ‘what’s going on in minds’, but also ‘what is computation’.
BobbyTables2 9 hours ago [-]
Wonder if the subconscious could simply be another LLM with a higher temperature setting…
Have long thought that creativity is little more than a finely tuned filter driven by noise…
Are LLMs even covered by ‘computational theory of mind’? Cause if I say “I like to bake” the LLM is only able to recognize and respond to that sentence due to the neural network making emergent connections between the tokens in the words—very different from the kind of computer programs these guys seem to have been thinking about.
timshell 7 hours ago [-]
I think many cognitive scientists think language is a core part of human cognition, and that LLMs were a major advance in AI/cognitive science due to connecting AI algorithms with language production, but LLMs aren’t considered (or even aiming) to be accurate models of human cognition
TheOtherHobbes 8 hours ago [-]
Infuriatingly, computation is subjective. The experience of the inputs, the results, and the processes depends on qualia. So does logic itself.
Automating the processes doesn't change that. There's a very large number of possible processes and operations, but only a small number are perceived as valuable.
overmann 3 hours ago [-]
Here's a new, unconventional approach to building a Computational Theory of Mind:
i think it's a rite of passage to have attempted encoding thinking and the scientific process for AI/ML researchers
dinunnob 3 hours ago [-]
omg welcome philosophers trying to be noticed by frontier labs, lets see if you can set everyone straight (I point to Einstein deliberately looking to philosophy (Spinozian & Machian) to guide his views - and he did all right.)
Just as the amplitudes program suggests that spacetime itself may be an emergent structure rather than the fundamental substrate of reality [1], the computational theory of mind may be mistaking an emergent representation for the source of consciousness. A more fundamental theory must therefore treat neural computation as an emergent interface within conscious-agent relations, rather than as the mechanism that generates consciousness.
Even the loosest, "multiply realizable" version of computationalism this entry lays out still needs something physical to run on like neurons, silicon, spacetime, whatever. Problem is, natural selection was never selecting for accurate perception of that substrate, just for fitness payoffs [2].
So computationalism has the order backwards: you can't generate a mind by computing over a brain when the brain, like any object in spacetime, is just a species-specific icon evolution gave you to see, not the hardware underneath it.
I can prove things are the way they are outside my mind's perception of them, through the use of extra sensory devices.
Even in a Donald Hoffman esque icon:hardware situation - and I think he said - "I don't take the icon literally, but I do take it seriously" meaning that even though it's an illusion, it still performs the function, I have trouble with mind preceding matter.
It's the same trouble I have with human-like intelligence preceding worlds.
People come out of worlds, not the other way around.
For those interested in topic, our book Journey of the Mind would be a great introduction from a systems-thinking perspective. As you'll see on goodreads https://www.goodreads.com/en/book/show/60500189-journey-of-t... the reaction tends to be a bit bimodal, so at least some of you will find it absolutely fascinating.
We go from the simplest possible model of a mind (a single sensor and doer coupled together,separate from the world around them)to consciousness and beyond.
I am quoting a couple because I find it fascinating how readers have such widely differing reactions. (JoM actually does have an explanation for this: we are each a unique bundle of experiences cohering into an identity and a perspective)
"This book is both dense with biology and science as well as equally dense with a holistic philosophy that will genuinely shift your perspective on your place as a human, person and animal."
"Written in a cutesy, "fellow kids" tone that feels less like a serious exploration of the mind and more like someone trying to pander to what they imagine zoomers want. The style makes it almost unreadable and wastes your time. "
"We are all neurons in a (currently diseased) supermind. We are not just ourselves but collectively a Self, made of past and present (and artificial) intelligence. Collectively, a God that dictates the fate of other neurons, life forms, and ultimately, the universe at its whim. Without a doubt, this is one of the best books I have ever read."
1. instantaneous and faithful, at the cost of depending upon substrate-specific persistence parameters;
2. faithful and substrate-neutral, at the cost of requiring historical information;
3. instantaneous and substrate-neutral, at the cost of losing faithfulness.
No implementation map can satisfy all three simultaneously.
https://philarchive.org/rec/GIOTIT
Doesn't introducing separate time slices automatically forces you into a continuum of time and therefore the "Instantaneous" requirement "I" become meaningless.
One can't evaluate physical state transitions without continuous time or atleast some form of mathematical ordering.
But if you strip away continuous time and keep only the abstract "ordering", you are again back in the world of pure mathematics where states never decay and memory never fades.
Zooming out of my specific work, the “best” success stories were connections across math/CS, neuro, and psych and the golden child is the dopamine model of reinforcement learning where firing patterns encode the TD prediction error
Conscious agents are creating spacetime. Not the other way around: https://pmc.ncbi.nlm.nih.gov/articles/PMC4060643/
I learned this years ago from Philip Ball, one of the editors of Nature. There have been other scientists helping to clear up the confusion recently, though.
Here is Sean Carroll explaining it: https://www.youtube.com/watch?v=RNOJV4G7Ips
Here is Sabine Hossenfelder explaining it: https://www.youtube.com/watch?v=npc6Mn2CZV8
Philip Ball explained it in Beyond Weird: https://www.audible.com/pd/Beyond-Weird-Audiobook/B07L5XK4BL
https://hn.algolia.com/?q=https%3A%2F%2Fplato.stanford.edu%2...
Then execute it as a program.
Or even better, even formulating the deterministic theory would require 1000000x energy than approximation. And that's how'd we learn that biological minds do the same trick and skip the hard work :-)
The vast amounts of energy spends on generating text tokens while biology only consumes a fraction of it makes me hopeful.
My bet is on the deterministic model. When one appears, the whole Jenga tower of AI comes crashing down if everyone can do it dirt cheap.
He's gonna send Tom to do his dirty work
The brain and body instead construct time, a cohesive internal one,from the ground up, with nesting oscillators driving the sequence of all functions and behaviors, over long and short time closes.
What we call “thinking” is a filtered layer of it, a slice that the attention spotlight is focussed on, and in humans this process can be turned to cognition over multilayered abstractions, and using language or other symbolic communications to communicate some of the dynamics of this process, and even evoke them in others.
Turing machines, and modern digital computers with their Von Neumann bottleneck, and LLMs are outside of time. They do not experience time. They can check the clock. They can count steps. But they are not active and agentic in time. Till we change that we won’t make machines that “think”. We can certainly make machines that simulate thinking, and that’s often good enough. But make the problem a real time one, with several conflicting verbal streams to parse and respond to serially and LLMs on their own fail catastrophically.
I think so long as the memory-processing divide remains, the simulation has to break apart processes for thinking in a way that’s alright for some cases, but not for others. And this is why we have the jaggedness of LLM intelligence I believe. They do not have internal timekeeping like almost all biological agents do.
It is theoretically possible, after all, to simulate a human brain at a subatomic level. Sure, the complexity is absurd and interface required to measure it seems far fetched, but technically there's no restriction.
Because of this I believe what we know as "thought" or consciousness exists in the space between the physical world. It's just a way to represent the world around us, with the ability to reference the past and systems to predict the future.
LLMs to me are like a simplified version thats stretched across the domain of active memory. The "thought" process is the same, an active state (current token) guided by the past state (previous tokens).
I think the next step is to reduce what a token is. Right now they are vectors that are linked to words. If instead of words we fed it direct video and audio, and a way to translate the raw data into something resembling the information passed from our eyes and ears in real time, and train it like a child learns from its parents, it's hard to draw the line on where thought begins.
It's an interesting experiment, but I still consider it anti-human.
In the future, when we have "robot" rights for robots that identify as humans, you'll find me on the side of "they aren't a real human". We'll be called bigots, and I'm okay with that.
When custom silicon is made to speed up LLM behaviour doesn't that operate fully parallel with mixed memory and processing?
When real time remote sensing signal processing pipelines are built isn't it possible to parallel in the kinds of filters that "anticipate" the expected future (and highlight deviations from the expected).
To which question?
The Connection Machine CM-2 (1987) has a non von Neumann architecture and mixes memory and processing .. so that's been done, 40 years ago.
Anticipating the future in real time remote sensing signal processing pipelines has been about for at least [redacted] decades.
I'm in your boat. Molecular biology looks quite analogous to something like Conway's Game of Life, but a particle automaton with freedom in 3D space rather than cells in a fixed 2D aetheric grid.
Conway's Game of Life is Turing complete, the fundamental laws/algorithms are simple. Per Wolfram (NKS), you don't need complex algorithms in order to exhibit complex behavior. GoL is just four rules and a[n infinite] grid. Wolfram's automata are simpler.
I will go even further out 'there' and state that we are the equivalent to Greek "city-gods" which arise out of the polity/demos composing them, except our polities are little biological von Neumann machines acting in concert to emerge us in their aggregate. What most people think of as 'god' being some top-down omniscient/omnipotent creator entity, I think is wrong and reverses the true heirarchy of nature. The fabric of spacetime, the quantum field, or whatever emerges it from a lower level with its fixed simple rules, is the primary force driving the universe's [and our] existence. If there are gods, we are categorically much closer to them in form than the pious fanatics say!
(Disclosure: I am petulantly Atheist.)
https://en.wikipedia.org/wiki/Mathematical_universe_hypothes...
That's known as optimization algorithm, and genetic algorithm is known to produce good enough results fast https://en.wikipedia.org/wiki/Evolved_antenna Computers are time constrained too and cut such corners just like humans.
> They do not experience time. They can check the clock.
Humans experience time because they check biological clock. And sometimes humans forget about time just fine, when they don't check the clock.
Any reasonable definition of "thinking" includes what LLMs do. One would have to craft a rather arbitrary and tortured definition of "thinking" to include human thinking but exclude LLM thinking.
The audacity to still claim biology is broadly different than computing is commendable.
Write a weasel program and you will see for yourself.
current = "XXXXXXXXXXXXXXXXXXXXX"
target = "THE LORD OF THE RINGS"
loop {
}You can thus make "Dog" turn into a "Cat". I rest my case :)
Computational Substrates: How Well-formed, Self-replicating Programs Emerge from Simple Interaction - https://arxiv.org/abs/2406.19108
Brainfuck, Forth, Subleq, and most recently Z80 assembly programs with mutations, interactions, and self-modification. Z80 has an LDIR instruction that copies a block of data from one location in memory to another, which allows the "organisms" to reach/discover reproduction quicker and more reliably in repeated experiments. As soon as that happens, it inevitably takes over the "world".
Hm, sounds more like "Lord of the Flies" than Rings.
I think this article is right that definitions really matter here cause one could argue “CTM is trivial, because we can describe almost any physical system as executing computations.” So we have a dual problem here—not just the difficult discussion of ‘what’s going on in minds’, but also ‘what is computation’.
Have long thought that creativity is little more than a finely tuned filter driven by noise…
What are your thoughts on this?
Automating the processes doesn't change that. There's a very large number of possible processes and operations, but only a small number are perceived as valuable.
https://creatingintelligence.org
Even the loosest, "multiply realizable" version of computationalism this entry lays out still needs something physical to run on like neurons, silicon, spacetime, whatever. Problem is, natural selection was never selecting for accurate perception of that substrate, just for fitness payoffs [2].
So computationalism has the order backwards: you can't generate a mind by computing over a brain when the brain, like any object in spacetime, is just a species-specific icon evolution gave you to see, not the hardware underneath it.
[1] https://vod.video.cornell.edu/media/1_sfv907qp
[2] https://pmc.ncbi.nlm.nih.gov/articles/PMC4060643
Even in a Donald Hoffman esque icon:hardware situation - and I think he said - "I don't take the icon literally, but I do take it seriously" meaning that even though it's an illusion, it still performs the function, I have trouble with mind preceding matter.
It's the same trouble I have with human-like intelligence preceding worlds.
People come out of worlds, not the other way around.