Kind: captions
Language: en
you know I can tell chat gbt create a
piece of code and then just run it on my
computer and I'm like you know that that
sort of personalizes for me the what
could what could possibly go wrong so to
speak was that exciting or scary that
possibility it was a little bit scary
actually because it's kind of like if
you do that right what is the sandboxing
that you should have and that's sort of
a that's a a version of of that question
for the world that is as soon as you put
the AIS in charge of things you know how
much how many constraints should there
be on these systems before you put the
AIS in charge of all the weapons and all
these you know all these different kinds
of systems well here's the fun part
about sandboxes is uh the AI knows about
them it has the tools to uh crack them
the following is a conversation with
Stephen Wolfram his fourth time on this
podcast he's a computer scientist
mathematician theoretical physicist and
the founder of Wolfram research a
company behind Mathematica well from
alpha or from language and the Wolfram
physics and meta mathematics projects He
has been a Pioneer in exploring the
computational nature of reality and so
he's the perfect person to explore with
together the new quickly evolving
landscape of large language models as
human civilization Journeys towards
building super intelligent AGI
this is the Lex Friedman podcast to
support it please check out our sponsors
in the description and now dear friends
here's Stephen Wolfram
you've announced the integration of chat
gbt and Wu from Alpha and Wolfram
language so let's talk about that
integration what are the key differences
from the high philosophical level
maybe the technical level between the
capabilities of
broadly speaking the two kinds of
systems large language models and this
computational gigantic computational
system infrastructure that is well from
alpha yeah so what does something like
chat GPT do it's it's mostly focused on
make language like the language that
humans have made and put on the web and
so on yeah so you know it's it's primary
sort of underlying technical thing is
you've given a prompt it's trying to
continue that prompt in a way that
somehow typical of what it's seen based
on a trillion words of text that humans
have written on the web and the way it's
doing that is with something which is
probably quite similar to the way we
humans do the first stages of that using
a neural net and so on and just saying
given these given this piece of text
let's Ripple through the neural net one
word and get one word at a time of
output and uh it's kind of a shallow
computation on a large amount of kind of
training data that is what we humans
have put on the web that's a different
thing from sort of the computational
stack that I spent the last I don't know
40 years or so building which has to do
with what can you compute many steps
potentially a very deep computation it's
not sort of taking the statistics of
what we humans have produced and trying
to continue Things based on that
statistics instead it's trying to take
kind of the formal structure that we've
created in our civilization whether it's
from mathematics or whether it's from
kind of systematic knowledge of all
kinds and use that to do arbitrarily
deep computations to figure out things
that that aren't just let's match what's
already been kind of said on the web but
let's potentially be able to compute
something new and different that's never
been computed before so as a practical
matter you know the the um what we're
you know the our goal is to have made as
much as possible of the world computable
in the sense that if there's a question
that in principle is answerable from
some sort of expert knowledge that's
been accumulated we can compute the
answer to that question and we can do it
in a sort of reliable way that's that's
the best one can do given what the
expertise that our civilization has
accumulated it's a very it's a it's a
much more sort of labor-intensive on the
side of kind of being creating kind of
the the computational system to do that
um obviously the in the the kind of the
chat GPT world it's like take things
which were produced for quite other
purposes namely the all the things we've
written out on the web and so on and
sort of forage from that things which
were are like what's been written on the
web so I think you know as a practical
point of view I I view sort of the chat
GPT thing as being wide and shallow and
what we're trying to do with sort of
building out computation as being this
sort of deep also broad but but most
importantly kind of a deep type of thing
I think another way to think about this
is if you go back in human history you
know I don't know a thousand years or
something and you say what what can the
typical person what's the typical person
going to figure out well the answer is
there's certain kinds of things that we
humans can quickly figure out that's
sort of what what our uh you know other
neural architecture and the kinds of
things we learn in our lives let us do
but then there's this whole layer of
kind of formalization that got developed
in which is you know the kind of whole
sort of story of intellectual history
and the whole kind of depth of learning
that formalization turned into things
like logic mathematics science and so on
and that's the kind of thing that allows
one to kind of build these towers of of
uh uh of of uh sort of towers of things
you work out it's not just I can
immediately figure this out it's no I
can use this kind of form to go step by
step and work out something which was
not immediately obvious to me and that's
kind of the story of what what we're
trying to do computationally is to be
able to build those kind of tall towers
of what implies what implies what and so
on
um and uh as opposed to kind of the yes
I can immediately figure it out it's
just like what I saw somewhere else in
something that I heard or remembered or
something like this what can you say
about the kind of formal structure the
kind of form of foundation you can build
such a formal structure on
about the kinds of things you would
start on in order to build this kind of
uh deep computable knowledge trees so
the question is sort of how do you how
do you think about computation and
there's there's a couple of points here
one is what computation intrinsically is
like and the other is what aspects of
computation we humans with our minds and
with the kinds of things we've learned
can sort of relate to in that
computational universe so if we start on
the kind of what can computation be like
it's something I've spent some big chunk
of my life studying is imagine that
you're you know we usually we write
programs where we kind of know what we
want the program to do and we carefully
write you know many lines of code and we
hope that the program does what we what
we intended it to do but the thing I've
been interested in is if you just look
at the kind of natural science of
programs so you just say I'm going to
make this program it's a really tiny
program maybe I even pick the pieces of
the program at random but it's really
tiny by really tiny I mean you know less
than a line of code type thing you say
what does this program do and you run it
and big discovery that I made in the
early 80s is that even extremely simple
programs when you run them can do really
complicated things really surprised me
it took me several years to kind of
realize that that was a thing so to
speak but that that realization that
even very simple programs can do
incredibly complicated things that we
very much don't expect
that Discovery I mean I realized that
that's very much I think how nature
works that is nature has simple rules
but yet does all sorts of complicated
things that we might not expect you know
as a big thing of the last few years has
been understanding that that's how the
whole universe and physics works but
that's a a quite separate topic but so
there's this whole world of programs and
what they do and very rich sophisticated
things that these programs can do but
when we look at many of these programs
we look at them and say well that's kind
of I don't really know what that's doing
it's not a very human kind of thing
so on the one hand we have sort of
what's possible in the computational
universe on the other hand we have the
kinds of things that we humans think
about the kinds of things that are
developed in kind of our intellectual
history and that's uh and the Really the
challenge to sort of making things
computational is to connect what's
computationally possible out in the
computational universe with the things
that we humans sort of typically think
about with our minds now that's a
complicated kind of moving Target
because the things that we think about
change over time we've learned more
stuff we've invented mathematics we've
invented various kinds of ideas and
structures and so on so it's gradually
expanding we're kind of gradually
colonizing more and more of this kind of
intellectual space of possibilities but
the the real thing the real challenge is
how do you take what is computationally
possible how do you take how do you
encapsulate the kinds of things that we
think about in a way that kind of plugs
into what's computationally possible and
and actually the the uh the big sort of
idea there is this idea of kind of
symbolic programming symbolic
representations of things and so the the
question is when you look at sort of
everything in the world and you kind of
you know you take some visual scene or
something you're looking at and you say
well how do I turn that into something
that I can kind of stuff into my mind
you know there are lots of pixels in my
visual scene but the things that I
remembered from that visual scene are
you know there's a there's a chair in
this place it's a kind of a symbolic
representation of the visual scene there
are two chairs on a table or something
rather than there are all these pixels
arranged in all these detailed ways and
so the question then is how do you take
sort of all all the things in the world
and make some kind of representation
that corresponds to the types of ways
that we think about things and human
language is is sort of one form of
representation that we have we talk
about chairs that's a word in human
language and so on how do we how do we
take but human language is is not in and
of itself something from that plugs in
very well to sort of computation it's
not something from which you can
immediately compute consequences and so
on and so you have to kind of find a way
to take sort of the the stuff we
understand from human language and make
it more precise and that's really this
story of of symbolic programming and you
know what what that turns into is
something which I didn't know at the
time it was going to work as well as it
has but back in the 1979 or so I was
trying to build my first big computer
system and trying to figure out you know
how should I represent computations at a
high level and I kind of invented this
idea of using kind of symbolic
Expressions you know structured as it's
kind of like a a function and a bunch of
arguments but that function doesn't
necessarily evaluate to anything it's
just a a thing that sits there
representing a structure and so building
up that structure and it's turned out
that structure has been extremely it's
it's a good match for the way that we
humans it seems to be a good match for
the way that we humans kind of
conceptualize higher level things and
it's been the last I don't know 45 years
or something it's served me remarkably
well so building up that structure using
this kind of symbolic representation but
what can you say about abstractions here
because you could just start with your
physics project you could start at a
hypograph at a very very low level and
build up everything from there but you
don't you type shortcuts right uh you
take the highest level of abstraction
convert that
uh the kind of abstraction that's
convertible to something computable
using symbolic representation
and then that's your new foundation for
that little piece of knowledge yeah
somehow all of that is integrated right
so the the sort of a very important
phenomenon that that is kind of a thing
that I've sort of realized is just it's
one of these things that sort of in the
in the future of kind of everything is
going to become more and more important
is this phenomenon of computational
irreducibility and the the question is
if you know the rules for something you
have a program you're going to run it
you might say I know the rules great I
know everything about what's going to
happen
well in principle you do because you can
just run those rules out and just see
what they do you might run them a
million steps you see what happens Etc
the question is can you like immediately
jump ahead and say I know it's going to
happen after a million steps and the
answer is 13 or something yes and the
the one of the very critical things to
realize is if you could reduce that
computation there isn't a sense no point
in doing the computation the place where
you really get value out of doing
computation is when you had to do the
computation to find out the answer but
this phenomenon that you have to do the
computation to find out the answer this
phenomenon of computational
irreducibility seems to be tremendously
important for thinking about lots of
kinds of things so one of the things
that happens is okay you've got a model
of the universe at the low level in
terms of atoms of space and hypographs
and rewriting typographs and so on and
it's happening you know 10 to the 100
times every second let's say well you
say great then we've we've nailed it
we've we know how the universe works
well the problem is the universe can
figure out what it's going to do it does
those 10 to the 100 you know steps but
for us to work out what it's going to do
we have no way to reduce that
computation the only way to do the
computation to see the result of the
computation is to do it and if we're
operating within the universe we're kind
of there's no there's no opportunity to
do that because the universe is doing it
as fast as the universe can do it and
that's you know that's what's happening
so what we're trying to do and a lot of
the story of science a lot of other
kinds of things is finding pockets of
reducibility that is you could have a
situation where everything in the world
is full of computational irreducibility
we never know what's going to happen
next the only way we can figure out
what's going to happen next is just let
the system run and see what happens so
in a sense the story of of most kinds of
science inventions a lot of kinds of
things is the story of finding these
places where we can locally jump ahead
and one of the features of computational
reducibility is that there are always
pockets of reducibility there are always
places there always an infinite number
of places where you can jump ahead
there's no way where you can jump
completely ahead but there are little
little patches little places where you
can jump ahead a bit and I think you
know we can talk about physics project
and so on but I think the thing we
realize is we kind of exist in a slice
of all the possible computational
irreducibility in the universe we exist
in a slice where there's a reasonable
amount of predictability and in a sense
as we try and construct these kind of
higher levels of of abstraction symbolic
representations and so on what we're
doing is we're finding these lumps of
reducibility that we can kind of attach
ourselves to and about which we can kind
of have fairly simple narrative things
to say because in principle you know I
say what's going to happen in the next
few seconds you know oh there are these
molecules moving around in the air in
this room and oh gosh it's an incredibly
complicated story
um and that's a whole computational
irresistible thing most of which I don't
care about and most of it is well you
know the air is still going to be here
and nothing much is going to be
different about it and that's a kind of
reducible fact about what is ultimately
a an underlying level of computational
irreducible process
and uh life would not be possible if we
didn't have a large number of such
reducible Pockets uh yes Pockets
amenable to uh reduction into something
symbolic yes I think so I mean life in
in the way that we experience it that I
mean you know one might you know
depending on what we mean by life so to
speak the the the experience that we
have of sort of consistent things
happening in the world the idea of space
for example where there's you know we
can just say you're here you move there
it's kind of the same thing it's still
you in that different place even though
you're made of different atoms of space
and so on this is this idea that it's uh
that there's sort of this level of
predictability of what's going on that's
us finding a slice of reducibility in
what is underneath this computationally
reducible kind of system and I think
that's that's sort of the thing which is
actually my favorite discovery of the
last few years is the realization that
it is sort of the interaction between
this sort of underlying computational
irreducibility and our nature as kind of
observers who sort of have to key into
computational reducibility that fact
leads to the main laws of physics that
we discovered in throughout his century
so this is we talked about this in in
more detail but this is a uh to me it's
kind of our nature as observers the fact
that we are computationally bounded
observers we don't get to follow all
those little pieces of computational
irreducibility to stuff what is about
out there in the world into our minds
requires that we are looking at things
that are reducible we are compressing
kind of we're extracting just some
Essence some kind of symbolic essence of
what's the detail of what's going on in
the world that together with one other
condition that at first seems sort of
trivial but isn't which is that we
believe we are persistent in time
that is yes you know uh so some sense of
causality here's the thing at every
moment according to our Theory we're
made of different atoms of space
at every moment sort of the microscopic
detail of what what the universe is made
of is being Rewritten and that's and in
fact the very fact that there's
coherence between different parts of
space is a consequence of the fact that
there are all these little processes
going on that kind of knit together the
structure of space it's kind of like if
you wanted to have a fluid with a bunch
of molecules in it if those molecules
weren't interacting you wouldn't have
this fluid that would pour and do all
these kinds of things it would just be
sort of a free-floating collection of
molecules so similarities with space
that the fact that space is kind of
knitted together as a consequence of all
this activity in space and the fact that
kind of what we consist of sort of this
this series of of uh you know we're
continually being Rewritten and the
question is why is it the case that we
think of ourselves as being the same us
through time that's kind of a key
assumption I think it's a key aspect of
what we see as sort of our Consciousness
so to speak is that we have this kind of
consistent thread of experience well
isn't that just another
limitation
of our mind that we want to reduce
reality into some that kind of temporal
yeah consistency is just a nice
narrative right tell ourselves well the
fact is I think it's critical to the way
we humans typically operate is that we
have a single thread of experience you
know if you if you imagine sort of a
mind where you have you know maybe
that's what's happening in various kinds
of Minds that aren't working the same
way other minds work is that you're
splitting into multiple threads of
experience it's also it's also something
where you know when you look at I don't
know Quantum Mechanics for example in
the insides of quantum mechanics it's
splitting into many threads of
experience but in order for us humans to
interact with it you kind of have to
have to knit all those different threads
together so that we say oh yeah a
definite thing happened and now the next
definite thing happens and so on and I
think you know sort of inside uh it's
it's sort of interesting to try and
imagine what's it like to have kind of
these uh fundamentally multiple threads
of experience going on I mean right now
different human Minds have different
threads of experience we just have a
bunch of Minds that are interacting with
each other but we don't have a you know
within each mind there's a single thread
and that's a that is indeed a
simplification I think it's a it's a
thing you know the general computational
system does not have that simplification
and um it's one of the things you know I
I people often seem to think that you
know Consciousness is the highest level
of kind of things that can happen in the
universe so to speak but I think that's
not true I think it's actually a a
specialization in which among other
things you have this idea of a single
threat of experience which is not a
general feature of anything that could
kind of computationally happen in the
universe so it's a feature of a
computationally limited system that's
only able to
observe
reducible Pockets so yeah so I mean this
word Observer it means something in
quantum mechanics it means something
in a lot of places it means something to
us humans right as conscious beings so
what what's the importance of the
Observer what is the Observer and what's
the importance of the observer in the
computational universe so this question
of what is an observer what's the
general idea of an observer it's
actually one of my next projects which
got somewhat derailed by the the current
sort of AI Mania but um is there a
connection there or is that uh do you do
you think the Observer is primarily a
physics phenomena is it related to the
whole AI thing yes yes it is related so
one question is what is a general
Observer so you know we know we have an
idea what is a general computational
system we think about Turing machines we
think about other models of computation
there's a question what is a general
model of an observer and the there's
kind of observers like us which is kind
of The Observers we're interested in you
know we could imagine an alien Observer
that deals with computational
irreducibility and it has a mind that's
utterly different from ours and and
completely incoherent with what what
we're like but the fact is that that you
know if we are talking about observers
like us that one of the key things is
this idea of kind of taking all the
detail of the world and being able to
stuff it into a mind being able to take
all the detail and kind of you know
extract out of it a smaller set of of
kind of degrees of freedom a smaller
number of elements that will sort of fit
in our minds and I think this this
question so I've been interested in
trying to characterize what is the
general Observer and the general
Observer is I think in part there are
many let me give an example of a you
know you have a gas it's got a bunch of
molecules bouncing around and the thing
you're measuring about the gas is its
pressure and the only anything you as an
observer care about is pressure and that
means you have a piston on the side of
this box and the Piston is being pushed
by the gas and there are many many
different ways that molecules can hit
that piston but all that matters is the
kind of aggregate of all those molecular
impacts because that's what determines
pressure so there's a huge number of
different configurations of the gas
which are all equivalent so I think one
key aspect of observers is this
equivalency of many different
configurations of a system saying all I
care about is this aggregate feature all
I care about is this this overall thing
and that's that sort of one one aspect
and when we see that in lots of
different again it's the same story over
and over again that there's there's a
lot of detail in the world but what we
are extracting from it is something a
sort of a thin a thin summary of that of
that detail is that thin summary
nevertheless true is can it be a crappy
approximation sure that on average is is
correct I mean if we look at the
Observer that's the human mind it seems
like there's a lot of very
um as represented by natural language
for example there's a lot of really
crappy approximation sure and that could
be maybe a feature of it well with this
ambiguity right right you don't know you
know it could be the case you're just
measuring the aggregate impacts of these
molecules but there is some tiny tiny
probability that molecules will arrange
themselves in some really funky way and
that just measuring that average isn't
going to be the main point yeah by the
way an awful lot of science is very
confused about this because you know you
look at you look at papers and people
are really Keen they draw this curve and
they have these you know these bars on
the curve and things it's just this
curve and it's this one thing and it's
supposed to represent some system that
has all kinds of details in it and this
is a way that lots of science has gotten
wrong because people say I remember
years ago I was studying snowflake
growth you know you have the Snowflake
and it's growing it has all these arms
it's doing complicated things but there
was a literature on this stuff and it
talked about you know what's the rate of
snowflake growth and you know it got
pretty good answers for the rate of the
growth of the Snowflake and they looked
at it more carefully and they had these
nice curves of you know snowflake growth
rates and so on I looked at it more
carefully and I realized according to
their models the snowflake will be
spherical
and so they got the growth rate right
but the detail was just utterly wrong
and you know that not only the detail
that the whole thing was was not
capturing you know it was capturing this
aspect of the system that was in a sense
missing the main point of what was going
on and what is the geometric uh shape of
a snowflake snowflakes start in in the
phase of water that's relevant to the
formation of snowflakes it's a phase of
ice which starts with a hexagonal
arrangement of of water molecules and so
it starts off growing as a hexagonal
plate and then what happens is is the
plate oh oh versus sphere sphere well no
no but it's it's much more than that I
mean snowflakes are fluffy you know
typical snowflakes have little little
dendritic Arts yeah and what actually
happens is it's kind of kind of cool
because you can make these very simple
discrete models with cellular automata
and things that that figure this out you
start off with this you know hexagonal
thing and then the places it starts to
grow little arms and every time a little
piece of ice it adds itself to the
snowflake the fact that that ice
condensed from the water vapor heats the
snowflake up locally and so it makes it
less likely for uh for another piece of
ice to accumulate right nearby so this
leads to a kind of growth inhibition so
you grow an arm and it is a separated
arm because right around the arm it got
a little bit hot and it didn't add more
ice there so what happens is it grows
you have a hexagon it grows out arms the
arms go arms and then the arms go arms
go arms and eventually actually it's
kind of cool because it actually fills
in another hexagon a bigger hexagon and
when I first looked at this we had a
very simple model for this I realized
you know when it fills in that hexagon
it actually leaves some holes behind so
I thought well you know that is that
really right so look at these pictures
of snowflakes and sure enough they have
these little holes in them that are kind
of scars of the way that these arms grow
out
um so you can't fill in backfill holes
yeah they don't backfill and presumably
there's a limitation of how big like you
can't arbitrarily grow
I'm not sure I mean the thing falls
through the the I mean I think it this
you know it hits the ground at some
point I think you can grow I think you
can grow in the lab I think you can grow
pretty big ones I think you can grow
many many iterations of this kind of
goes from hexagon it grows out arms it
turns back it fills back into a hexagon
it grows more arms again in 3D no it's
flat usually why is it flat why doesn't
it uh span out okay wait a minute you
said it's fluffy and fluffy is a
three-dimensional property no or no it's
it's fluffy snow is okay so you know
what makes we're really uh we're really
in it
it's multiple snowflakes become fluffy a
single snowflake is not fluffy no no
single snowflake is Fluffy and what
happens is you know if if you have snow
that it's just pure hexagons they they
can you know they they fit together
pretty well it's not it doesn't it
doesn't make it doesn't have a lot of
air in it and they can also slide
against each other pretty easily and so
the snow can be pretty you know can I
think avalanches happen sometimes when
when the things tend to be these you
know hexagonal plates and it kind of
slides but then when the thing has all
these arms that have grown out it's not
they don't fit together very well and
that's why the snow has lots of air in
it and if you look at one of these
snowflakes and if you catch one you'll
see it has these little arms and people
actually people often say you know no
two snowflakes are alike
um that's mostly because as a snowflake
grows they do grow pretty consistently
with these different arms and so on but
you capture them at different times as
they you know they fell through through
the air in a different way you'll catch
this one at this stage and as it goes
through different stages they look
really different and so that's why you
know kind of looks like no two slime
flakes are alike because you caught them
at different at different times so the
rules under which they grow are the same
it's just the timing is yes okay so the
point is science is not able to uh
describe the full complexity of
snowflake growth well science if you if
you do what people might often do just
say okay let's make it scientific let's
turn into one number and that one number
is kind of the growth rate of the arms
or some such other thing that fails to
capture sort of the detail of what's
going on inside the system and that's in
a sense a big challenge for science is
how do you extract from the natural
world for example those aspects of it
that you are interested in talking about
now you might just say I don't really
care about the fluffiness of the
snowflakes all I care about is the
growth rate of the arms in which case
you know you have you can have a good
model without knowing anything about the
fluffiness
um but the fact is as a practical you
know when if you if you say what's the
what is the most obvious feature of a
snowflake oh that it has this
complicated shape well then you've got a
different story about what you model I
mean this is one of the features of sort
of modeling and science that you know
what is a Model A model is some way of
reducing the actuality of the world to
something where you can readily sort of
give a narrative for what's happening
where you can basically make some kind
of abstraction of what's happening and
answer questions that you care about
answering if you want to answer all
possible questions about the system
you'd have to have the whole system
because you might care about this
particular molecule where did it go and
you know your model which is some big
abstraction of that has nothing to say
about that so you know one of the things
that's that's often confusing in science
is people will have I've got a model
somebody says somebody else will say I
don't believe in your model because it
doesn't capture the feature of the
system that I care about you know
there's always this controversy about
you know is the is it a correct model
well no model is a except for the actual
system itself is a correct model in the
sense that it captures everything
questions does It capture what you care
about capturing sometimes that's
ultimately defined by what you're going
to build technology out of things like
this the one counter example to this is
if you think you're modeling the whole
universe all the way down then there is
a notion of a correct model but even
that is more complicated because it
depends on kind of how observers sample
things and so on that's a that's a
separate story but at least at the first
level to say you know this thing about
oh it's an approximation you're
capturing one aspect you're not
capturing other aspects when you really
think you have a complete model for the
whole universe you better be capturing
ultimately everything even though oh to
actually run that model is impossible
because of computational reducibility
the only the only thing that
successfully runs that model is the
actual running of the universe is the
universe itself but okay so what you
care about
is an interesting concept so that's a
that's a human concept so that's what
you're doing with uh wolf from Alpha and
Wolfram language is you trying to come
up with symbolic representations yes as
simple as possible
uh so a model that's as simple as
possible that fully captures stuff we
care about yes so I mean for example you
know we could we'll have a thing about
you know data about movies let's say we
could be describing every individual
pixel in every movie and so on but
that's not the level that people care
about and it's yes this is a I mean and
and that level that people care about is
somewhat related to what's described in
natural language but what what we're
trying to do is to find a way to sort of
represent precisely so you can compute
things see see one thing when you say
you give a piece of natural language
question is you feed it to a computer
you say does the computer understand
this natural language
well you know the computer process it in
some way it does this maybe it can make
a continuation of the natural language
you know maybe it can go on from The
Prompt and say what it's going to say
you say does it really understand it
hard to know but for in this kind of
computational world there is a very
definite definition of does it
understand which is could it be turned
into this symbolic computational thing
from which you can compute all kinds of
consequences and that's the that's the
sense in which one has sort of a target
for the understanding of natural
language and that's kind of our goal is
to have as much as possible about the
world that can be computed in a in a
reasonable way so to speak be able to be
sort of captured by this kind of
computational language that's that's
kind of the goal and and I think for us
humans the the main thing that's
important is as we formalize what we're
talking about it gives us a way of kind
of building a structure where we can
sort of build this Tower of consequences
of things so if we're just saying well
let's talk about it in natural language
it doesn't really give us some hard
Foundation that lets us you know build
step by step to work something out I
mean it's kind of like what happens in
math if we were just sort of vaguely
talking about math but didn't have the
kind of full structure of math and all
that kind of thing we wouldn't be able
to build this kind of big tower of
consequences and so you know in a sense
what we're trying to do with the whole
computational language effort is to make
a formalism for describing the world
that makes it possible to kind of build
this this Tower of consequences well can
you talk about this dance between
natural language and Wolfram language
so there's this gigantic thing called
the internet where people post memes and
diary type thoughts and very important
sounding articles and all of that that
makes up the training data set for GPT
and then there's a wolf from language
how can you map from the natural
language of the internet to the Wolfram
language is there an
manual is there an automated way of
doing that as we look into the future
well so wolf from alpha what it does
it's kind of front end is turning
natural language into computational
language right what you mean by that is
there's a prompt you ask a question what
is the capital of some yeah and it turns
into you know what's the distance
between you know Chicago and London or
something and that will turn into you
know geo-distance of entity City you
know Etc et cetera Etc each one of those
things is very is very well defined we
know you know given that it's the entity
City Chicago et cetera et cetera et
cetera you know Illinois United States
you know we know the geolocation of that
we know it's population we know all
kinds of things about it which we have
you know curated that data to be able to
to know that with some degree of
certainty so to speak and then
then we can compute things from this and
that's that's kind of the
um yeah that's that's that's the idea
but then something like GPT large
language models do they allow you to uh
make that conversion much more powerful
okay so that's an interesting thing
which we still don't know everything
about okay the um I mean this question
of going from natural language to
computational language yes in will from
alpha we've now you know wolfenovo's
been out and about for what 13 and a
half years now and you know we've
achieved I don't know what it is 98 99
success on queries that get put into it
now obviously there's a sort of feedback
loop because the things that work are
things people go on putting into it so
that that um uh but you know we've got
to a very high success rate of the the
little fragments of natural language
that put people put in you know
questions math calculations chemistry
calculations whatever it is you know we
can we can we we do very well at that
turning those things into to
computational language now I from the
very beginning of Orphan Alpha I thought
about for example uh writing code with
natural language in fact I had a I was
just looking at this recently I had a
post that I wrote in 2010 2011 called
something like programming with natural
language is actually going to work okay
and so you know we had done a bunch of
experiments using methods that were a
little bit some of them a little bit
machine learning like but certainly not
the same you know the same kind of idea
of vast training data and so on that is
the story of large language models
actually I know that that post a piece
of utter trivia but that that post um uh
Steve Jobs forwarded that post around to
all kinds of people at Apple you know
that was because he never really liked
programming languages so he was very
happy to see the idea that that that
that you could get rid of this kind of
layer of kind of engineering like
structure he would have liked you know I
think what's happening now because it
really is the case that you can you know
this idea that you have to kind of learn
how the computer works to use a
programming language is something that
is I think a a thing that you know just
like you had to learn the details of the
op codes to know how Assembly Language
worked and so on it's kind of a thing
that's that's that's a limited time
Horizon but but kind of the the you know
so this idea
of how elaborate can you make kind of
the prompt how elaborate can you make
the natural language and Abstract from
it computational language it's a very
interesting question and you know what
chat gbt you know gbt4 and so on can do
is pretty good
um it isn't it's very interesting
process I'm still trying to understand
this workflow we've been working out a
lot of tooling around this workflow the
natural language to computational
language right and the process
especially if it's conversation like
dialogue it's like multiple queries kind
of thing yeah right there's so many
things that are really interesting that
that work and so on so first thing is
can you just walk up to the computer and
expect to sort of specify a computation
what one realizes is humans have to have
some idea of kind of this way of
thinking about things computationally
without that you're kind of out of luck
because you just have no idea what
you're going to walk up to a computer I
remember when I should tell a silly
story about myself the very first
computer I saw which is when I was 10
years old it was a big Mainframe
computer and so on and I didn't really
understand what computers did and it's
like somebody's showing me this computer
and it's like uh you know can the
computer work out the weight of a
dinosaur it's like that isn't a sensible
thing to ask that's kind of you know you
have to give it that's not what
computers do I mean in Wolfram Alpha for
example you could say what's the typical
weight of a Stegosaurus and we'll give
you some answer but that's a very
different kind of thing from what one
thinks of computers as doing and so the
the kind of the the question of uh you
know first thing is people have to have
an idea of what what computation is
about
um you know I think it's a very you know
for Education that is the key thing it's
kind of this this sir this notion not
computer science not so the details of
programming but just this idea of how do
you think about the world
computationally computation thinking
about the world computationally is kind
of this formal way of thinking about the
world we've had other ones like logic
was a formal way you know as a way of
sort of abstracting and formalizing some
aspects of the world mathematics is
another one computation is this very
broad way of sort of formalizing the way
we think about the world and the thing
that's that's cool about computation is
if we can successfully formalize things
in terms of computation computers can
help us figure out what the consequences
are it's not like you formalized it with
math well that's nice but now you have
to if you're you know not using a
computer to do the math you have to go
work out a bunch of stuff yourself so I
think but that this idea let's see I
mean that you know we're trying to take
kind of the we're talking about sort of
natural language and its relationship to
computational language the uh the thing
the sort of the typical workflow I think
is first human has to have some kind of
idea of what they're trying to do that
if if it's something that they want to
sort of build a tower of of capabilities
on something that they want to sort of
formalize and make computational so then
human can type something in to you know
some llm system and uh uh sort of say
vaguely what they want in sort of
computational terms then it does pretty
well at synthesizing wealth language
code and it'll probably do better in the
future because we've got a huge number
of examples of of natural language input
together with the wolf and language
translation of that so it's kind of a a
um uh you know that that's a thing where
you can kind of extrapolating from all
those examples uh makes it easier to do
that that toss is the prompter task
could also kind of debug in the from
language code
or is your hope to not do that debugging
no no no I mean so so there are many
steps here okay so first the first thing
is you type natural language it
generates woven language give examples
by the way you have an example that is
the the dinosaur example do you have an
example that jumps to mind that we
should be thinking about some dumb
example it's like take my heart rate
data and uh you know figure out whether
I uh you know make a moving average
every seven days or something and work
out what the um and make a plot of the
result okay so that's a thing which is
you know
about two-thirds of a line of language
code I mean it's you know list plot of
moving average of some data bin or
something of the of the data and then
you'll get the result
um and you know the vague thing that I
was just saying in natural language
could would almost certainly correctly
turn into that very simple piece of
language code so you start mumbling
about heart rate
yeah and it kind of you know you arrive
at the moving average kind of idea but
you say average over seven days maybe
it'll figure out that that's a moving
you know that that can be encapsulated
as this moving average idea I'm not sure
but then the typical workflow but I'm
seeing is you generate this piece of
often language code it's pretty small
usually
um it's uh and if it isn't small it
probably isn't right but um you know if
it's it's pretty small and you know
welcome language is one of the ideas of
open languages it's a language that
humans can read it's not a language
which you know programming languages
tend to be this one-way story of humans
write them and computers execute from
them orphan language is intended to be
something which is sort of like math
notation something where you know humans
write it and humans are supposed to read
it as well and so kind of the workflow
that's emerging is kind of this this
human mumbles some things you know large
language model produces a fragment of
awesome language code then you look at
that you say yeah that looks well
typically you just run it first you see
does it produce the right thing you look
at what it produces you might say that's
obviously crazy you look at the code you
see I see why it's crazy you fix it if
you really care about the result you
really want to make sure it's right you
better look at that code and understand
it because that's the way you have this
sort of checkpoint of did it really do
what I expected it to do now you go
beyond that I mean it's it's it's you
know what we find is for example let's
say the code does the wrong thing then
you can often say to the large language
model can you adjust this to do this and
it's pretty good at doing that
interesting so you're using the output
of the code
to give you hints
about the the function of the code so
you're debugging yeah based on the
output of the code itself right the
plug-in that we have the the you know
for chat GPT it does that routinely you
know it will send the thing in it will
get a result it will discover the llm
will discover itself that the result is
not plausible and it will go back and
say oh I'm sorry it's very polite and it
you know it goes back and says I'll
rewrite that piece of code and then it
will try it again and get the result the
other thing is pretty interesting is
when you're just running so one of the
new Concepts that we have we invented
this whole idea of notebooks back 36
years ago now and so now there's the
question of sort of how do you combine
this idea of notebooks where you have
you know text and code and output how do
you combine that with the notion of of
chat and so on and there's some really
interesting things there like for
example a very typical thing now is we
have these these notebooks where as soon
as the if if the thing produce uses
errors if the you know run this code and
it produces messages and so on the the
llm automatically not only looks at
those messages it can also see all kinds
of internal information about stack
traces and things like this and it can
then it does a remarkably good job of
guessing what's wrong
and telling you so in other words it's
it's looking at things sort of
interesting it's kind of a typical sort
of ai-ish thing that it's able to have
more sensory data than we humans are
able to have because they're able to
look at a bunch of stuff that we humans
would kind of glaze over looking at and
it's able to then come up with oh this
is the explanation of what's happening
and and what is the data the stack trace
the the code you've written previously
the natural language you've written yeah
it's also what's happening is one of the
things that's uh is is for example when
there's these messages there's
documentation about these messages
there's examples of where the messages
have occurred otherwise nice all these
kinds of things the other thing that's
really amusing with this is when it
makes a mistake one of the things that's
in our prompt when the code doesn't work
is read the documentation
and we have a you know another piece of
the plugin that lets it read
documentation and that again is very
very useful because it it will you know
it will figure out sometimes it'll get
it'll make up the name of some option
for some function that doesn't really
exist read the documentation it'll have
you know some wrong structure for the
function and so on it's um that's a
powerful thing I mean the thing that you
know I've realized is we built this
language over the course of all these
years to be nice and coherent and
consistent and so on so it's easy for
humans to understand turns out there was
a side effect that I didn't anticipate
which is it makes it easy for AIS to
understand so it's almost like another
natural language but yeah so so what
formal language is a kind of foreign
language yes yes you have a lineup
English French Japanese Wolfram language
and then uh I don't know Spanish and
then the system is not going to notice
well yes I mean maybe you know that's an
interesting question because it really
depends on what I see as being a a
important piece of fundamental science
that basically just jumped out at us
with Chachi BT
um because I think you know the the real
question is why does chat GPD work how
is it possible to encapsulate you know
to successfully reproduce all these
kinds of things in natural language
um you know with a you know a
comparatively small he says you know a
couple hundred billion you know weights
of neural net and so on and I think that
uh you know that that relates to kind of
a fundamental fact about language which
uh you know the the main the main thing
is that I think there's a structure to
language that we haven't kind of really
explored very well as kind of the
semantic grammar I'm talking about about
um about language I mean we kind of know
that when we when we set up human
language we know that it has certain
regularities we know that it has a
certain grammatical structure you know
noun followed by verb followed by noun
adjectives Etc et cetera et cetera
that's its kind of grammatical structure
but I think the thing that chat gbt is
showing us is that as an additional kind
of regularity to language which has to
do with the meaning of the language
Beyond just this pure you know part of
speech combination type of thing and I
think the uh the the kind of the the one
example of that that we've had in the
past is logic and you know I I think my
my sort of uh uh uh kind of picture of
how was logic invented how was logic
discovered uh it really was the thing
that was discovered in its original
conception it was discovered presumably
by Aristotle who kind of listened to a
bunch of people orators you know giving
speeches and this one made sense that
one doesn't make sense this one and you
know you see these patterns of you know
if the uh you know I don't know what you
know if the uh if the Persians do this
then this does that
Etc et cetera et cetera and what what
Aristotle realized is there's a
structure to those sentences there's a
structure to that rhetoric that doesn't
matter whether it's the Persians and the
Greeks or whether it's the cats and the
dogs it's just you know p and Q you can
abstract from this the the details of
these particular sentences you can lift
out this kind of formal structure and
that's what logic is that's a heck of a
discovery by the way logic you're making
me realize now
yeah it's not obvious the fact that
there is an abstraction from natural
language that has where you can fill in
any word you want yeah is a very
interesting Discovery now it took a long
time to mature I mean Aristotle had this
idea of syllogistic logic where there
were these particular patterns of how
you could argue things so to speak and
you know in the Middle Ages part of
Education was you memorize the
syllogisms I forget how many there were
but 15 of them or something and they all
had names they all had mnemonics like I
think Barbara and celerant were two of
the the mnemonics for the the syllogisms
and people would kind of this is a valid
argument because it follows the Barbara
syllogism so to speak and and it took
until 1830
um you know with uh George boole to kind
of get beyond that and kind of see that
there was a a level of abstraction that
was beyond the this particular template
of a sentence so to speak
um and that's you know what what's
interesting there is in a sense you know
you know Chachi BT is operating at the
Aristotelian level it's essentially
dealing with templates of sentences by
the time you get to Bool and Boolean
algebra and this idea of you know you
can have arbitrary depth nested
collections of ands and ores and Knots
and you can resolve what they mean
um that's the kind of thing that's a
computation story that's you know you've
gone beyond the pure sort of templates
of natural language to something which
is an arbitrarily deep computation
but the thing that I think we realize
from from chat GPT is you know Aristotle
stopped too quickly and there was more
that you could have lifted out of
language as formal structures and I
think there's you know in a sense we've
captured some of that in in you know
some of what what is in language that
there's there's a there's a lot of kind
of little calculator little algebras of
of what you can say what language talks
about I mean whether it's I don't know
if you say uh
I go from place a to place B Place B to
place C then I know I've gone from place
a to place C if a is a friend of B and B
is a friend of C it doesn't necessarily
follow that a is a friend of C these are
things that are uh you know that there
are if if you go from from place a to
place B plus b to place C it doesn't
matter how you went like logic it
doesn't matter whether you flew there
walked there swam there whatever you
still this transitivity of of where you
go is still valid and there are there
are many kinds of kind of features I
think of the way the world Works uh that
are captured in these aspects of of
language so to speak and I think what
what chat gbt effectively has found just
like it discovered logic you know people
are really surprised it can do these
these logical inferences it discovered
Logic the same way Aristotle discovered
logic by looking at a lot of sentences
effectively and noticing the patterns in
those sentences but it feels like it's
discovering something much more
complicated than logic so this kind of
semantic grammar I think he wrote about
this
um
maybe we can call it the laws of
language I believe you call or which I
like the laws of thought yes that was
the title that George boole had for his
for his Boolean algebra back in 1830 but
yes I was a thought yes that was what he
said
all right so he thought he thought he
nailed it with blue in algebra yeah
there's more to it yeah it's a good
question how much more
is there to it and it seems like one of
the reasons as you uh imply that the
reason gbt Works chat GPT works is that
uh
there's a finite number of things to it
yeah I mean it's discovering the laws in
some sense GPT is discovering this laws
of semantic grammar that underlies
language yes what's sort of interesting
is in the computational universe there's
a lot of other kinds of computation that
you could do they're just not ones that
we humans have cared about and and
operate with and that's probably because
our brains are built in a certain way
and you know the neural Nets of our
brains are not that different in some
sense from the neural Nets of of uh of a
large language model and that's kind of
and and so when we think about and you
know maybe we can talk about this some
more but when we think about sort of
what will AIS ultimately do
the answer is insofar as AIS are just
doing computation they can run off and
do all these kinds of crazy computations
but the ones that we sort of have have
decided we care about are there is this
kind of very limited set
that's where the
uh reinforcement learning with human
feedback seems to come in the more the
AI say the stuff that kind of interests
us the more we're impressed by it
so you can do a lot of interesting
intelligent things but we're only
interested in the AI systems when they
communicate human in a human-like way
you ask about human-like topics yes well
it's it's like technology I mean in a
sense the physical world provides all
kinds of things you know there's all
kinds of processes going on in physics
only a limited set of those are ones
that we capture and use for technology
because they're only a limited set but
we say you know this is a thing that we
can sort of apply to the human purposes
we currently care about I mean you might
have said okay you pick up a piece of of
rock you say okay this is a nice
silicate it contains all kinds of
silicon I don't care then you realize oh
we could actually turn this into a you
know semiconductor wafer and make it
microprocessor out of it and then we
care a lot about it yes um and it's it's
you know it's this thing about what do
we you know in the evolution of our
civilization what things do we identify
as being things we care about I mean
it's like you know when when there was a
little announcement recently of the
possibility of a high temperature
superconductor that involved you know
the element lieutium which you know
generally nobody has cared about and you
know it's kind of um but suddenly if
there's this application that relates to
kind of human purposes we start to care
a lot so given your thinking that GPT
may have discovered inklings of laws of
thought
do you think such laws exist can we
Linger on that what's your intuition
here oh definitely I mean the fact is
look the the logic is but the first step
there are many other kinds of calculi
about things that uh we consider you
know about sort of things that happen in
the world or things that are meaningful
well how do you know logic is not the
last step you know what I mean so
because we can plainly see that that
thing I mean if you say here's a
sentence that is syntactically correct
okay you look at it and you're like you
know the happy electron you know eight
I don't know what some something that it
just it you look at and it's like this
is meaningless it's just a bunch of
words it's syntactically correct the
nouns and the verbs are in the right
place but it just doesn't mean anything
um and so there clearly is some rule
that there are rules that determine when
a sentence is has the potential to be
meaningful that go beyond the pure parts
of speech syntax and so the question is
what are those rules and are there
fairly finite set of those rules my
guess is that there's a fairly finite
set of those rules and they you know
once you have those rules you have a
kind of a construction kit just like
this the rules of syntactic grammar give
you a construction kit for making
syntactically correct sentences so you
can also have a construction kit for
making semantically correct sentences
those sentences may not be realized in
the world I mean I think you know the
elephant flew to the moon yeah a a
syntactic a semantically you know we
know we have an idea if I say that to
you you kind of know what that means but
the fact is it hasn't been realized in
the world so to speak so semantically
correct perhaps there's things that can
be imagined with the human mind no uh
things that are
consistent with both our imagination and
our understanding of physical reality I
don't yeah good question I mean it's a
good question it's a good question I
mean I think it is it is given the way
we have constructed language it is
things which which fit with the things
we're describing in language it's a bit
circular in the end because you know you
can and and the and the the sort of
boundaries of what is physically
realizable okay let's take the example
of motion okay motion is a complicated
concept it might seem like it's a
concept that should have been figured
out by the Greeks you know long ago but
it's actually really pretty complicated
concept because what is motion motion is
you can go from place a to place B and
it's still you when you get to the other
end right you you take an object you
move it and it's still the same object
but it's in a different place now even
in ordinary physics that doesn't always
work that way if you're near a
space-time singularity in a black hole
for example and you take your teapot or
something you don't have much of a
teapot by the time it's near the
space-time Singularity it's been
completely you know deformed beyond
recognition but so that's a case where
pure emotion doesn't really work you
can't have a thing stay the same but so
this idea of motion is is something that
sort of is a slightly complicated idea
but once you have the idea of motion you
can start once you have the idea that
you're going to describe things as being
the same thing but in a different place
that sort of abstracted idea then has
you know that has all sorts of
consequences like this transitivity of
motion go from A to B B to C you've gone
from a to c
um and that's so that level of
description you can have what are sort
of uh inevitable consequences they're
inevitable features of the way you've
sort of set things up and that's I think
what this sort of semantic grammar is
capturing is things things like that and
I you know I think that it's a question
of what does the word mean when you say
I go from I move from here to there well
it's complicated to say what that means
this is this whole issue of you know is
pure motion possible et cetera et cetera
et cetera but once you have kind of got
an idea of what that means then there
are inevitable consequences of that idea
but the very idea of meaning it seems
like there's some words that become
um
it's like there's a latent ambiguity to
them I mean it's the word like
emotionally loaded words like hate and
love
right it's like what what are they what
do they mean exactly like what
um so especially when you have
relationships between complicated
objects we seem to take this kind of
shortcut descriptive shortcut of to
describe like right object a hates
object B what's that really mean right
well words are defined by kind of our
social use of them I mean it's not you
know a word
in computational language for example
when we say we have a a construct there
we expect that that construct is a
building block from which we can
construct an arbitrarily tall tower so
we have to have a very solid building
block and you know we have to it turns
into a piece of code it has
documentation it's you know it's a whole
it's a whole thing but the word hate you
know the documentation for that word
well there isn't a standard
documentation for that word so to speak
it's a complicated thing defined by kind
of how we use it when you know if it
wasn't for the fact that we were using
language I mean so so what is language
at some level language is a way of
packaging thoughts so that we can
communicate them to another mind can
these complicated words
be converted into something that a
computation engine can use right so so I
think the answer to that is that that
what one can do in computational
language is Define make a def make a
specific definition and if you have a
complicated word like let's say the word
eat okay you'd think that's a simple
word it's you know animals eat things
whatever else but you know you do
programming you say this function eats
arguments which is sort of poetically
similar to the animal eating things but
if you start to say well what are the
implications of you know uh the function
eating something you know does it can
can the function be poisoned well maybe
it can actually but um uh you know if
there's a tight mismatch or something in
some language but but you know in what
how far does that analogy go and it's
it's just an analogy whereas if you use
the word eat in a computational language
level you would Define there isn't a
thing which you anchor to the kind of
natural language concept eat but it is
now some precise definition of that that
then you can compute things from but
don't you think the analogy is also per
se software eats the world don't you
think there's a
there's something Concrete in terms of
meaning about analogies sure but the
thing that sort of is the first Target
for computational language is to take
sort of the ordinary meaning of things
and try and make it precise make it
sufficiently precise you can build these
towers of computation on top of it so
it's kind of like if you start with a
piece of poetry and you say I'm going to
Define my program with this piece of
poetry it's kind of like that's that's a
difficult thing it's better to say I'm
going to just have this boring piece of
prose and it's using words in the
ordinary way and that time communicating
with my computer and that's how I'm
going to build the solid building block
from which I can construct this whole
kind of computational Tower so there's
some sense where if you take a poem and
reduce it to something computable you're
going to have very few things left so
maybe there's a bunch of human
interaction that's just poetic
aimless nonsense
well that's just like recreational like
hamster in a wheel it's not actually
producing anything well I I I think that
that's a complicated thing because in a
sense human linguistic communication is
there's one mind it's producing language
that language is having an effect on
another mind yeah and the question of
there's sort of a a type of effect that
is well defined let's say where where
for example it's very independent of the
two minds that the it doesn't you know
that there there's communication where
it can matter a lot sort of what the
experience of of um uh of one mind is
versus another one and so on
yeah but uh
what is the purpose of natural English
communication
well I think I think versus so
computation computational language
somehow feels more amenable to the
definition of purpose it's like yeah
you're given to
clean representations of a concept and
you can build a tower based on that is
is natural language the same thing but
more fuzzy what well I think the the
story of natural language right in the
the that's the great invention of our
species we don't know whether exists in
other species but we know it exists in
our species it's the thing that allows
you to sort of communicate abstractly
from like one generation of the species
to another you can you know there is an
abstract version of knowledge that can
be passed down it doesn't have to be you
know genetics it doesn't have to be you
know you don't have to Apprentice the
next species you know the next
generation of birds to the previous one
to show them how something works yeah
there is this abstracted version of
knowledge that can be kind of passed
down now that you know it relies on it
still tends to rely because language is
fuzzy it does tend to rely on the fact
that you know if we look at the you know
some ancient language that where we
don't have a chain of translations from
it until what we have today we may not
understand that ancient language
um and we may not understand you know
its Concepts may be different from the
ones that we have today we still have to
have something of a chain but it is
something where we can realistically
expect to communicate abstract ideas and
that's you know that's one of the big
big roles of a language I think you know
in in um
uh it's you know that that's been this
this ability to sort of concretify
abstract things is what what language
has provided do you see natural language
and thought as the same the stuff that's
going inside your mind
well
that's been a long debate in philosophy
it seems to be become more important now
when we think about
how intelligent GPT is
whatever that means whatever that means
but it seems like the stuff that's going
on in the human mind seems something
like intelligence is language but we
call it intelligence yeah we call it
well yes and so you start to think of
okay what's the relationship between
thought
the language of thought the laws of
thought the laws of the words like
reasoning
and the laws of language and how that
has to do with computation which seems
like a more rigorous precise ways of
reasoning right which are Beyond human I
mean much of what computers do human
humans do not do I mean you might say
humans are a subset yeah presumably yes
hopefully yes the the yes right you know
you might say who needs computation when
we have large language models large
language models can just you know
eventually you'll have a big enough
neuron that it can do anything but
they're really doing the kinds of things
that humans quickly do and there are
plenty of sort of formal things that
humans never quickly do for example I
don't know I you know you can some
people can do mental arithmetic they can
do a certain amount of math in their in
their minds I don't think many people
can run a program in their minds of any
sophistication it's just not something
people do it's not something people have
even thought of doing because just it's
kind of a it's kind of not you know you
can easily run it on a computer when
another portray program yeah aren't we
running specialized programs yeah yeah
but if I say to you here's a turing
machine yeah you know tell me what it
does after 50 steps and you're like
trying to think about that in your mind
that's really hard to do it's not what
people do I mean well in some sense
people program they build a computer
they program it just to answer your
question about what the system does
after 50 steps I mean humans build
computers yes yes yeah that's right but
they've created something which is then
you know then when they run it it's
doing something different than what's
happening in their minds I mean they've
outsourced that that piece of
computation from something that is
internally happening in their minds to
something that is now a tool that's
external to their minds so whether we're
humans to you didn't invent computers
they discovered them
they discovered computation which they
invented the technology of computers
this the computer is just a
kind of way to plug into this whole this
stream of computation which probably
other are the ways for sure I mean the
the you know the particular ways that we
make computers out of semiconductors and
electronics and so on that's the
particular technology stack we built I
mean the story of a lot of what people
try to do with Quantum Computing is
finding different sort of underlying
physical you know infrastructure for
doing computation you know biology does
lots of computation it does it using an
infrastructure that's different from
semiconductors and electronics it's a
you know it's a molecular scale uh sort
of computational process that hopefully
we'll understand more about I have some
ideas about understanding more about
that but uh you know that's that's
another ins you know it's another
representation of computation things
that happen in the physical Universe at
the level of you know these evolving
hypographs and so on that's another sort
of implementation layer for this
abstract idea of computation so if GPT
or a large language models starting to
form
starting to develop or implicitly
understand the laws of language and
thought do you think they can be made
explicit yes how okay he had a bunch of
effort I mean so do they are doing
Natural Science I mean what is happening
in Natural Science you have the world
that's doing all these complicated
things and then you discover you know
Newton's laws for example this is how
motion works this is the way that this
particular sort of idealization of the
world this is how we describe it in a
simple computationally reducible way and
I think it's the same thing here it's
there are sort of computationally
reducible aspects of what's Happening
that you can get a kind of narrative
theory for just as we've got narrative
theories in physics and so on
God
do you think it will be
depressing or exciting
when all the laws of thought are made
explicit human thought made explicit I
think that once you understand
computational reducibility it is uh it's
neither of those things because the fact
is people say for example that people
will say oh but you know I have free
will I I kind of um you know I operate
in a way that is uh uh you know you you
the the the the they have the idea that
they're doing something that is sort of
of internal to them that they're
figuring out what's what's happening but
in fact we think there are laws of
physics that ultimately determine you
know every uh every nerve you know every
electrical impulse and a nerve and
things like this so you might say isn't
it depressing that we are ultimately
just determined by the rules of physics
so to speak
um it's the same thing it's at a higher
level it's like it's it's it's a shorter
distance to get from kind of semantic
grammar to the way that we might
construct a piece of text than it is to
get from Individual nerve firings to how
we construct a piece of text but it's
not fundamentally different and by the
way as soon as we have this kind of
level of you know this other level of
description it's kind of it helps us to
go even further so we'll end up being
able to produce more and more
complicated Kinds of Kinds of things
that just like when we you know if we
didn't have a computer and we knew
certain rules we could write them down
and go a certain distance but once we
have a computer we can go vastly further
and this is the same kind of thing you
wrote a blog post titled what is Chad
GPT doing and where does it work we've
been talking about this but can we just
step back and Linger on this question
what what's it what's Chad GPT doing
what are these
um
a bunch of billion parameters trained on
a large number of words
why does it seem to work again is it is
it because to the point you made that
there's laws of language that can be
discovered by such a process is there
something well let's let's talk about
sort of the low level of what chat GPT
is doing I mean ultimately you give it a
prompt it's trying to work out you know
what should the next word be right which
is wild isn't that isn't that surprising
to you that this kind of low level dumb
training procedure can create something
syntactically correct first and then
semantically correct you know the thing
that has been sort of a story of my life
is realizing that simple rules can do
much more complicated things than you
imagine that something that starts
simple and start simple to describe can
grow a thing that is you know vastly
more located than you can imagine and
honestly it's taken me I don't know I've
sort of been thinking about this now 40
years or so and it always surprises me I
mean even for example in our physics
project sort of thinking about the whole
universe growing from these simple rules
I still resist because I keep on
thinking you know how can something
really complicated arise from something
that simple it just seems you know it
seems wrong but yet you know the
majority of my life I've kind of known
from from things I've studied that this
is the way things work so yes I it is
wild that it's possible to write a word
at a time and produce a coherent essay
for example but it's worth understanding
kind of how that's working I mean it's
kind of like if if it was going to say
you know the cat sat on the what's the
next word okay so how does it figure out
the next word well it's seen a trillion
words written on the internet and it's
seen the cat sat on the floor the cat
sat on the sofa the cat sat on the
whatever so it's minimal thing to do is
just say let's look at what we saw on
the internet we saw you know 10 000
examples of the cat sat on the what was
the most probable next word let's just
pick that out and say that's the next
word and that's that's kind of what it
is at some level is trying to do now the
problem is there isn't enough text on
the internet to uh for if you have a
reasonable length of prompt to that that
Pro that specific prompt will never have
occurred on the internet and as you as
you kind of go further there just won't
be a place where you could have trained
you know where you could just worked out
probabilities from what was already
there
um you know like if you say two plus two
there'll be a zillion examples of two
plus two equaling four and a very small
number of examples of two plus two
equals five and so on and you can pretty
much know what's going to happen so then
the question is well if you can't just
work out from examples what's going to
happen just no probabilistic for you for
example is what's going to happen you
have to server model and there's kind of
an idea this idea of making models of
things is an idea that really I don't
know I think Galileo probably was one of
the first people who sort of worked this
out and it's kind of like like you know
I think I gave an example of that the
book I wrote about about Chachi BT where
it's kind of like you know Galileo was
dropping cannonballs off the off the
different floors of the of the Tower of
Pisa and it's like okay you drop a
cannonball off this floor you drop a
cannonball off this floor you miss floor
five or something for whatever reason
but you know the time it took the
Cannonball to fall to the ground from
floors one two three four six seven
eight for example then the question is
can you work out can you can you make a
model which figures out how long does it
take the ball how long would it take the
ball to fall to the ground from the
floor you didn't explicitly measure and
the thing Galileo realized is that you
can use math you can use mathematical
formulas to make a model for how long it
will take the ball to fall so now the
quest question is well okay you want to
make a model for for example something
much more elaborate like you've got this
arrangement of pixels and is this
arrangement of pixels an A or a B does
it correspond to something we'd
recognize as an A or B and you can make
a similar kind you know each pixel is
like a parameter in some equation and
you could write down this giant equation
where the answer is either you know a or
you know one or two A or B
um and the question is then what kind of
a model successfully reproduces the way
that we humans would would conclude that
this is an A and this is a b you know if
there's a complicated extra tail on the
top of the a would we then conclude
something different what is the type of
model that Maps well into the way that
we humans make distinctions about things
and the big kind of meta Discovery is
neural Nets are such a model it's not
obvious they would be such a model it
could be that human distinctions are not
captured you know we could try searching
around for a Titan model that could be a
mathematical model it could be some
model based on something else that
captures kind of typical human
distinctions about things it turns out
this model that actually is very much
the way that we think the architecture
brains works that perhaps not
surprisingly that model actually
corresponds to the way we make these
distinctions and so you know the the
core next point is that the the kind of
model that's neural net model makes sort
of distinctions and generalizes things
in sort of the same way that we humans
do it and that's why when you say you
know the cat set on the green blank even
though it never didn't see many examples
of the cat set on the green whatever it
can make a or the aardvark sat on the
green whatever I'm sure that particular
sentence does not occur on the internet
and so it has to make a model that
concludes what you know it has to kind
of generalize from what it's from the
actual examples that it's seen and so so
you know that that's the fact is that
neural Nets generalized in the same kind
of a way that we humans do if if we were
you know the aliens might look at our
neural net generalizations and say
that's crazy you know that thing when
you put that extra little dot on the a
that isn't an a anymore that's you know
that messed the whole thing up but for
us humans we make distinctions which
seem to correspond to the kinds of
distinctions that neural Nets make so
then you know the the thing that is just
amazing to me about chat gbt is how
similar the structure it has is to the
very original way people imagine neural
Nets might work back in 1943 and you
know there's a lot of detailed
engineering you know great cleverness
but it's really the same idea and in
fact even the sort of elaborations of
that idea where people said let's put in
some actual particular structure to try
and make the neural net more elaborate
to be very clever about it most of that
didn't matter I mean there's some things
that seem to you know when you when you
train this neural net you know the one
thing this kind of Transformer
architecture this attention idea that
really has to do with does every one of
these neurons connect to every other
neuron or is it somehow causally
localized so to speak does it like we're
making a sequence of words and the words
depend on previous words rather than
just everything can depend on everything
and that seems to be important in just
organizing things so that you don't have
a sort of a giant mess but the thing you
know the thing worth understanding about
what is chat gpg in the end I mean what
is a neuron that's in the end a neural
net in the end is each neuron has a it
it's taking inputs from a bunch of other
neurons it's it's eventually it's going
to have it's going to have a a numerical
value it's going to compute some number
and it's it's saying I'm going to look
at the the neurons above me it's kind of
a series of layers it's going to look at
the neurons above me and it's going to
say what are the values of all those
neurons then it's going to add those up
and multiply them by these weights and
then it's going to apply some function
that says if it's bigger than zero or
something then make it one or an
otherwise make it zero or some slightly
more complicated function you know very
well how this works but it's a giant
equation although a lot of variables you
mentioned figuring out where the ball
Falls when you don't have data on the
fourth floor
um this the equation here is not as
simple as right the equation with 175
billion terms and it's quite surprising
that in some sense a simple procedure of
uh training such an equation can lead to
well I think that good representation of
natural language right the the real
issue is you know this architecture of a
neural net where where what's happening
is you know you've you've you've turned
so neural Nets always just deal with
numbers and so you know you've turned
the sentence that you started with into
a bunch of numbers like let's say by
mapping you know each word of the 50 000
words in English you just map each word
or each part of a word into some number
they feed all those numbers in
and then the thing is going to and then
those numbers just go into the values of
these neurons and then what happens is
it's just Rippling down going layer to
layer until it gets to the end I think
chat gpg has about 400 layers and you're
just you know it just goes once through
it just every every new word it's going
to compute just says here are the here
are the numbers from the words before
let's compute the what is it compute it
computes the probabilities that it
estimates for each of the possible 50
000 words that could come next and then
it decides sometimes it will use the
most probable word sometimes it will use
not the most probable word it's an
interesting fact that there's this
so-called temperature parameter which
you know at temperature zero it's always
using the most probable word that it can
that it estimated was the the most
probable thing to come next you know if
you increase the temperature it'll be
more and more kind of random in its
selection of words it'll go down to
lower and lower probability words thing
I was just playing with actually
recently was the transition that happens
as you increase the temperature the
thing goes Bonkers at a particular you
know sometimes at a particular
temperature I think maybe about 1.2 is
the thing I was noticing from yesterday
actually
um that you know usually it's giving
reasonable answers and then uh at that
temperature with some probability it
just starts spouting nonsense
um and you know nobody knows why this
happens I mean it's it's uh uh and by
the way I mean the thing to understand
is it's putting down one word at a time
but the outer loop of the fact that it
says okay I put down a word now let's
take the whole thing I wrote so far
let's feed that back in let's put down
another word that outer loop which seems
almost trivial is really important to
the operation of the thing and and for
example one of the things that is kind
of funky is it'll give an answer and you
say to it is that answer correct and
it'll say no
and why is that happening right right
why couldn't it do that well the answer
is because it is going one word at a
time sort of forwards and it didn't you
know it it came along with some sort of
chain of of thought in a sense and it
came up with completely the wrong answer
but as soon as you feed it the whole
thing that it came up with it
immediately knows that that isn't right
it immediately can recognize that was a
you know a bad syllogism or something
and uh can see what happened even though
as it was being led down this Garden
Path so to speak it didn't it came to
the wrong place but it's fascinating
that this kind of procedure converges to
something that forms a pretty good
compressed representation of language on
the internet yeah that's quite right
right no I'm not sure what to make of it
well look I think you know there are
many things we don't understand okay so
for example you know 175 billion weights
it's maybe about a trillion bytes of
information which is very comparable to
the training set that was used
um and uh you know why that why kind of
it sort of stands to some kind of reason
that the number of Weights in the neural
net I don't know where I can't really
argue that I can't really give you a
good uh you know in a sense the very
fact that you know the insofar as there
are definite rules of what's going on
you might expect that eventually we'll
have a much smaller neural net that will
successfully capture what's happening I
I don't think the best way to do it is
probably a neural net I think a neuron
that is what you do when you don't know
any other way to structure the thing and
it's a very good thing to do if you
don't know any other way to structure
the thing and for the last 2000 years we
haven't known any other way to structure
it so this is a pretty good way to start
but that doesn't mean you can't find
sort of in a sense more symbolic rules
for what's going on that you know much
of which will then be you can kind of
get rid of much of the structure of the
neural Nets and replace it by things
which are sort of pure steps of
computation so to speak sort of with
neural net stuff around the edges and
that becomes just a you know just a much
simpler way to do it so then you're on
that you hope will reveal
to us good symbolic rules that make the
need then you're on that less and less
and less right and and there will still
be some stuff that's kind of fuzzy just
like you know that they're things that
it's like this question of what can we
formalize what can we turn into
computational language what is just sort
of oh it happens that way just because
brains are set up that way
what do you think are the limitations of
uh large language models just to make it
explicit well I mean I think that deep
computation is not what large language
models do I mean that's just it's a
different kind of thing you know the
outer loop of a large language model if
if you're trying to do many steps in a
computation the only way you get to do
that right now is by spooling out you
know all that the whole Chain of Thought
as a bunch of words basically and you
know you can make a turing machine out
of that if you want to I just was make
doing that construction you know in
principle you can make an arbitrary
computation by just spooling out the
words but it's an it's a bizarre and
inefficient way to do it
um but it's something where uh the you
know I I think that's you know sort of
the the Deep computation is it's it's
really what a humans can do quickly
large language models will probably be
able to do well anything that you can do
kind of off the top of your head type
thing is is really you know is good for
large language models and the things you
do off the top of your head you may not
get them always right but you know
you'll it it's it's thinking it through
the same way we do but I wonder if
there's an automated way to do something
that humans do
well much faster to where it like Loops
so generate arbitrary large code bases
of Wolfram language for example
well the question is what does what do
you want the code base to do
um Escape control and take over the
world okay so you know the thing is when
people say you know we want to build
this giant thing right a giant piece of
computational language in a sense it's
sort of a failure of computational
language if the thing you have to build
in other words if we have a description
if you have a small description
that's the thing that you represent in
computational language and then the
computer can compute from that yes so in
a sense in you know when as soon as
you're giving a description that you
know if you have to somehow make that
description something you know definite
something formal and once and and to say
to say okay I'm going to give this piece
of natural language and then it's going
to splurt out this giant formal
structure that in a sense that doesn't
that doesn't really make sense because
acceptanceofar as that piece of natural
language kind of plugs into what we
socially know so to speak it plugs into
kind of our Corpus of knowledge then you
know that's the way we're capturing a
piece of that Corpus of knowledge but
hopefully we will have done that in
computational language how do you make
it do something that's big well you know
you have to have a way to describe what
you want okay I can make it more
explicit if you want how about I just
pop into my head
um iterate
through all the members of Congress
and figure out how to convince them that
they have to
let
me
this meaning the system become president
pass all the laws that allows AI systems
to take control and be the president I
don't know so that's a very explicit
like figure out the individual life
story of each Congressman that each
Senator anybody I don't know
what's required to really kind of pass
legislation and figure out how to
control them and manipulate them get all
the information what would be the
biggest fear of this congressman and uh
in such a way that you can take action
on it in the digital space so maybe
threaten the destruction reputation or
something like this right if I can
describe what I want yeah you know to
what extent can a large language model
automate that
with the help uh with the help of the
concretization of something like Wolfram
language
that makes it more
um yeah rather a long way I'm also
surprised how quickly I was able to
generate yeah yeah right an attack
that that's the here you know I I swear
I swear I did not think about this
before it is funny how quickly which is
a very concerning thing because that
that probably this idea will probably do
quite a bit of damage and there might be
a very large number of other such ideas
well I'll give you a much more benign
version of that idea okay you're going
to make an AI tutoring system and you
know that is a that's a benign version
of what you're saying is
um I want this person to understand this
point yes you know you're essentially
doing machine learning where the where
the where the you know the the loss
function the the thing you're trying to
get to is get the human to understand
this point and and when you do a test on
the human that they yes they correctly
understand how this or that works and I
I am confident that uh you know sort of
the large language model type technology
combined with computational language is
going to be able to do pretty pretty
well at teaching us humans things and
it's going to be an interesting
phenomenon because you know sort of
individualized teaching is is a thing
that has been kind of a you know a goal
for a long time I think we're going to
get that I think more you know that it
has many consequences for you know like
like just you know if you know me as an
if you the AI know me tell me I'm about
to do this thing what is the what are
the three things I need to know you know
given what I already know you know
what's the what's let's say I'm I'm
looking at some paper or something right
it's it's like there's a version of this
summary of that paper that is optimized
for me so to speak and where it really
is and I think that's really going to
work it could understand the the major
gaps in your knowledge yes that if field
would actually give you uh
a deeper understanding of the topic here
right and that's a you know that's an
important thing because it really
changes actually I think you know when
when you think about education and so on
it really changes kind of what's worth
doing what's not worth doing and so on
it makes you know I know in my life I've
learned lots of different fields and you
know so I yeah I don't know I have every
time I'm always think that this is the
one that's going to I'm not going to be
able to learn yeah but turns out sort of
there are sort of meta methods for
learning these things in the end
um and you know I think this this idea
that it becomes easier to you know it
becomes easier to be fed knowledge so to
speak and it becomes you know if you
need to know this particular thing you
can you know you can get taught it in an
efficient way it's something I think is
sort of a an interesting feature and I
think it makes the um
you know things like the value of of big
towers of specialized knowledge become
less significant compared to the kind of
meta knowledge of sort of understanding
kind of the the big picture and being
able to connect things together I think
that you know there's been this huge
trend of let's be more and more
specialized because we have to you know
we we have to sort of Ascend these
towers of knowledge but by the time you
can get you know more automation of
being able to get to that place on the
tower without having to go through all
those steps I think it sort of changes
that picture interesting so your
intuition is that in terms of the the
the collective intelligence of the
species in the individual minds they
make up that Collective
there'll be more there will Trend
towards being generalists and being kind
of philosophers that's what I think I
think that's where the humans are going
to be useful I think that a lot of these
kind of
the drilling the the the mechanical
working out of things is much more
automatable it's much more ai ai
territory so to speak no more phds
well that's a it's interesting yes I
mean that you know the the kind of the
specialization this kind of tower of
specialization which has been a feature
of you know we've accumulated lots of
knowledge in our in our species and and
you know in a sense every time we every
time we have a kind of automation a
building of tools it becomes less
necessary to know that whole Tower and
it becomes something where you can just
use a tool to get to the top of that
Tower I I think that um uh you know the
thing that is ultimately you know when
we think about okay what do the AIS do
versus what do the humans do it's like
ai's you tell them you say go achieve
this particular objective okay they can
maybe figure out a way to achieve that
objective we say what objective would
you like to achieve
the AI has no intrinsic idea of that
it's not a defined thing that's a thing
which has to come from some other you
know some other entity and insofar as we
are in charge so to speak or whatever it
is and our kind of web of society and
history and so on is the thing that is
defining what objective we want to go to
that's you know that that's that's a
thing that we humans are necessarily
involved in so to push back a little bit
don't you think that GPT feature
versions of GPT
would be able to give a good answer to
what objective would you like to achieve
from on what basis I mean if they say
look here's the terrible thing that
could happen Okay that taking the
average of the internet and they're
saying you know from the average of the
internet what do people want to do well
that's the uh the Elon Musk outage of
the most entertaining outcome is the
most likely
okay that could be um
that could be one objective
is maximize
global
entertainment the dark version of that
is drama the the good version of that is
fun
right so I mean this this question of
what uh you know if you say to the AI
um you know uh what does the species
want to achieve
yes okay there'll be an answer right
that'll be an answer it'll be what the
average of the internet says the species
wants to achieve well this this I think
you're using the word average very
Loosely there right so
I think you I think the answers will
become more and more interesting as
these language models are trained better
and better no but I mean in the end it's
a reflection back of what we've already
said
yes but it's uh there's a deeper wisdom
to the collective intelligence
presumably than each individual maybe
isn't that what we're trying to just
Society uh to have well I mean that's
that's a that's an important no that's
an interesting question I mean in you
know insofar as some of us you know work
on trying to innovate and figure out new
things and so on it is sometimes it's a
it's a complicated interplay between
sort of the individual doing the crazy
thing often some some spur so to speak
versus the collective that's trying to
do sort of the the the the high inertia
average thing and it's you know
sometimes the collective you know is is
bubbling up things that are interesting
and sometimes it's pulling down kind of
the attempt to make this kind of
innovative Direction but don't you think
the large language models would see
beyond that simplification will say
maybe intellectual and career diversity
is really important so you need the
crazy people from the outlier on the
outskirts and so like the actual what's
the purpose of this whole thing is to
explore
through this kind of dynamics that we've
been using as a human civilization which
is most of us focused on one thing and
then there's the crazy people on the
outskirts doing the opposite of that one
thing and you kind of pull the whole
society together there's the mainstream
science and then there's the crazy
science and it's just been about the
history of human civilization and maybe
the AI system will be able to see that
and the more and more impressed we are
by a language model telling us this the
more control we'll give it to it and the
more we'll be willing to let it run our
society and hence there's this kind of
loop where the society could be
manipulated to let the AI system run it
right well I mean look one of the things
that sort of interesting is we might say
we always think we're making progress
but yet if you know in a sense by by
saying let's take what already exists
and use that as a model for what should
exist yeah then you know it's
interesting that for example you know
many religions have taken that point of
view there is a you know a sacred book
that got written at Timex and it defines
how people should act for all future
time and that's you know it's it's a
model that people have operated with and
in a sense you know this is a version of
that that kind of statement it's like
take the 2023 version of sort of how the
world has exposed itself
and use that to Define what the world
should do in the future but it's not
it's an imprecise definition right
because just like with religious text
and with GPT the human interpretation of
what GPT says
will be the um
uh will be the perturbation in the
system it'll be the noise it'd be full
of uncertainty it's not like GPT will
tell you exactly what to do
it'll tell you approx A Narrative of
what like uh uh you know it's like turn
the other cheek kind of narrative right
that's that's not a fully instructive
narrative well until until the AIS
control all the systems in the world
they will be able to very precisely tell
you what to do but they'll do what they
you know they'll they'll just do this or
that thing and and that and and not only
that they'll be Auto suggesting to each
person you know do this next do that
next so I think it's a it's a slightly
more prescriptive situation than one has
typically seen but you know I think this
this whole question of sort of what
what's left for the human so to speak to
what extent do we uh you know this idea
that there is an existing kind of Corpus
of purpose for humans defined by what's
on the internet and so on that's an
important thing but then the question of
sort of as we explore what we can think
of as the computational universe as we
explore all these different
possibilities for what we could do all
these different inventions we could make
all these different things the question
is which ones do we choose to follow
those choices are the things that in a
sense if the humans want to still have
kind of human progress that's what we we
get to make those choices so to speak in
other words the the there's this idea if
you say let's take
the uh kind of what exists today and use
that as the determiner of all of what
there is in the future
the thing that is sort of the
opportunity for humans is there will be
many possibilities thrown up there are
many different things that could happen
or be done and the insofar as we want to
be in the loop the thing that makes
sense for us to be in the loop doing is
picking which of those possibilities we
want
but the degree to which there's a
feedback loop
of the idea that we're picking something
starts becoming questionable because
we're influenced by the various systems
absolutely like if that becomes more and
more source of our education and wisdom
and knowledge
right the AIS take over I mean my you
know I've thought for a long time that
you know it's the you know AR Auto
suggestion that's really the thing that
makes the AIS take over it's just then
the humans just follow you know yeah we
will no longer write emails to each
other we'll just send the auto suggested
email yeah yeah but the the thing where
humans are potentially in the loop is
when there's a choice and when there's a
choice which we could make based on our
kind of whole web of history and so on
yeah and and that's you know that's
insofar as it's all just you know
determined uh you know the humans don't
have a place and and by the way I mean
you know at some level uh you know it's
all kind of a complicated philosophical
issue because at some level the universe
is just doing what it does we are parts
of that universe that are necessarily
doing what we do so to speak yet we feel
we have sort of agency in what we doing
and that's that's its own separate kind
of interesting issue and we also kind of
feel like we're the Final Destination
what the universe was meant to create
uh but we very well could be and likely
are some kind of intermediate step
obviously yeah well we're we're most
certainly some intermediate step the
question is if there's some cooler more
complex more interesting uh thing that's
going to be materialized this
computational universe is full of such
things but in our particular pocket
specifically if this is the best we're
going to do or not that's kind of a we
can make all kinds of interesting things
in the computational universe we when we
look at them we say yeah you know that's
that's a thing we don't it doesn't
really connect with our current uh our
current way of thinking about things
it's like in mathematics you know we've
got certain theorems they're about three
or four million that human
mathematicians have written down and
published and so on but they're an
infinite number of possible mathematical
theorems we just go out into the
universe of possible theorems and pick
another theorem and then people will say
well you know that's the you know they
look at it and they say I don't know
what this theorem means it's not
connected to the things that are part of
kind of the web of history that we're
dealing with you know I think one one
point to make about sort of
understanding Ai and its relationship to
us is as we have this kind of whole
infrastructure of AIS doing their thing
and doing their thing in a way that is
perhaps not readily understandable by us
humans you know you might say that's a
that's a very weird situation how can we
have built this thing that behaves in a
way that we can't understand that's full
of computational irreducibility Etc et
cetera et cetera you know what what is
this what's it going to feel like when
the world is run by AIS whose operations
we can't understand and the thing one
realizes is actually we've seen this
before that's what happens when we exist
in the natural world the natural world
is full of things that operate according
to definite rules they have all kinds of
you know computational irreducibility we
don't understand what the natural world
is doing occasionally and you know when
you say you know are the AIS going to
wipe us out for example well it's kind
of like is the machination of the AIS
going to lead to this thing that
eventually comes and destroys the
species well we can also ask the same
thing about the natural world or the
machination of the natural world going
to eventually lead to this thing that's
going to you know make make the earth
explode or something like this those are
those are questions those are and
insofar as we think we understand what's
happening in the natural world that's a
result of Science and natural science
and so on one of the things we can
expect when there's this giant
infrastructure of the AIS is that's
where we have to kind of invent a new
kind of natural science that kind of is
the natural science that explains to us
how the AIS work it's kind of like we
can we can you know we have a I don't
know a horse or something and we're
trying to get it we're trying to you
know ride the horse and go from here to
there we don't really understand how the
horse Works inside but we can get
certain rules and certain you know
approaches that we take to persuade the
horse to go from here to there and and
and take us there and that's the same
type of thing that we're kind of dealing
with with the sort of incomprehensible
computationally irreducible AIS but we
can identify these kinds of we can find
these kind of pockets of reducibility
that we can kind of uh you know the I
don't know we're grabbing onto the main
of the horse or something to be able to
to write it
um or we figure out you know if we if we
do this or that to to ride the horse
that that's a a successful way to to get
it to do what what we're interested in
doing there does seem to be a difference
between a horse
and a um
a large language model
or something that could be called AGI
connected to the internet so let me just
ask you about big philosophical question
about the threats of these things
there's a lot of people like Eliezer
adkowski who worry about the existential
risks of AI systems is that something
that you worry about you know sometimes
when you're building an incredible
system like wolf from alpha you can
kind of get lost in it I like try and
think a little bit about the
implications of what one's doing you
know it's like the Manhattan Project
kind of situation where you're like it's
some of the most incredible physics in
engineering being done but it's like huh
where's this gonna go I think some of
these arguments about kind of you know
they'll always be a smarter AI they'll
always be you know and eventually the
AIS will get smarter than us and then
all sorts of terrible things will happen
to me some of those arguments remind me
of kind of the ontological arguments for
the essence of God and things like this
they're kind of arguments that are based
on some particular model fairly simple
model often of kind of there is always a
greater this that and the other you know
this is um and that's you know those
arguments ten what tends to happen in
the sort of reality of how these things
develop is that it's more complicated
than you expect that the kind of simple
logical argument that says oh eventually
there'll be a super intelligence and
then it will you know do this and that
turns out not to really be the story it
turns out to be complicated story so for
example here's an example of an issue is
there an apex intelligence just like
there might be an apex predator in some
you know ecosystem is there going to be
an apex intelligence the most
intelligent thing that there could
possibly be right I think the answer is
no and in fact we already know this and
it's a kind of a back to the whole
computational reducibility story there's
kind of a question of you know even if
you have um
if you if you have sort of a uh a turing
machine and you have a turing machine
that that runs as long as possible
before before it halts you say is this
the machine is this the Apex machine
that does that there will always be a
machine that can go longer and as you go
out to the infinite collection of
possible Turing machines you'll never
have reached the end so to speak you'll
never you'll always be able to it's kind
of like the same same question of
whether there'll always be another
invention will you always be able to
invent another thing the answer is yes
there's an infinite Tower of possible
inventions that's one definition of apex
uh but the other is like
which I also thought you were which I
also think might be true is is there a
species that's the Apex intelligence
right now on Earth
so it's not trivial to say that humans
are that yeah it's not trivial I agree
it's a you know I think one of the
things that I I've long been curious
about kind of other intelligences so to
speak
um I mean I you know I I view
intelligence is like computation and
it's kind of a you know you're sort of
you have the set of rules you deduce
what happens
um I have tended to think now that
there's this kind of specialization of
computation that is sort of a
consciousness-like thing that has to do
with these you know computational
boundedness single thread of experience
these kinds of things that are the
specialization of computation that
corresponds to a somewhat human-like
experience of the world now the question
is so so that's you know there may be
other intelligences like you know you
know the aphorism you know the weather
has a mind of its own it's a different
kind of intelligence that can compute
all kinds of of things that are hard for
us to compute but it is not well aligned
with us with the way that we think about
things it doesn't it doesn't it doesn't
think the way we think about things and
you know in this idea of different
different intelligences every different
mind every different human mind is a
different intelligence that thinks about
things in different ways and you know in
in terms of the kind of formalism of our
physics project we talk about this idea
of rule space the space of all possible
sort of rule systems and different minds
are in a sense of different points in
real space human Minds ones that have
grown up with the same kind of culture
and ideas and things like this might be
pretty close in real space pretty easy
for them to communicate pretty easy to
translate pretty easy to move from one
place in rural space that corresponds to
one mind to another place in rule your
space that corresponds to another sort
of nearby mind when we deal with kind of
more distant things in rural space like
you know the the pet cat or something
um you know the pet cat has some aspects
that are shared with us the emotional
responses of the cat are somewhat
similar to ours but the cat is further
away in real space than people are and
so then the question is you know can we
identify sort of the can we make a
translation from our thought processes
to the thought process of a cat or
something like this and you know what
what will we get when we you know what
what will happen when we get there and I
think it's the case that that many you
know many animals I don't know dogs for
example you know they have a labradal
factory systems they you know they they
have sort of the smell architecture of
the of the of the world so to speak in a
way that we don't and so you know if if
you were sort of talking to the dog and
you could you know communicate in a
language the dog will say well this is a
you know a a you know a flowing smelling
this that and the other thing Concepts
that we just don't have any idea about
now what's what's interesting about that
is one day we will have chemical sensors
that do a really pretty good job you
know we'll have artificial noses that
work pretty well and we might have our
augmented reality system show us kind of
the same map that the dog could see and
things like this so that you know
similar to what happens in the dog's
brain and eventually we will have kind
of expanded in real space to the point
where we will have those same sensory
experiences that dogs have and we will
have internalized what it means to have
you know the smell landscape or whatever
and and so then we will have kind of
colonized that part of Royal space
um until you know we haven't gone you
know some things that that you know
animals and so on do we've sort of
successfully understand others we do not
and the question of of what kind of what
is the uh you know what what
representation you know how how do we
convert things that animals think about
to things that we can think about that's
not a trivial thing
um and you know I've I've long been
curious I've had a very bizarre project
at one point of of trying to make an
iPad game that a cat could win against
its owner right it feels like there's a
deep philosophical
go there though yes yes I mean the the
you know I was curious if you know if
pets can work in Minecraft or something
and can construct things what will they
construct and will what they construct
be something where we look at and we say
yeah I recognize that or will it be
something that looks to us like
something that's out there in the
computational universe that one of my
you know cellular automata might have
produced where we sell yeah I can kind
of see it operates good into some rules
I don't know why you would use those
rules I don't know why you would care
yeah I uh actually it's just a link on
that seriously is there a connector in
the Royal space between you and a cat
where the cat could legitimately win so
iPad is a very limited
um interface yeah I I wonder if there's
a game where cats win I think the
problem is the cats don't tend to be
that interested in what's happening on
the iPad yeah that's an interface issue
probably yeah right right no I I think
it is likely that I mean you know there
are plenty of animals that would
successfully eat us if we were you know
if we were exposed to them and so
there's you know it it's gonna pounce
faster than we can get out of the way
and so on so there are plenty of and and
probably it's going to you know we think
we've hidden ourselves but we haven't
successfully hidden ourselves that's a
physical strength I wonder if there's uh
something in more in the realm of uh
intelligence where an animal like a cat
could out well I think there are things
certainly in terms of the the speed of
processing certain kinds of things for
sure I mean the the question of what you
know is there a game of chess for
example is there cat chess yeah that the
cats could play against each other and
if we tried to play a cat we don't
always lose I don't know you might have
to do with speed but it might have to do
with Concepts also there might be
Concepts and the cats had I I tend to
think that our species from its
invention of language has managed to
build up this kind of tower of
abstraction that for things like a
chess-like game will make us win in
other words we've become through the
fact that we've kind of experienced
language and learned abstraction you
know we've sort of become smarter at
those kinds of abstract kinds of things
now you know that doesn't make us
smarter at catching a mouse or something
it makes us smarter at the things that
we've chosen to to sort of con you know
concern ourselves which are these kind
of abstract things yeah and I think you
know this is again back to the question
of of you know what does one care about
you know if one's a if one's the you
know the cat if you if you have the
discussion with a cat if we can if we
can translate things to have the
discussion with a cat the cat will say I
you know I'm very excited that uh this
light
is moving and we'll say why do you care
and the cat will say that's the most
important thing in the world yeah that
this thing moves around I mean it's like
when you ask about I don't know you you
look at archaeological remains and you
say these people had this you know
belief system about this and you know
that was the most important thing in the
world to them
um and and now we look at it and say we
don't know what the point is but it was
I mean I I've been curious you know that
these hand prints on caves from 20 000
or more years ago and it's like nobody
knows what these handprints were there
for you know that they may have been a
representation of the most important
thing you can imagine they may just have
been some you know some kid who who
rubbed their hands in the mud and stuck
them on the walls of the cave you know
we don't we don't know and I think but
this this whole question of what
um you know is
when you say uh this question of sort of
what's the smartest thing around there's
the question of what kind of computation
are you trying to do if you're saying
you know if you say you've got some
well-defined computation and how do you
implement it well you could implement it
by nerve cells you know firing you can
implement it with silicon and
electronics you can implement it by some
kind of molecular computation process in
the human immune system or in some
molecular biology kind of thing they're
different ways to implement it and you
know I think this question of of uh of
sort of which you know those different
implementation methods will be of
different speeds they'll be able to do
different things if you say uh you know
which so an interesting question would
be
um
what kinds of abstractions are most
natural in these different kinds of
systems so for a cat it's for example
you know the visual scene that we see
you might you know we pick out certain
objects we recognize you know certain
things in that visual scene a cat might
in principle recognize different things
I I suspect you know Evolution
biological evolution is very slow and I
suspect what a cat notices is very
similar we even know that from some
neurophysiology what a catnosis is is
very similar to what we notice of course
there's a you know one obvious
differences cats have only two kinds of
color receptors so they don't see in the
same kind of color that we do now you
know we say we're we're better we have
three color receptors you know red green
blue we're not the overall winner I
think the the I think the mantis shrimp
is the overall winner winner with 15
color receptors I think so it can it can
kind of make distinctions that with our
current you know like the Mantis
shrimp's view of reality is inside at
least in terms of color is much richer
than ours
um now but what's interesting is how do
we get there so imagine we have this
augmented reality system that is even
you know it's singing into the infrared
into the ultraviolet things like this
and it's Translating that into something
that is connectable to our brains either
through our eyes or more directly into
our brains
um you know then eventually our kind of
web of the types of things we understand
will extend to those kinds of constructs
just as they have extended I mean there
are plenty of things where we see them
in the modern world because we made them
with technology and now we understand
what that is but if we'd never seen that
kind of thing we wouldn't have a way to
describe it we wouldn't have a way to
understand it and so on all right so
that actually stemmed from our
conversation about whether AI is going
to kill all of us and you we've
discussed this kind of
spreading of intelligence through rule
space that in practice it just seems
that things get more complicated things
are more complicated than the story of
well if you build a thing that's plus
one intelligence that thing will be able
to build the thing that's plus two
intelligence and plus three intelligence
and that will be exponential it'll
become uh more intelligent exponentially
faster and so on until it completely
destroys everything
um but you know that intuition might
still not be so simple but I might still
care carry validity and there's two
interesting trajectories here one a
super intelligence system remains in
rulio
proximity to humans to where we're like
holy crap this thing is really
intelligent uh let's select the present
and then there could be perhaps more
terrifying intelligence that starts
moving away they might be around us now
they're moving far away in rural space
but they're still sharing physical
Resources with us yes yes so they can
rob us of those physical resources and
Destroy Humans just kind of casually
yeah just just uh like nature code like
nature could but it seems like there's
something unique about AI systems where
um
there is a this kind of exponential
growth like the way well sorry Nature
has so many things in it one of the
things that nature has which is very
interesting are viruses for example
there is systems within nature that have
this kind of exponential effect
and that terrifies us humans because it
can you know there's only eight billion
of us and you can just kind of
it's not that hard to just kind of whack
them all real quick so uh I mean is that
something you think about that yeah I
thought about that yes the threat of it
yeah you as concerned about it as uh
somebody like Elias yakovski for example
just big big painful negative effects of
AI about Society
you know no but perhaps that's because
I'm intrinsically an optimist
um I mean I think that there are things
I think the thing that one you know one
sees is there's going to be this one
thing and it's going to just zap
everything somehow you know I maybe I
have faith in computational
irreducibility so to speak that there's
always unintended little Corners that
you know it's just like somebody says
I'm going to oh I don't know somebody
has some some bio weapon and they say
we're going to release this and it's
going to do all this harm but then it
turns out it's more complicated than
that because you know the kind of some
humans are different and you know the
exact way it works is a little different
than you expect it's something where
sort of the the the great big you you
know you smash the thing with something
you know you the asteroid collides with
the Earth yeah and it kind of you know
and yes you know the Earth is cold for
two years or something and you know then
lots of things die but not everything
dies and it's you know there's there's
usually I mean I I kind of this is in a
sense the sort of story of computational
irreducibility there are always
unexpected Corners there always
unexpected consequences and I don't
think that they're kind of whack it over
the head with something and then it's
all gone is you know that can obviously
happen the Earth can be swallowed up in
a black hole or something and then it's
kind of presumably presumably all over
um the uh but but you know I think this
question of of what you know what what
do I think the realistic paths are I
think that there will be sort of an
increasing I mean the the people have to
get used to phenomena like computational
irreducibility there's an idea that we
built the machines so we can understand
what they do and we're we're going to be
able to control what happens well that's
not really right now the question is is
the result of that lack of control going
to be that the machine's kind of
conspire and sort of wipe us out maybe
just because I'm an optimist I don't
tend to think that that's you know
that's in the cards I think that the you
know as a realistic thing I suspect you
know what will sort of emerge maybe is
kind of an ecosystem of the AIS just as
you know again I I don't really know I
mean this is something that's it's hard
to it's hard to be clear about what will
happen
um I mean I think that the you know
there are there are a lot of sort of
details of you know what could we do
what systems in the world could we
connect and AI to you know I have to say
I was just a couple of days ago I was
working on this uh chat gbt plug-in kit
that we have for open language okay
where you can you know you can create a
plugin and it runs with language code
um and it can run more from language
code back on your own computer yeah and
I was thinking well I can just make it
you know I can tell chat gbt create a
piece of code and then just run it on my
computer and I'm like you know that that
sort of personalizes for me the what
could what could possibly go wrong so to
speak was that exciting or scary that
possibility
it was a little bit scary actually
because it's kind of like like I realize
I'm I'm delegating to the AI just write
a piece of code you know you're in
charge write a piece of code run it on
my computer
and pretty soon all my files exactly
let's take a uh that's like Russian
Roulette but like much more complicated
yeah yes yes right that's a good
drinking game I don't know so well uh
right I mean
it's an interesting question then if you
do that right what is the sandboxing
that you should have and that's sort of
a that's a a version of of that question
for the world that is as soon as you put
the AIS in charge of things you know how
much how many constraints should there
be on these systems before you put the
AIS in charge of all the weapons and all
these you know all these different kinds
of systems well here's the fun part
about sandboxes is uh the AI knows about
them it has the tools to uh crack them
look the fundamental problem of computer
security is computational irreducibility
yes because the fact is any sandbox is
never any you know it's never going to
be a perfect sandbox if you want the
system to be able to do interesting
things I mean this this is the the
problem that's happened the generic
problem of computer security that as
soon as you have your you know firewall
that is sophisticated enough to be a
universal computer that means it can do
anything and so long as if you find a
way to poke it so that you actually get
it to do that Universal computation
thing that's the way you kind of crawl
around and get her to do the thing that
it wasn't intended to do and that's sort
of a another version of computational
irreducibility is you can you know you
can kind of you get it to do the thing
you didn't expect it to do so to speak
there's so many interesting
possibilities here that manifest
themselves from the compute
computationally reducibility here
it's just so many things could happen
because in digital space things move so
quickly you can have a chat bot you can
have a piece of code that
you could basically have chat GPT
generate viruses accidentally or on
purpose and they uh digital viruses yes
and uh they could be brain viruses too
they they convince kind of like uh
phishing emails yes they can convince
you of stuff yes and no doubt you can
you know in a sense we've had the loop
of the machine learning Loop of making
things that convince people of things is
surely going to get easier to do yeah
and you know then what does that look
like well it's again you know we humans
are you know we're this is a new
environment for us and admittedly it's
an environment which a little bit
scarily is is changing much more rapidly
but than I mean you know people worry
about you know climate change is going
to happen over hundreds of years and you
know the environment is changing but the
environment for you know in the the kind
of digital environment might change in
in six months
so one of uh the relevant
concerns here in terms of the uh impact
of GPT on society is the nature of Truth
that's relevant to Wolfram Alpha because
computation
through symbolic reasoning that's
embodied in Wolfram Alpha is the
interface there's a kind of sense that
what Wolfram Alpha tells me is true
so we hope
yeah I mean you could probably analyze
that you could show
you can't prove that it's always going
to be true computational disability uh
but it's gonna be more
true than not it's look the fact is it
will be the correct consequence of the
rules you've specified and insofar as it
talks about the real world you know that
is our job in sort of curating and
collecting data to make sure that that
data is quotes as true as possible now
what does that mean well you know it's
always an interesting question I mean
for us our operational definition of
truth is
you know somebody says who's the best
actress
who knows but somebody won the Oscar and
that's a definite fact yeah and so you
know that's the kind of thing that we
can make computational as a piece of
truth if you ask you know these things
which you know a sensor measured this
thing it did it this way a machine
Learning System this particular machine
Learning System recognized this thing
that's a that's a sort of a definite uh
effect so to speak and that's uh you
know there are there is a good network
of those things in the world it's
certainly the case that uh particularly
when you say is so and so a good person
you know that's a that's a hopelessly
you know we might have a computational
language definition of good I don't
think it'd be very interesting because
that's a very messy kind of concept not
really amenable to uh kind of you know
that I think as far as we will get with
those kinds of things is I want X
there's a kind of meaningful calculus of
I want X
um and that has various consequences I
mean I'm not sure I haven't I haven't
thought this through properly but I
think you know a concept like is so and
so a good person is that true or not
that's a mess that's a mess that's
amenable to computation I think I think
it's a mess when humans try to Define
what's good uh like through legislation
but when humans try to Define what's
good through literature
through uh history books through poetry
it starts well I don't know I mean that
particular thing it's kind of like you
know we're we're going into kind of the
ethics of what what counts as good so to
speak and you know what do we think is
right and so on and I think that's a
thing which you know one feature is uh
we don't all agree about that there's no
theorems about kind of uh you know
there's no there's no theoretical
framework that says this is this is the
way that ethics has to be well first of
all there's stuff we kind of agree on
and there's some empirical backing for
what works and what doesn't from just
even the morals and ethics within
religious texts so we seem to mostly
agree that murder is bad the certain
universals that seem to emerge I wonder
where the murder of an AI is bad
well I tend to think yes but uh and I
think we're gonna have to contend with
that question
oh and I wonder what AI would say yeah
well I think you know one of the things
with with AIS is it's one thing to wipe
out that AI that is only you know it has
no owner you can even easily imagine an
AI kind of hanging out on the on the you
know on on the internet without having
any particular owner or anything like
that
um and then you say well well what harm
does it you know it's it's okay to get
rid of that AI because of the AI has 10
000 friends who are humans and all those
you know all those ten thousand humans
will be incredibly upset that this AI
just got exterminated it becomes a
slightly different more entangled story
but yeah I know I think that this
question about what do humans agree
about it's uh you know there are certain
there's certain things that you know
human laws have tended to uh
consistently agree about
um you know there have been times in
history when people have sort of gone
away from certain kinds of laws even
ones that we would now say how could you
possibly have not not done it that way
um you know that just doesn't seem right
at all
um but I think I mean this question of
what
I don't think one can say Beyond saying
if you have a set of rules that will
cause the species to go extinct
that's probably you know you could say
that's probably not a winning set of
laws because even to have a thing on
which you can operate laws requires that
the species not be extinct but between
sort of what's the distance between
Chicago and New York
that wolf from alpha can answer and the
question of if this person is good or
not there seems to be a lot of gray area
and that starts becoming really
interesting I think you're
uh since the creation of Wolfram Alpha
have been a kind of arbiter of Truth at
a large scale so this system is
generates more truth and try to make
sure that the things are true I mean
look it's a practical matter when people
write computational contracts and it's
kind of like you know if this happens in
the world then do this yes and this
hasn't developed as as quickly as it
might have done you know this has been a
sort of a blockchain story in part and
so on although blockchain is not really
necessary for the idea of computational
contracts but you can imagine that
eventually sort of a large part of
what's in the world are these giant
chains and networks of computational
contracts and then something happens in
the world and this whole giant domino
effect of contracts firing autonomously
that cause other things to happen and
you know for us you know we've been the
main sort of source the Oracle of of
quotes facts or truth or something for
things like blockchain computational
contracts and such much like and there's
a question of you know what you know I
consider that responsibility to actually
get the stuff right and one of the
things that is tricky sometimes is when
is it true when is it a fact when is it
not a fact yes I think the best we can
do is to say uh you know we we have a
procedure we follow the procedure we
might get it wrong but at least we won't
be corrupt about getting it wrong so to
speak so that's beautifully put and have
a transparency about the procedure
foreign
the problem starts to emerge when the
things that you convert into
computational language start to expand
for example into the realm of politics
so this is where it's almost like this
nice dance of wolf from Alpha and uh
Chad gbt Chad gbt like you said is a
shallow and Broad so it's going to give
you an opinion on everything but it
writes fiction as well as fact which is
exactly how it's built I mean that's
exactly it is making language and it is
making both even in code it writes
fiction I mean it's kind of fun to see
sometimes you know it'll write fictional
World from language code yeah that that
term it kind of looks right yeah it
looks right but it's actually not
pragmatically correct yeah um but but
yes it's it's a it has a view of kind of
roughly how the world works at the same
level as as books of fiction talk about
roughly how the world Works they just
don't have happened to be the way the
world actually worked or whatever but
yes that that's um no I I agree that's
sort of a um you know we are attempting
with with our whole you know wolfen
language computational language thing to
represent uh at least well it's either
it doesn't necessarily have to be how
the actual World works because we can
invent a set of rules that aren't the
way the actual World works and run those
rules but then we're saying we're going
to accurately represent the results of
running those rules which might or might
not be the actual rules of the world but
we also try to capture features of the
world uh as accurately as possible to
represent what happens in the world now
again as we've discussed you know the
the atoms in the world arranged you know
you say I don't know you know was there
a tank that showed up you know that that
you know drove somewhere okay well you
know what is a tank it's in a management
of atoms that we abstractly describe as
a tank and you could say well you know
there's some arrangement of atoms that
is a different derangement of atoms but
it's and it's not you know we didn't we
didn't decide it's like this Observer
Theory question of you know what what
arrangement of atoms counts as a tank
versus not a tank so there's there's
even things that would consider strong
facts
you could start to kind of disassemble
them and show that they're not right I
mean so so the question of whether oh I
don't know
was this gust of wind strong enough to
blow over this particular thing well a
gust of wind is a complicated concept
you know it's full of little pieces of
fluid dynamics and little vortices here
and there and you have to Define you
know was it you know what the aspect of
the gust of wind that you care about
might be it put this amount of pressure
on this you know blade of some some you
know wind turbine or something
um and uh you know that that's the um um
and but but you know if you say if you
have something which is the fact of the
gust of wind was the strong or whatever
that you know that is you have to have
some definition of that you have to have
some measuring device that says
according to my measuring device that
was constructed this way the Guster wind
was this so what can you say about the
nature of truth that's useful for us to
understand chat GPT because you've been
con
you've been contending with this idea of
what is fact and not and it seems like
Chachi Patrice used a lot now I've seen
it used by journalists to write articles
and so you have
um people that are working with large
language models trying to desperately
figure out how do we
essentially censor them through
different mechanisms either manually or
through reinforce and learning with
human feedback try to align them to to
not say fiction just to say non-fiction
as much as possible this is the
importance of computational language as
an intermediate it's kind of like you've
got the large language model it's able
to suffice something which is a formal
precise thing that you can then look at
and you can run tests on it and you can
do all kinds of things it's always going
to work the same way and it's precisely
defined what it does and then the large
language model is the interface I mean
the way I view these large language
models one of their important I mean
there are many use cases and you know
it's a remarkable thing could talk about
some of these you know literally you
know every day we're coming up with a
couple of new use cases
um some of which are very very very
surprising
um and things where I mean but the best
use cases are ones where it's you know
even if it gets it roughly right it's
still a huge win like a use case we had
from a week or two ago is read our bug
reports you know we've got hundreds of
thousands of bug reports that will be
accumulated over decades and it's like
you know can we have it just read the
bug report figure out where the where is
the bug likely to be and you know home
in on that piece of code maybe they'll
even suggest some some you know sort of
way to fix the code it might get that it
might be nonsense what it says to about
how to fix the code but it's incredibly
useful that it was able to you know so
awesome it's so awesome because even the
nonsense will somehow be instructive I
don't I don't quite understand that yet
I've
yeah there's so many programming related
things like uh for example uh
translating for one programming language
to another is really really interesting
it's extremely effective but then you
the failures
reveal the path forward also yeah but I
think I mean the the big thing I mean in
that kind of discussion the unique thing
about our computational language is it
was intended to be read by humans yes
and so it's really important right and
so it has this thing where you can but
but you know thinking about sort of
church EBT and its use and so on the one
of the big things about it I think is
it's a linguistic user interface that is
so a typical use case might be in the
take the journalist case for example
it's like let's say I have five facts
that I'm trying to turn into an article
or I'm trying to I'm trying to write a
report where I have basically five facts
that I'm trying to include in this
report but then I feed those five facts
to chat gbt it Puffs them out into this
big report
and then and then that's a good
interface for uh you know another if I
just gave
if I just had in my terms those five
bullet points and I gave them to some
other person the person would say I
don't know what you're talking about
because these are you know this is your
version of this sort of quick notes
about these five bullet points but if
you puff it out into this thing which is
kind of connects to the collective
understanding of language then somebody
else can look at it and say okay I
understand what you're talking about now
you can also have a situation where that
thing that was puffed out is fed to
another large language model you know
it's kind of like you know you're
applying for the permit to you know uh I
don't know grow fish in some place or
something like this and it uh you know
it it um
um and and you have these facts that
you're putting in you know I'm going to
have a you know I'm gonna you know have
this kind of water and I don't know what
it is
um you just got a few bullet points it
Puffs it out into this big application
you fill it out then at the other end
the you know the Fisheries Bureau has
another large language model that just
crushes it down because the Fisheries
Bureau cares about these three points
and it knows what it cares about and it
then so it's really the the natural
language produced by the larger language
model is sort of a transport layer that
you know is really llm communicates with
llm I mean it's kind of like the you
know I write a piece of email using my
llm and you know puff it out from the
things I want to say your llm turns it
into and the conclusion is X now the
issue is you know that the thing is
going to make this thing that is sort of
semantically plausible
and it might not actually be what you
you know it might not be kind of relate
to the world and the way that you think
it should relate to the world now I've
seen this you know I've been doing okay
I'll give you a couple of examples
um I was doing this thing when we
announced this this uh plugin for for
chat GPT I had this lovely example of a
math word problem some complicated thing
and it did a spectacular job of taking
apart this elaborate thing about you
know this person has twice as many
chickens as this et cetera et cetera et
cetera and it turned it into a bunch of
equations it fed them to wolfen language
we solved the equations everybody did
great we gave back the results and I
thought okay I'm going to put this in
this blog post I'm writing okay I
thought I'd better just check and turns
out it got everything all the hard stuff
it got right in the very end last two
lines it just completely goofed it up
and gave the wrong answer and I would
not have noticed this same thing
happened to me two days ago okay so I I
thought you know I made this with this
chat gbt plug-in kit I made a thing that
would emit a sound would play a tune on
my local computer right so chat gbt
would produce you know a series of notes
and it would play this tune on my
computer very cool okay so I thought I'm
gonna ask it play the tune that Hal
sang when Hal was being disconnected in
2001 okay so it it there it is Daisy was
it Daisy yes Daisy yeah right so it's
okay so I think you know and so it
produces a bunch of notes and I'm like
this is spectacular this is amazing and
then I thought I was just going to put
it in and then I thought I'd better
actually play this
and so I did and it was Mary had a
little lamb oh wow
oh wow but it was Mary had a little lamb
yeah yes wow so it's correct but wrong
yes it was uh it could easily be
mistaken yes right and in fact I I kind
of gave the I had this quote from Hal to
explain you know it's it's as it the the
how you know states in the movie you
know it's uh the HAL 9000 is you know
the thing was just a rhetorical device
because I'm realizing oh my gosh you
know this Chachi BT you know could have
easily fooled me I mean it did this it
did all the it did this amazing thing of
knowing this thing about the movie and
being able to turn that into the the
notes of the song except it's the wrong
song yeah and uh you know how in in the
movie Hal says you know I think it's
something like you know know how No 9
000 series computer has ever been found
to make an error we are for all
practical purposes perfect and um
incapable era and I thought that was
kind of a Charming sort of uh quote from
uh from Hal to make in connection with
uh with what church yeah in that case
the interesting things about the L alums
like you said that they are very willing
to admit their error
well yes I mean that's a question of the
RH uh the reinforcement learning human
feedback thing oh right that that that's
you know the thing
the the really remarkable thing about
chat GPT is you know I had been
following what was happening with large
language models and I'd play with them a
whole bunch and they were kind of like
yeah you know it's kind of like what you
would expect based on sort of sort of
statistical continuation of language
it's interesting but it's not breakout
exciting and then I think the kind of
the the kind of reinforcement that the
human feedback reinforcement learning uh
you know in in making chat gbt try and
do the things that humans really wanted
to do that broke through that kind of
reached the threshold where the thing
really is interesting to us humans and
by the way it's interesting to see how
you know you change the temperature
something like that the thing goes
Bonkers and it no longer is interesting
to humans it's producing garbage yeah um
and it's it's kind of right it's somehow
it managed to get this this Above This
threshold but it really is well aligned
to what we humans are interested in and
uh and kind of that that's um and I
think you know nobody saw that coming I
think uh certainly nobody I've talked to
and nobody who is involved in in that
project seems to have known that was
coming it's just one of these things
that is a sort of remarkable threshold I
mean you know when we built wolf from
alpha for example I didn't know it was
going to work you know we tried to build
something that would have enough
knowledge of the world that it could
answer a reasonable set of questions
that we could do but good enough natural
language understanding the typical
things you type in would work we didn't
know where that threshold was I mean I
was not sure that it was the right
decade to try and build this even the
right you know 50 years to try and build
it you know and I think that was it's
the same type of thing with chat GPT
that I don't think anybody could have
predicted did that you know 2022 would
be the year that this this became
possible I think uh can you tell a story
about Marvin Minsky and showing it to
him and saying that like no no this time
it actually works yes yes you know it's
the same thing for me looking at these
large language models it's like when
when people were first saying the first
few weeks of chat gbt it's like oh yeah
you know I've seen these large language
models
um and then uh you know and then I
actually try it and uh you know oh my
gosh it actually works and I think uh
it's uh but it but you know the things
and the thing I found you know I
remember one of the first things I tried
was uh write a persuasive essay that a
wolf is the bluest kind of animal okay
so it writes this thing and it starts
talking about these uh wolves that live
on the Tibetan plateau and they are
named some Latin name and so on and I'm
like really and I'm starting to look it
up on the web and it's like well it's
actually complete nonsense
but it's extremely plausible I mean it's
plausible enough that I was going and
looking up on the web and wondering if
there was a wolf that was blue you know
I mentioned this on some live streams
I've done and so people have been
sending me these pictures
maybe I was on to something
can you kind of give your wise Sage
advice about what humans who have never
interacted with the eye systems
uh not even like with Will from Alpha
are now interacting with Chad GPT
because it becomes
it's accessible to a certain demographic
they may have not touched AI systems
before what do we do with truth like
journalists for example
yeah how do we think about the output of
these systems I think this idea
the idea that you're going to get
factual output is not a very good idea I
mean it's just this is not it is a
linguistic interface it is producing
language and language can be truthful or
not truthful and that's a different
slice of what's going on I think that
you know what we see
and for example uh kind of you know go
check this with your fact source for
example you can do that to some extent
but then it's going to not check
something it's going you know that is
again a thing that is sort of a does it
check in the right place I mean we see
that in you know does it call the you
know the Wolfram plug-in in the right
place you know often it does sometimes
it doesn't you know I I think the the
real thing to understand about what's
happening is which I think is very
exciting is kind of the the great
democratization of access to computation
yeah and and um you know I think that
when you look at
sort of the there's been a long period
of time when computation and the ability
to figure out things with computers has
been something that kind of only the
only The Druids at some level can can
achieve you know I myself have been
involved in trying to sort of
de-druidify
um access to computation I mean back
before Mathematica existed you know in
1988 if you were a you know a physicist
or something like that and you wanted to
do a computation you would find a
programmer you would go and you know
delegate the the computation to that
programmer hopefully they'd come back
with something useful maybe they
wouldn't they'd be this long you know
multi-week you know Loop that you go
through and then it was actually very
very interesting to see 1988 you know
like first people like physicists
mathematicians and so on then other lots
of other people but this very rapid
transition of people realizing they
themselves could actually type with
their own fingers and you know make make
some piece of code that would do a
computation that they cared about and
you know it's been exciting to see lots
of discoveries and so on made by by
using that tool and I think the same
thing is you know we see the same thing
you know wolfen Alpha is dealing with uh
is not as deep computation as you can
achieve with whole world from language
Mathematica stack but the thing that's
to me particularly exciting about kind
of the large language model linguistic
interface mechanism is it dramatically
broadens the access to kind of deep
computation I mean it's kind of like one
of the things I've sort of thought about
recently is you know what's going to
happen to all these programmers what's
going to happen to all these people who
you know a lot of what they do is write
slabs of boilerplate code and in a sense
you know I've been saying for 40 years
that's not a very good idea you know you
can automate a lot of that stuff with a
high enough level language that slab of
code that's designed in the right way
you know that slab of code turns into
this one function we just implements it
that you can just use
um so in a sense that the fact that
there's there's all of this activity of
doing sort of lower level programming is
something for me it seemed like I don't
think this is the right thing to do but
you know and and lots of people have
used our technology and not had to do
that but the fact is that that's you
know so when you look at I don't know
computer science departments that have
that have turned into places where
people are learning the trade of
programming so to speak
it's it's sort of a question of what's
going to happen
and I think there are two Dynamics one
is that kind of uh sort of uh
boilerplate programming is going to
become you know it's going to go the way
that Assembly Language went back in the
day of something where it's really
mostly specified by at a higher level
you know you start with natural language
you turn it into a computational
language that's you look at the
computational language you run tests you
understand that's what's supposed to
happen you know if we do a great job
with compilation of the of the the you
know of the computational language it
might turn into llvm or something like
this but um uh you know or just directly
gets gets run through the algorithms we
have and so on but
but then so that's kind of a a tearing
down of this kind of this big structure
that's been built of of teaching people
programming but on the other hand the
other Dynamic is vastly more people are
going to care about computation so all
those Departments of you know art
history or something that really didn't
use computation before now have the
possibility of accessing it by virtue of
this kind of linguistic interface
mechanism and uh if you create an
interface that allows you to interpret
the debug and interact with the
computational language
then that makes it even more accessible
yeah well I mean the I think the thing
is that right now you know the average
you know art history student or
something probably isn't going to you
know they're not probably they don't
think they know about programming and
things like this but by the time it
really becomes a kind of purely you know
you just walk up to it there's no
documentation you start just typing you
know compare these pictures with these
pictures and you know see the use of
this color whatever and you generate
this piece of of computational language
code that gets run you see the results
you say oh that looks roughly right or
you say that's crazy
um and maybe then you eventually get to
say well I better actually try and
understand what this computational
language code did
um and and that becomes the thing that
you learn just like it's kind of an
interesting thing because unlike with
mathematics where you kind of have to
learn it before you can use it this is a
case where you can use it before you
have to learn it well I get a sad
possibility here or maybe exciting
possibility that very quickly people
won't even look at the computational
language they'll trust that it's
generated correctly as you get better
and better generating that language uh
yes I think that there will be enough
cases where people see you know because
you can make it generate tests too and
and so you'll say
um we're doing that I mean that's it's a
pretty cool thing actually yes because
you you know say this is the code and
you know here are a bunch of examples of
running the code yeah okay people will
at least look at those and they'll say
that example is wrong and you know then
it'll kind of wind back from there and I
agree that that the the kind of the
intermediate level of people reading the
computational language code in some case
people will do that in other case people
just look at the tests
and or even just look at the results and
sometimes it'll be obvious that you got
the thing you wanted to get because you
were just describing you know make me
this interface that has two sliders here
and you can see it has that those two
sliders there and that's that's kind of
that's that's the result you want but I
I think you know one of the questions
then is in that setting where you know
you have this kind of ability broad
ability of people to access computation
what should people learn you know in
other words right now you you know you
go to Computer Science school so to
speak and a large part of what people
end up learning I mean it's been a funny
historical development because back you
know 30 40 years ago computer science
departments were quite small and they
taught you know things like final
automata Theory and compiler Theory and
things like this
um you know company like mine rarely
hired people who'd come out of those
programs because the stuff they knew was
I think it's very interesting I love
that theoretical stuff but in terms you
know it wasn't that useful for the
things we actually had to build build in
software engineering and then kind of
there was this big pivot in the in the
90s I guess where there was a big demand
for sort of I.T type programming and so
on and software engineering and then you
know big demand from students and so on
you know we want to learn this stuff and
uh and and I think you know the thing
that really was happening in part was
lots of different fields of human
endeavor were becoming computational you
know for all acts there was a there was
a computational x and this is a um uh
and that was the thing that um that
people were responding to
um and but then kind of this idea
emerged that to get to that point the
main thing you had to do was to learn
this kind of trade or or skill of doing
you know programming language type
programming and and that uh you know it
kind of is a strange thing actually
because I you know I remember back when
I used to be in the professor in
business which is now 35 years ago so
gosh that's rather long time
the um you know it was it was right when
they were just starting to emerge kind
of computer science departments that
sort of a fancy research universities
and so on I mean some had already had it
but the the other ones that that um were
just starting to have that and it was
kind of a a thing where they were kind
of wondering are we going to put this
thing that is essentially a a trade-like
skill are we going to somehow attach
this to the rest of what we're doing and
a lot of these kind of knowledge work
type activities have always seemed like
things where that's where the humans
have to go to school and learn all this
stuff and that's never going to be
automated yeah and you know this is It's
kind of shocking that rather quickly you
know a lot of that stuff is clearly
automatable and I think you know but the
question then is okay so if it isn't
worth learning kind of uh you know how
to do car mechanics you only need to
know how to drive the car so to speak
what do you need to learn and you know
in other words if you don't need to know
the mechanics of how to tell the
computer in detail you know make this
Loop you know set this variable you know
set up this array whatever else if you
don't have to learn that stuff you don't
have to learn the kind of under the hood
things what do you have to learn I think
the answer is you need to have an idea
where you want to drive the car in other
words you need to have some notion of
you know your you know you need to have
some picture of sort of what the what
the architecture of what is
computationally possible is well there's
also this kind of artistic element of um
of conversation because you ultimately
you use natural language to control the
car
so it's not just the where you want to
go well yeah you know it's interesting
it's a question of who's going to be a
great prompt engineer yeah okay so my
current theory this week good expository
writers are good prompt Engineers what's
an expository range so like uh somebody
who can explain stuff well huh police
department does that come from in the
University yeah I have no idea I think
they killed off all the expository
writing departments well there you go
strong words with Stephen Wolfram well I
don't know I don't I'm not sure if
that's right I mean I I actually am
curious because in fact I just sort of
initiated this kind of study of of
what's happened to different fields at
universities because like you know there
used to be geography departments at all
universities and then they disappeared
actually right before GIS became common
I think they disappeared you know
Linguistics departments came and went in
many universities it's kind of
interesting because these things that
people have thought were worth learning
at one time and then they kind of die
off and then you know I do think that
it's kind of interesting that for me
writing prompts for example people I
realize you know I think I'm an okay
expository writer and I realize when I'm
sloppy writing a prompt and I don't
really think because I'm thinking that
I'm just talking to an AI I don't need
to you know try and be clear in
explaining things that's when it gets
totally confused and I mean in some
sense you have been writing prompts for
a long time with wolf from alpha
thinking about this kind of stuff yeah
how do you convert natural language into
competition well right but that's a you
know the one thing that I'm wondering
about is uh you know it is remarkable
the extent to which you can address an
llm like you can address a human so to
speak and and I think that is because it
you know it learned from all of us
humans it's it's uh the reason that it
responds to the ways that we will
explain things to humans is because it
is a representation of how humans talk
about things but it is bizarre to me
some of the things that kind of are sort
of expository mechanisms that I've
learned in trying to write clear you
know expositions in English that you
know just for humans that those same
mechanisms seem to also be useful for
for for the llm but on top of that
what's useful is the kind of mechanisms
that maybe a psychotherapist employs
which is a kind of uh like almost
manipulative or game theoretic
interaction where Maybe
you would do with a friend like a
thought experiment that if this is the
last day you were to live or yeah if if
I ask you this question and you answer
wrong I will kill you those kinds of
problems seem to also help yes in
interesting ways yeah so it makes you
wonder like the way a therapist I think
would like a good therapist probably you
we create layers
in our human mind to between like uh
between between the outside world and
we'll just true what is true to us and
um maybe about trauma and all those
kinds of things so projecting that into
an llm maybe there might be a deep truth
that's it's concealing from you it's not
aware of it you get to that truth you
have to kind of really kind of
manipulate this yeah yeah right it's
like these jailbreaking jailbreaking for
llms and but the space of jailbreaking
techniques
as opposed to being fun little hacks
that could be an entire system sure yeah
I mean just think about the computer
security aspects of of how you you know
phishing and and computer security you
know fishing of humans yeah and fishing
of llms is is a is a they're very
similar kinds of things but I think I
mean this this um
uh you know this whole thing about kind
of the AI Wranglers AI psychologists all
that stuff will come the thing that I'm
curious about is right now the things
that are sort of prompt hacks are quite
human they're quite sort of
psychological human kinds of hacks the
thing I do wonder about is if we
understood more about kind of uh the
science of the llm will there be some
totally bizarre hack that is you know
like repeater word three times and put a
this that and the other there that
somehow plugs into some aspect of how
the llm works
um that is not you know that's kind of
like like an optical illusion for humans
for example like one of these mind hacks
for humans what are the Mind hacks for
the llms I don't think we know that yet
and that becomes a kind of
us figuring out reverse engineering the
language that controls the llms and the
thing is the reverse engineering can be
done by a very large percentage of the
population now because it's natural
language interface right it's kind of
interesting to see that you were there
at the birth of the computer science
department
as a thing and you might be there at the
death of the computer science term as
the thing well yeah I don't know there
were computer science departments that
existed earlier but the ones that the
broadening of of every University had to
have a computer science department yes I
was I was uh I watched that so to speak
and but I think the thing to understand
is okay so first of all there's a whole
theoretical area of computer science
that I think is great and you know
that's a fine thing the the
you know in a sense you know people
often say any field that has the word
science tacked onto it probably isn't
one yeah um and strong words right and
let's see uh nutrition science
Neuroscience that one's an interesting
one because that one is also very much
you know there's a that's a chat GPT
informed science in a sense because it's
it's kind of like the the big problem
Neuroscience has always been we
understand how the individual neurons
work we know something about the
psychology of how overall thinking works
yeah what's the kind of Intermediate
Language of the brain and nobody has
known that and that's been in a sense if
you ask what is the core problem of
Neuroscience I think that is the core
problem that is what is the level of
description of brains that's above
individual neuron firings and Below
psychology so to speak and I think what
chat GPT is showing us is well one one
thing about Neuroscience is you know one
could have imagined There's Something
Magic in the brain there's some weird
quantum mechanical phenomenon that we
don't understand one of the important
you know discoveries from chatgpt is
it's pretty clear you know brains can be
represented pretty well by simple
artificial neural net type models and
that means that's it that's what we have
to study now we have to understand the
science of those things we don't have to
go searching for you know exactly how
did that molecular biology thing happen
inside the synapses and you know all
these kinds of things we've we've got
the right level of modeling to be able
to explain a lot of what's going on and
thinking we don't necessarily have a
science of what's going on there that's
the that's the remaining challenge so to
speak but we you know we know we don't
have to dive down to some some different
layer but anyway we were talking about
things that had science in their name
yes and um you know I think that the uh
um
you know what what happens to computer
science well I think the thing that um
uh you know there is a thing that
everybody should know and that's how to
think about the world computationally
and that means you know you look at all
the different kinds of things we deal
with and there are ways to kind of have
a formal representation of those things
you know it's like well what is a what
is an image you know what how do we
represent that what is color how do we
represent that what is you know what are
all these different kinds of things what
is I don't know smell or something how
should we represent that what are the
shapes molecules and things that
correspond to that what is uh you know
these things about how do we represent
the world in some kind of formal level
and I think my my current thinking I'm
not real happy with this yet but um you
know it's kind of computer science it's
kind of Cs and what really is important
is kind of computational X for all X and
there's this kind of thing which is kind
of like CX not Cs and CX is a this kind
of computational understanding of the
world that isn't the sort of details of
programming and programming languages
and the details of how particular
computers are made it's this kind of way
of formalizing the world it's kind of
kind of a little bit like what logic was
going for back in the day and we're now
trying to find a formalization of
everything in the world you can kind of
see you know we made a poster years ago
of kind of the uh the the growth of
systematic data in the world so all
these different kinds of things that you
know there were sort of systematic
descriptions found for those things like
you know what point do people have the
idea of having calendars dates you know
a systematic description of what day it
was at what point did people have the
idea you know systematic descriptions of
these kinds of things and as soon as one
can you know people you know as a way of
sort of formulating how do you how do
you think about the world in a sort of a
formal way so that you can kind of build
up a tower of of cable abilities you
kind of have to know sort of how to
think about the world computationally it
kind of needs a name and it isn't you
know we implement it with computers so
that's we talk about it as computational
but really what it is is a formal way of
talking about the world what is the
formalism of the world so to speak and
how do we learn about kind of how to
think about different aspects of the
world in a formal way so I think
sometimes when you use the word formal
it uh kind of implies highly constrained
and perhaps that's not doesn't have to
be highly constrained so computational
thinking does not mean like logic it
knows it's a really really broad thing I
wonder I mean
I wonder if it's if you think natural
language will evolve such that
everybody's doing computational thinking
oh yes well so one question is whether
there will be a pigeon of computational
language and natural language yeah and I
found myself sometimes you know talking
to chat GPT trying to get it to write
wolf language code and I write it in
Pigeon form so that means I'm combining
you know uh you know Nest list this
collection of you know whatever you know
Nest list is a term from open language
and I'm combining that and chat does a
decent job of understanding that pigeon
probably would understand the pigeon
between English and French as well of
you know as a smooshing together of
those languages but yes I think that's
you know that's far from impossible and
what's the incentive for young people
that are like eight years old nine ten
they're starting to interact with Chad
GPT to learn the normal natural language
right the the full poetic language
what's the why
the same way we learn emojis and
shorthand when you're texting yes
they'll learn like language will have a
strong incentive to evolve into uh
maximally uh computational kind of like
perhaps you know I had this experience a
number of years ago I happened to uh be
visiting a person I know on the on the
west coast who's worked with a bunch of
kids aged I don't know 10 11 years old
or something who'd learned woven
language really well and these kids
learned it so well they were speaking it
and so show up in that like saying oh
you know this thing and they're speaking
this language I never heard it as a
spoken language they were very
disappointed that I couldn't understand
it at the speed that they were speaking
at it's like kind of I'm it's um and so
I think that's some I mean I've actually
thought quite a bit about how to turn
computational language into a convenient
spoken language I haven't quite figured
that out oh spoken because it's readable
right yeah it's readable as a you know
as a way that we would read text but if
you actually want to speak it and it's
useful you know if you're trying to talk
to somebody about writing a piece of
code it's useful to be able to say
something and it should be possible and
I think it's very frustrating it's one
of those problems that maybe I maybe
this is one of these things where I
should try and get an llm to help me how
to make it speakable maybe maybe it's
easier than you realize when you watch I
think it is easier I think it's one idea
or so I think it's I think it's going to
be something where you know the fact is
it's a tree structured language just
like human language is a tree structured
language and I think it's going to be
one of these things where one of the
requirements that I've had is that
whatever the spoken version is that
dictation should be easy that is that
shouldn't be the case that you have to
relearn how the whole thing works it
should be the case that you know that
open bracket is just a uh ah or
something and it's you know and then
um but you know human language has a lot
of tricks that are I mean for example
human language
has has features that are sort of
optimized keep things within the bounds
that our brains can easily deal with
like I you know I tried to teach a
Transformer neural net to do parenthesis
matching it's pretty crummy at that it
it um and then chat gbt is similarly
quite crummy at parenthesis matching you
can do it for small parenthesis things
for the same size of parenthesis things
where if I look at it as a human I can
immediately say these are match these
are not matched but as soon as it gets
big as soon as it gets kind of to the
point where sort of a deeper computation
is hopeless and but the fact is that
human language has avoided for example
the Deep subclauses you know we don't um
uh you know we we arrange things so we
don't end up with these incredibly deep
things
um because brains are not well set up to
deal with that and we it's found lots of
tricks and maybe that's what we have to
do to make sort of a spoken version a a
human speakable version because because
what we can do visually is a little
different what we can do in the very
sequentialized way that we that we hear
things in in the audio domain
let me just ask about MIT briefly so
there's now there's a College of
Engineering and there's a new College of
computing it's just interesting I want
to linger on this computer science
department thing so MIT has ex
electrical engineering computer science
um what do you think college of
computing will be doing like in 20 years
what what like well you see what happens
to computer science like really this is
the question this is you know everybody
should learn kind of whatever CX really
is okay this how to think about the
world computationally everybody should
learn those Concepts and uh you know
it's uh and and some people will learn
them at a quite quite formal level and
they'll learn computational language and
things like that other people will just
learn you know uh sound is represented
as you know Digital Data and they'll get
some idea of spectrograms and
frequencies and things like this and
maybe that doesn't or they'll learn
things like you know a lot of things
that are sort of data science
statistics-ish like if you say oh I've
got these you know these people who who
um uh picked their favorite kind of
candy or something and I've got um you
know what's the best kind of candy given
that I've done the sample of all these
people and they all rank the candies in
different ways you know how do you think
about that that's sort of a
computational x kind of thing you might
say oh it's I don't know what that is is
it statistics is it data science I don't
really know but kind of how to think
about a question like that oh like a
ranking of preferences yeah yeah and
then how to aggregate those those ranked
preferences into an overall thing you
know how does that work
um you know how how should you think
about that you know because you can just
tell you might just tell chat gbt sort
of I don't know even even the concept of
an average it's not obvious that you
know that's a concept that people it's
worth people knowing that's a rather
straightforward concept people people
you know have learned in kind of mathy
ways right now but there there are lots
of things like that about how do you
kind of have these ways to sort of
organize and formalize the world and
that's and these things sometimes they
live in math sometimes they live in in I
don't know what they you know I don't
know what you know learning about color
space I have no idea what I mean you
know there's obviously a field of
there's uh it could be vision science or
no color space you know color space
that's that would be Optics so like
they're not really it's not Optics
Optics is about you know lenses and
chromatic aberration of lenses and
things like that because it's more like
design and art is that no I mean it's
it's like you know RGB space XYZ space
you know Hue saturation brightness space
all these kinds of things these are
different ways to describe colors right
but doesn't the application Define what
that like because obviously artists and
designers use the colors to explore sure
no I mean it's just an example of kind
of how do you you know the typical
person how do you how do you describe
what a color is or there are these
numbers that describe what a color is
well it's worth you know if you're an
eight-year-old you won't necessarily
know you know it's not something we're
born with to know that you know colors
can be described by three numbers
um that's something that you have to you
know it's a thing to learn about the
world so to speak
um and I think that you know that whole
Corpus of things that are learning about
the formalization of the world or the
computationalization of the world that's
something that should be part of kind of
standard education and you know there
isn't a you know there isn't a course
the curriculum for that and by the way
whatever might have been in it just got
changed because of llms and so on
significantly and yeah I would some
watching closely with interest seeing
how universities adapt well you know so
so one of my projects for hopefully this
year I don't know is to try and write
sort of a a reasonable textbook so to
speak of whatever this thing CX whatever
it is you know what should you know you
know what should you know about like
what a bug is what is the intuition
about bugs what's intuition about you
know software testing what is it what is
it you know these are things which are
you know they're not I mean those are
things which have gotten taught in in
computer science as part of the trade of
programming but but kind of the the
conceptual points about what these
things are you know it surprised me just
at a very practical level you know I
wrote this little explainer thing about
Chachi PT and I thought well you know
I'm writing this partly because I wanted
to make sure I understood it myself and
and so on and it's been you know it's
been really popular and um uh
surprisingly so and I then I realized
well actually you know I was sort of
assuming I didn't really think about it
actually I just thought this is
something I can write and I realized
actually it's a level of description
that is kind of you know what has to be
it's not the engineering level
description it's not the kind of just
the qualitative kind of description it's
some kind of sort of expository
mechanistic description of what's going
on together with kind of the bigger
picture of the philosophy of things and
so on and I realized actually this is a
pretty good thing for me to write I you
know I kind of know those things and I
kind of realized it's not a collection
of things that you know it's it's I've
sort of been I was sort of a little
shocked that it's as much of an outlier
in terms of explaining what's going on
as it's turned out to be and that makes
me feel more of an obligation to kind of
write the kind of uh you know what is
you know what is this thing that you
should learn about about the
computationalization the formalization
of the world
um because well I've spent much of my
life working on the kind of tooling and
mechanics of that and the science you
get from it so I guess this is my my
kind of obligation to try to do this but
I think so if you ask what's going to
happen to like the computer science
departments and so on there's there's
some interesting models so for example
let's take math you know math is the
thing that's important for for all sorts
of fields you know engineering you know
even you know chemistry psychology
whatever else
and I think different universities have
kind of evolved that differently I mean
some say all the math is taught in the
math department
um and some say well we're going to have
a you know a math for chemists or
something that is taught in the
chemistry Department
um and you know I think that this this
question of whether there is a
centralization of the teaching of sort
of CX is an interesting question and I
think you know the way it evolved with
math
you know people understood that math was
sort of a separately teachable thing and
um I was kind of a you know a a an
independent element as opposed to just
being absorbed into now so if you take
the example of of writing English or
something like this
the first point is that that you know at
the college level at least fancy
colleges there's a certain amount of
English writing that that people do but
mostly it's kind of assumed that they
pretty much know how to write you know
that's something they learned at a at an
earlier stage in education maybe rightly
or wrongly believing that but that's
different issue
um the uh uh well I think it it reminds
me of my kind of as I've tried to help
people do technical writing and things
I'm I'm always reminded of my zeroth law
of technical writing which is if you
don't understand what you're writing
about your readers do not stand a chance
yeah and so it's it's um uh I think the
um
the thing that uh has some uh you know
in when it comes to like writing for
example
um you know people in different fields
are expected to write English essays and
they're not you know mostly the you know
the history department or the
engineering department they don't have
their own you know let's you know it's
it's not like there's a I mean it's a
thing which sort of people are assumed
to have a knowledge of how to write that
they can use in all these different
fields and the question is you know some
level of knowledge of math is kind of
assumed by the time you get to the
college level but plenty is not and
that's sort of still centrally taught
the question is sort of how tall is the
Tower of kind of CX that you need before
you can just go use it in all these
different fields and you know there will
be experts who want to learn the full
elaborate Tower and that will be kind of
the the cscx whatever department but
there'll also be everybody else who just
needs to know a certain amount of that
to be able to go and do their art
history classes and so on
yes it's just a single class that
everybody's required to take I don't
know I don't know how big it is yet I
hope to kind of Define this curriculum
and I'll figure out whether it's um my
guess is that
I I don't know I don't really understand
universities and professoring that well
but my my rough guests would be a year
long a year of college class will be
enough to get to the point where most
people have a a reasonably broad
knowledge of you know what we sort of
literate in this kind of uh uh
computational way of thinking about
things yeah basic literacy right
I'm still stuck perhaps because I'm
hungry in the uh in the rating of human
preferences for candy so I have to ask
what's the best candy I like this ELO
rating for candy somebody should come up
because you're somebody who says you
like chocolate what's what do you think
is the best I'll probably put Milk Duds
up there I don't know if you know do you
have a preference for chocolate or candy
oh I have lots of preferences I've I've
uh I'm one of my all-time favorites is
my whole life is these things these
flake things Cadbury flakes which are
not much sold in the US and I've always
thought that was a sign of a of a a lack
of respect for the American Consumer
because they're these sort of aerated
chocolate that's made in a in a whole
sort of uh it's kind of a sheet of
chocolate that's kind of folded up and
when you eat it flakes fall all over the
place ah so it requires a kind of
Elegance it requires you to have an
Elegance well I know what I usually do
is I eat tomatoes you know a piece of
paper or something else and clean it up
after no I actually eat the I eat the
flakes I said that because you know it
turns out the way food tastes depends a
lot on its physical structure and you
know it really you know I've noticed
when I eat piece of chocolate I usually
have some little piece of chocolate and
I I always break off little pieces
partly because then I eat it less fast
yeah but also because it actually tastes
different
um you know the the small pieces you
know have a different you have a
different experience than if you have
the big slab of chocolate for many
reasons yes slower
more intimate
because it's I think it's also just pure
physicality or detection yes right it's
fascinating now I dig back my milk dust
because that's such a basic answer okay
do you think Consciousness is
fundamentally uh computational
so when you think about CX what can we
turn into computation
and you're thinking about llms
do you think
the uh the display of Consciousness and
the experience of cautiousness the hard
problem is is fundamentally uh
that computation yeah what it feels like
inside so to speak is
you know I I did a little exercise
eventually I'll I'll post it of uh you
know what it's like to be a computer
yeah right it's kind of like well you
get all this sensory input you have a
kind of the way I see it is from the
time you boot a computer to the time the
computer crashes is like a human life
you you're building up a certain amount
of State in memory you remember certain
things about your quote's life
eventually it's kind of like the the uh
you know the next generation of humans
is is born from the same genetic
material so to speak with a little bit
left over left on the disc so to speak
um and then you know the the new fresh
generation starts up and eventually all
kinds of crud builds up in the in the
memory of the computer and eventually
the thing crashes or whatever or maybe
it has some trauma because you plugged
in some weird thing to some Port of the
computer and that made it crash and that
um uh you know that that's kind of but
but you have this this picture of you
know from from startup to to to shut
down you know what is the life of a
computer so to speak and what does it
feel like to be that computer and what
inner thoughts does it have and how do
you describe it and it's kind of kind of
interesting as you start writing about
this to realize it's awfully like what
you'd say about yourself that is it's
awfully like even an ordinary computer
forget all the AI stuff and so on you
know it's kind of it has a memory of the
past it has certain sensory experiences
it can communicate with other computers
but it has to package up how it's
communicating in some kind of language
like form so it can you know send so it
can kind of map what's in its memory to
what's in the memory of some other
computer it's it's a surprisingly
similar thing you know I hadn't
experience just a week or two ago I I
had I'm a collector of all possible data
about myself and other things and so I
you know I collect all sorts of weird
medical data and so on and one thing I
hadn't collected was I'd never had a
whole body MRI scan so I went and got
one of these yes okay so I get that get
all the data back right I'm looking at
this thing I never looked at the kind of
insides of my brain so to speak
um in in physical form and it's really I
mean it's kind of psychologically
shocking in a sense that you know here's
this thing and you can see it has all
these folds and all these you know the
structure and it's like that's where
this experience that I'm having of you
know existing and so on yeah that's
where it is and you know it feels very
you know you look at that and you're
thinking how can this possibly be all
this experience that I'm having and
you're realizing well I can look at a
computer as well and it's it's kind of
this it it I think this idea that you
are having an experience that is somehow
um
you know transcends the mere sort of
physicality of that experience I I I I
you know it's something that's hard to
come to terms with but I think you know
and I I don't think I'm necessarily you
know my my personal experience you know
I look at the you know the MRI of the
brain and then I you know know about all
kinds of things about neuroscience and
all that kind of stuff and I still feel
the way I feel so to speak and it it
sort of seems disconnected but yet as I
try and rationalize it I can't really
say that there's something kind of
different about how I intrinsically feel
from the thing that I can plainly see in
the sort of physicality of what's going
on so do you think the computer a large
language model will experience that
Transcendence
how does it make you feel like I I tend
to believe it will I think an ordinary
computer is already there I think an
ordinary computer is already you know
kind of it's it's now a large language
model may experience it in a way that is
much better aligned with us humans that
is it's much more you know if you could
have the discussion with the computer
it's intelligent so to speak is not
particularly well aligned with ours but
the large language model is you know
it's built to be aligned with our way of
thinking about things you'll be able to
explain that it's uh afraid of being
shut off and deleted it'd be able to say
that it's sad of the way you've been
speaking to it over the past two days
but you know that's a weird thing
because when it says it's afraid of
something right we know that it got that
idea from the fact that it read on the
internet yeah what did you get it Steven
where did you get it when you say you're
afraid you aren't quite that's the
question yeah right I mean it's it's
parents your friends right or or my
biology I mean in other words there's a
certain amount that is you know the
endocrine system kicking in and and you
know the the um uh these kinds of
emotional overlay type things that
happen to be that are actually much more
physical even they're much more sort of
straightforwardly chemical than the the
then kind of all of the higher level
thinking yeah but your biology didn't
tell you to say I'm afraid just at the
right time when people that love you are
listening and so you know you're
manipulating them by saying so that's
not your biology that's no that's a well
but the you know it's a large language
model and that biological neural network
of yours yes but I mean the intrinsic
thing of you know something sort of
shocking is just happening and you have
some sort of reaction which is you know
some neurotransmitter gets secreted and
it's
um uh you know that that is the
beginning of some you know that is
that's one of the pieces of input that
then draw lives it's kind of like the uh
like a prompt for for the large language
model I mean just like when we dream for
example you know no doubt there are all
these sort of random inputs they're kind
of these random prompts and that's
percolating through in kind of the way
that a large language model does of kind
of putting together things that seem
meaningful
I I mean are you uh are you worried
about this world where you you teach a
lot on the internet and there's people
asking questions and comments and so on
uh you have people that work remotely
um are you worried about this world when
um large language models
create human-like Bots
that are
leaving the comments asking the
questions I might even become fake
employees yeah
I mean or or or uh worse or better at
yet friends friends of yours right look
I mean one point is my mode of life has
been I build tools and then I use the
tools yeah and in a sense kind of you
know I'm I'm building this Tower of
automation yes which you know and in a
sense you know when you make a company
or something you are making sort of
automation but it has some humans in it
but also as much as possible it has it
has uh you know computers in it and so I
think it's sort of an extension of that
now now if I really didn't know that um
you know it's a it's a it's a funny
question it's a it's a funny issue when
you know if we think about sort of
what's going to happen to the future of
kind of jobs people do and so on and
there are places where kind of having a
human in the loop there are different
reasons to have a human in a loop for
example you might want a human in the
loop because you want somebody to you
want another human to be invested in the
outcome you know you want a human flying
the plane who's going to die if the
plane crashes along with you so to speak
and that gives you sort of confidence
that the right thing is going to happen
or you might want you know right now you
might want a human in the loop in some
kind of sort of human encouragement
persuasion type profession whether that
will continue I'm not sure for those
types of professions because it may be
that the the greater efficiency of uh
you know of being able to have sort of
just the right information delivered at
just the right time will overcome the
kind of the the kind of oh yes I want a
human there yeah imagine like a
therapist or even higher stake like a
suicide hotline operated by a large
language model yeah who boy is a pretty
high stake situation right but I mean
but you know it might in fact do the
right thing yeah because it might be the
case that that um you know and that's
really partly a question of sort of how
complicated is the human you know one of
the things that's that's always
surprising in some sense is that you
know sometimes human psychology is not
that complicated in some sense
you wrote the blog post the 50-year
quest my personal Journey good title my
personal Journey with a second law of
Thermodynamics so
what is this law and what have you
understood about it in the 50-year
journey you had with it right so second
vote of thermodynamics sometimes called
law of entropy increase is this
principle of physics that says
well my version of it would be things
tend to get more random over time a
version of it that uh there are many
different sort of formulations of it
that are things like heat doesn't
spontaneously go from a hotter body to a
colder one when you have uh mechanical
work kind of gets dissipated into heat
you have friction and and uh kind of
when you systematically move things
eventually there'll be they'll be sort
of that the energy of moving things gets
kind of ground down into heat so people
first sort of paid attention to this
back in the 1820s when steam engines
were a big thing and the big question
was how efficient could a steam engine
be and there's this chap called Sadi
Kano who was a a French engineer
actually his father was a a sort of
elaborate uh mathematical engineer in
France
um but he figured out these this kind of
rules for how uh kind of the the
efficiency of of the possible efficiency
of something like a steam engine and in
sort of a side part of what he did was
this idea that mechanical energy tends
to get dissipated as heat that you that
you end up going from sort of systematic
mechanical motion to this kind of random
thing well at that time nobody knew what
heat was at that time people thought
that heat was a fluid like they called
it caloric and it was a fluid that kind
of kind of was absorbed into substances
and when when heat when one hot thing
would transfer heat to a colder thing
that this fluid would flow from the hot
thing to the colder thing but anyway
then by the by the 1860s people had uh
kind of come up with this idea that
systematic energy tends to degrade into
kind of random heat that would uh that
that could then not be easily turned
back into systematic mechanical energy
um and then that that quickly became
sort of a global principle about how
things work question is why does it
happen that way so you know let's say
you have a bunch of molecules in a box
and they're arranged these molecules
arranged in a very nice sort of uh
flotiller of molecules in one corner of
the box and then what you typically
observe is that after a while these
molecules were kind of randomly arranged
in in the Box question is why does that
happen and people for a long long time
tried to figure out is there from the
laws of mechanics that just determine
how these molecules that say these
molecules like hard spheres bouncing off
each other from the laws of mechanics
that describe those molecules can we
explain why it tends to be the case that
we see things that are orderly sort of
degrade into disorder we tend to see
things that uh you know you you uh you
scramble an egg you um that you know you
take something's quite ordered and you
you disorder it so to speak that's the
thing that sort of happens quite
regularly or you you put some ink into
water and it will eventually spread out
and and fill up you know fill up the
water
um but you don't see those little
particles of ink in the water all
spontaneously kind of arrange themselves
into a Big Blob and then you know jump
out of the water or something
um and so the question is why do things
happen in this kind of irreversible way
where you go from order to disorder why
does it happen that way and so
throughout in the later part of the
1800s a lot of work was done on trying
to figure out can one derive this
principle this second law of
Thermodynamics this law about the the
Dynamics of heat so to speak can one
derive this from uh from some
fundamental principles of mechanics you
know in the laws of thermodynamics the
first law is basically the law of energy
energy conservation that the total
energy associated with heat plus the
total energy associated the mechanical
kinds of things plus other kinds of
energy that that total is constant and
that became a pretty well understood
principle but the the second law of
Thermodynamics was always mysterious
like why does it work this way can it be
derived from underlying mechanical laws
and so when I was uh well 12 years sold
actually I had gotten interested well
I've been interested in in space and
things like that because I thought that
was kind of the the future and um
interesting sort of technology and so on
and for a while kind of uh you know
every deep space probe was sort of a
personal friend type thing and I knew
all all kinds of characteristics of it
and uh uh was kind of writing up all
these all these things when I was well I
don't know eight nine ten years old and
so on and then I I got interested from
being interested in kind of spacecraft I
got interested so like how do they work
what all the instruments on them and so
on and that got me interested in physics
which was just as well because if I'd
stayed interested in space in the you
know mid to late 1960s I would have had
a long wait before you know space really
blossomed as a as an area but uh editing
is everything right I got interested in
physics and uh then well the actual sort
of detailed story is when I when I kind
of graduated from elementary school at
age 12. that's the time when in England
where you've finished Elementary School
um I sort of my my gift sort of I
suppose more or less for myself was I
got um this collection of um
physics books which were some college
Physics course of college physics books
and volume Five about statistical
physics it has this picture on the cover
that shows a bunch of kind of idealized
molecules sitting in one side of a box
and then it has a series of frames
showing how these molecules sort of
spread out in the box and I thought
that's pretty interesting you know what
what causes that and you know I read the
book and and the book the book actually
one of the things that was really
significant to me about that was the
book kind of claimed although I didn't
really understand what it said in detail
it kind of claimed that this sort of
principle of physics was derivable
somehow and you know other things I'd
learned about physics it was all like
it's a fact that energy is conserved
it's a fact that relativity works or
something not it's something you can
derive from some fundamental sort of it
has to be that way as a matter of kind
of of mathematics or logic or something
so it was sort of interesting to me that
there was a thing about physics that was
kind of inevitably true and derivable so
to speak and so I think that um so then
I was like this picture on this book and
I was trying to understand it and so
that was actually the first serious
program that I wrote for a computer was
probably 1973
um written for this computer the size of
a desk program with paper tape and so on
and I tried to reproduce this picture on
the book and I didn't succeed what was
the failure mode there like what do you
mean he didn't succeed so it's a bunch
of looked like it didn't look like okay
so what happened is
okay many years later I learned how the
picture on the book was actually made
and that it was actually kind of a fake
but I didn't know that at that time
um but uh and that picture was actually
a very high-tech thing when it was made
in the beginning of the 1960s was made
on the largest supercomputer that
existed at the time and uh even so it
couldn't quite simulate the thing that
it was supposed to be simulating but
anyway I didn't know that until many
many years later so at the time it was
like you have these balls bouncing
around in this box but I was using this
computer with eight kilowatts of memory
there were 18 bit words of memory words
okay so it was um whatever 24 kilobytes
of memory
um and it had you know it had these
instructions I probably still remember
all of its machine instructions
um and it didn't really like dealing
with floating Point numbers or anything
like that and so I had to simplify this
this model of of you know particles
bouncing around a box and so I thought
well I'll put them on a grid and I'll
make you know make the things just sort
of move one square at a time and so on
and so I did the simulation and the
result was it didn't look anything like
the actual pictures on the book now many
years later in fact very recently I
realized that the thing I'd simulated
was actually an example of a whole sort
of computational irreducibility story
that I absolutely did not recognize at
the time at the time it just looked like
it did something random and it looks
wrong as opposed to it did something
random and it's super interesting that
it's random
um but I didn't recognize that at the
time and so as it was at the time I kind
of I got interested in particle physics
and I got interested in in other kinds
of physics and but this whole second
order of the Dynamics thing this idea
that sort of orderly things tend to
degrade into disorder continued to be
something I was really interested in and
I was really curious for the whole
universe why doesn't that happen all the
time like we start off at the in the Big
Bang at the beginning of the universe
was this thing that seems like it's this
very disordered collection of of stuff
and then it spontaneously forms itself
into galaxies and creates all of this
complexity and order in the universe and
so I was very curious how that happens
and I but I was always kind of thinking
this is kind of somehow the second order
of thermodynamics is behind it trying to
sort of pull things back into disorder
so to speak and how was order being
created and so actually I was was
interested this is probably now 1980 I
got interested in kind of this you know
Galaxy formation and so on in the
universe I also at that time was
interested in neural networks and I was
interested in kind of how how brains
make complicated things happen and so on
okay what's the connection between the
formation of galaxies and how brains
make complicated things happen because
they're both a matter of how complicated
things come to happen
from simple Origins yeah from some sort
of known Origins I had the sense that
that what I was interested in was kind
of in all these different this sort of
different cases of where complicated
things were arising from rules and you
know I also looked at snowflakes and
things like that
um I was curious and fluid dynamics in
general I was just sort of curious about
how does complexity arise and the the
thing that I didn't you know it took me
a while to kind of realize that there
might be a general phenomenon you know I
sort of assumed oh there's galaxies over
here there's brains over here that
they're very different kinds of things
and so what happened this is probably
1981 or so I decided okay I'm I'm going
to try and make the minimal model of how
these things work yes it was sort of an
interesting experience because I had
built starting in 1979 I built my first
big computer system to think called SMP
symbolic manipulation program it's kind
of Runner of modern morpheme language
with many of the same ideas about
symbolic computation and so on
um but the thing that was very important
to me about that was you know in
building that language I'd basically
tried to figure out what were the sort
of what were the relevant computational
Primitives which have turned out to stay
with me for the last 40 something years
but it was also important because in
building a language was very different
activity from natural science which is
what I've mostly done before because in
Natural Science you start from the
phenomena of the world and you try and
figure out so how can I make sense of
the phenomena of the world
and you know kind of the world presents
you with what it has to offer so to
speak and you have to make sense of it
when you build a a you know computer
language or something you are creating
your own Primitives and then you say
come so what can you make from these
sort of the opposite way around from
what you do in Natural Science but I'd
had the experience of doing that and so
I was kind of like okay what happens if
you sort of make an artificial physics
what happens if you just make up the
rules by which systems operate and then
I was thinking you know for all these
different systems whether it was
galaxies or brains or whatever what's
the absolutely minimal model that kind
of captures the things that are
important about those systems The
computational Primitives of that system
yes and so that's what ended up with the
cellular automata where you just have a
line of black and white cells you just
have a rule that says you know given a
cell and its neighbors what will the
color of the cell be on the next step
and you just run it in a series of steps
and the sort of the ironic thing is that
seller automata are great models for
many kinds of things but galaxies and
brains are two examples where they do
very very badly they're really
irrelevant to those two is there a
connection to the second law of
Thermodynamics and cellular automata oh
yes so the things you the things you've
discovered About Cellular automata yes
okay so when I first started selling
salad automata my first papers about
them were you know the first sentence
was always about the second row of
thermodynamics it was always about how
does order manage to be produced even
though there's a second row of
thermodynamics which tries to pull
things back into disorder and I kind of
my early understanding of that had to do
with these are intrinsically
irreversible processes in cellular
automata that that form uh it's kind of
conform orderly structures even from
random initial conditions but then what
I realized this was uh well actually
it's it's one of these things where it
was a discovery that I should have made
earlier but didn't so you know I had
I've been studying so a little automata
what I did was the sort of most obvious
computer experiment you just try all the
different rules and see what they do
it's kind of like you know you've
invented a computational telescope you
just pointed at the most obvious thing
in the sky and then you just see what's
there and so I did that and I you know I
was making all these pictures of of how
cellular automata work and and I studied
these pictures I started in great detail
There Was You Can number the rules for
cellular automata and one of them is you
know rule 30. so I made a picture of
rule 30 back in 1981 or so and Rule 30
well it's and at the time I was just
like okay it's another one of these
rules I don't really it happens to be
asymmetric Left Right asymmetric and
it's like let me just consider the case
of the symmetric ones just to keep
things simpler
et cetera et cetera et cetera and I just
kind of ignored it yeah and then
sort of in and actually in 1984
strangely enough I ended up having a an
early laser printer which made very high
resolution pictures and I thought I'm
going to print out an interesting you
know I want to make an interesting
picture let me take this rule 30 thing
and just make a high resolution picture
of it I did and it's it has this very
remarkable property that it's rule is
very simple you started off just from
one black cell at the top and it makes
this kind of triangular pattern but if
you look inside this pattern it looks
really random there's you know you look
at the center column of cells and you
know I studied that in great detail and
it's so far as one can tell it's
completely random and it's kind of a
little bit like digits of pi once you
you know you know the rule for
generating the digit Supply but once
you've generated them you know 3.14159
Etc they seem completely random and in
fact I put up this prize back in what
was it 2019 or something for prove
anything about the sequence basically
has anyone been able to do anything on
that uh people have sent me some things
but it's you know I don't know how these
problems are I mean I was kind of
spoiled because I 2007 I put up a prize
for uh determining whether a particular
turing machine that I thought was the
simplest candidate for being Universal
turing machine determine whether it is
or isn't a universal turing machine and
somebody did a really good job of of
winning that prize and proving that it
was a universal turing machine in about
six months and so I you know I didn't
know whether that would be one of these
problems that was out there for hundreds
of years or whether in this particular
case young chap called Alex Smith
um you know nailed it in six months and
so with this little 30 collection I
don't really know whether these are
things that are 100 years away from
being able to to get or whether
somebody's going to come and do
something very clever it's such a means
like for Mars Last Theorem Essentia rule
30 is such a simple formulation it feels
like anyone can look at it understand it
yeah and feel like it's within grasp
to be able to predict something to do to
direct some kind of law
right it allows you to predict something
about this the middle column of rule 30
right but you know this is yeah you
can't yeah right this is the intuitional
surprise of computational irreducibility
and so on that even though the rules are
simple you can't tell what's going to
happen and you can't prove things about
it and I think so so anyway the the the
thing uh I I still started in 1984 or so
I started realizing there's this
phenomenon that you can have very simple
rules they produce apparently random
Behavior okay so that's a little bit
like the second orthodynamics because
it's like you have this simple initial
condition you can you know readily see
that it's very you know you can describe
it very easily and yet it makes this
thing that seems to be random now turns
out
thus some technical detail about the
secular thermodynamics and about the
idea of reversibility when you have a if
you have kind of a a a movie of two you
know billiard balls colliding and you
see them Collide and they bounce off and
you run that movie In Reverse you can't
tell which way was the forward direction
of time and which way was the backward
direction of time when you're just
looking at individual billiard Balls by
the time you've got a whole collection
of them you know a million of them or
something then it turns out to be the
case and this is the the sort of the The
Mystery of the second law that the
orderly thing you start with the orderly
thing and it becomes disordered and
that's the forward Direction in time and
the other way around of it starts to
sorted and becomes ordered you just
don't see that in the world
now in principle if you you know if you
sort of traced the detailed motions of
all those molecules backwards you would
be able to it it will it will the
reverse of time makes you know as you as
you go forwards in time order goes to
disorder as you go backwards in time
order goes to disorder perfectly so yes
right so the the mystery is why is it
the case that one version of the mystery
is why is it the case that you never see
something which happens to be just the
kind of disorder that you would need to
somehow evolve to order why does that
not happen why do you always just see
order goes to disorder not the other way
around
so the thing that I I kind of realized I
started realizing in the 1980s is kind
of like it's a bit like cryptography
it's kind of like you start off from
this this key that's pretty simple and
then you kind of run it and you can get
this you know complicated random mess
and uh the thing that that um
well I sort of started realizing back
then was that the second law is kind of
a a story of computational reducibility
it's a story of you know what seems you
know what what we can describe easily at
the beginning we can only describe with
a lot of computational effort at the end
okay so now we come many many years
later and um uh I was trying to sort of
uh well having done this big project to
understand fundamental physics I
realized that sort of a key aspect of
that is understanding what observers are
like
and then I realized that the second
orthodynamics is the same story as a
bunch of these other cases
um it is a story of a a computationally
bounded Observer trying to observe a
computationally irreducible system so
it's a story of you know underneath the
molecules are bouncing around they're
bouncing around in this completely uh
determined way determined by rules
but the point is that that we as
computationally bounded observers can't
tell that there were these sort of
simple underlying rules to us it just
looks random and when it comes to this
question about can you prepare the
initial state so that
um you know the disordered thing is you
know how exactly the right disorder to
make something orderly a computationally
bounded Observer cannot do that we'd
have to have done all of this sort of
irreducible computation to work out very
precisely what this disordered State
what the exact right disordered state is
so that we would get this ordered thing
produced from it what does it mean to be
computationally bounded Observer
so observing a computational reducible
system so the computationally bounded is
there something formal you can say there
right so it means
okay you can you can talk about Turing
machines you can talk about
computational uh complexity Theory and
uh you know uh polynomial time
computation and things like this there
are a variety of ways to make something
more precise but I think it's more
useful the intuitive version of it is
more useful yeah which is basically just
to say that you know how much
computation are you going to do to try
and work out what's going on and the
answer is you're not allowed to do a lot
of we're not able to do a lot of
computation when we you know we've got
you know in this room there will be a
trillion trillion trillion molecules a
little bit less it's a big room right
and uh you know at every moment you know
that every microsecond or something
these molecules molecules are colliding
and that's a lot of computation that's
getting done and the question is in our
brains we do a lot less computation
every second than the computation done
by all those molecules if there is
computational irreducibility we can't
work out in detail what all those
molecules are going to do what we can do
is only a much smaller amount of
computation and so the the second law of
Thermodynamics is this kind of interplay
between the underlying computational
irreducibility and the fact that we as
preparers of initial States or as
measures of what happens are you know
are not capable of doing that much
computation so to us another big
formulation of the second order of
thermodynamics is this idea of the law
of entropy increase
the characteristic that this universe
the entropy sees to be always increasing
what does that show to you about the
evolution of
yes okay and that's very confused in the
history of thermodynamics because
entropy was first introduced by a guy
called Rudolf clausius and he did it in
terms of heat and temperature okay
subsequently it was reformulated by a
guy called Ludwig boltzmann
um and uh he formulated it in a much
more kind of combinatorial type way
but he always claimed that it was
equivalent to clausius's thing and then
in one particular simple example it is
but that connection between these two
formulations of entropy they've never
been connected I mean it's there there's
really so okay so the more general
definition of entropy due to boltzmann
is is the following thing so you say I
have a system and has many possible
configurations molecules can be in many
different Arrangements Etc et cetera Etc
if we know something about the system
for example we know it's in a box it has
a certain pressure it has a certain
temperature we know these overall facts
about it then we say how many
microscopic configurations of the system
are possible given those overall
constraints
um and the entropy is the logarithm of
that number that's the definition and
that's the kind of the general
definition of entropy that turns out to
be useful now in Boltzmann's time he
thought these molecules can be placed
anywhere you want he didn't think a he
said oh actually we can make it a lot
simpler by having the molecules be
discrete well actually he didn't know
molecules existed right and in those in
his time 1860s and so on uh the idea
that Mata might made of discreet stuff
had been floated ever since ancient
Greek times but it had been a long time
debate about you know is math or
discrete is it continuous at the moment
at that time people mostly thought that
Mata was continuous and it was all
confused with this question about what
heat is and people thought heat was this
fluid and um it was it was a big big
model and um the uh and this but
boltzmann said let's assume there are
discrete molecules let's even assume
they have discrete energy levels let's
say everything is discrete then we can
do sort of combinatorial mathematics and
work out how many configurations of
these things they would be in the box
and we can say we can compute the
centropy quantity but he said said but
of course it's just a fiction that these
things are discreet so he said this is
an interesting piece of History by the
way that that you know that was at that
time people didn't know molecules
existed there were other hints from from
looking at uh kind of chemistry that
there might be discrete atoms and so on
just from the combinatorics of you know
two hydrogens and one oxygen make water
you know two two amounts of hydrogen
plus one amount of oxygen together make
water things like this but it wasn't
known that discrete molecules existed
and in fact the um uh people
you know it wasn't until the beginning
of the the 20th century that Brownian
motion was the final giveaway Brown in
motion is you know you look under a
microscope at these little pieces from
pollen grains you see they're being
discreetly kicked and those kicks are
water molecules hitting them and they're
discreet
um and uh in fact it was um it was
really quite interesting history I mean
boltzmann had worked out how things
could be discreet and have basically
invented something like quantum theory
in in the 1860s and uh but he just
thought it wasn't really the way it
worked and then just a piece of physics
history because I think it's kind of
interesting in in 1900 this guy called
Max Planck who'd been a long time
thermodynamics person who was trying to
everybody was trying to prove the second
order of thermodynamics including Max
Planck and Max Planck believed that
radiation like electromagnetic radiation
somehow the interaction of that with
Mata was going to prove the second law
of thermodynamics but he had these
experiments that people had done on
black body radiation and there were
these curves and you couldn't fit the
curve based on his idea for how
radiation interacted with Mata those
curves you couldn't figure out how to
fit those curves except he noticed that
if he just did what boltzmann had done
and assumed that electromagnetic
radiation was discrete he could fit the
curves he said but you know this is just
a you know it just happens to work this
way then Einstein came along and said
well by the way you know uh the
electromagnetic field might actually be
discrete it might be made of photons and
then that explains how this all works
and that was you know in 1905 that was
that was how
um kind of that was how Quant that piece
of quantum mechanics got started kind of
interesting interesting piece of History
I didn't know until I was researching
this recently in 1904 and 1903 Einstein
wrote three different papers and uh so
you know just sort of a well-known
physics history in 1905 Einstein wrote
these three papers one introduced
relativity Theory one explained Brownian
motion and one introduced basically
photons so kind of you know kind of a a
big deal year for physics and for
Einstein but in the years before that
he'd written several papers and what
were they about they were about the
second war of thermodynamics and they
were an attempt to prove the second
order of thermodynamics and their
nonsense
and so I I had no idea that he'd done
this interesting oh neither and in fact
what he did those three papers in 1905
well not so much the relativity paper
the one on Brownian motion the one on
photons both of these were about the
story of sort of making the world
discreet
um and then he got those like that idea
from boltzmann yeah um but boltzmann
didn't think you know boltzmann kind of
died believing you know he said but he
has a quote actually you know uh you
know in the end things are going to turn
out to be discreet and I'm going to
write down what I have to say about this
because uh uh you know eventually the
stuff will be rediscovered and I want to
leave you know what I can about how
things are going to be discreet but you
know
um I think he has some quotes about how
you know one person can't stand against
the tide of history in um uh in saying
that you know matter is discrete so so
he stopped by his guns it doesn't matter
is discrete yes he did and and the you
know what's interesting about this is uh
at the time everybody including Einstein
kind of assumed that space was probably
going to end up being discreet too but
that didn't work out technically because
it wasn't consistent with relativity
Theory it didn't seem to be and so then
in the history of physics even though
people had determined that Mata was
discrete electoral magnetic field was
discrete space was a holdout of not
being discreet and in fact Einstein 1916
has this nice letter he wrote where he
says in the end it will turn out space
is discrete but we don't have the
mathematical tools necessary to figure
out how that works yet
and so you know I think it's kind of
cool that 100 years later we do yes for
you you're pretty pretty sure that at
every layer of reality it's discreet
right and that space is discrete and
that uh the I mean and in fact one of
the things I realized recently is this
kind of theory of heat that um uh
that the um you know that heat is really
this continuous fluid
um it's it's kind of like uh the the you
know the caloric theory of heat which
turns out to be completely wrong because
actually heat is the motion of a
discrete molecules unless you know there
are discrete molecules it's hard to
understand what heat could possibly be
well you know I think space is is
discrete and the question is kind of
what's the analog of the mistake that
was made with caloric
in the case of space and so I'm my my
current guess is that dark matter is as
I've my little sort of aphorism of the
of the last few months has been you know
dark matter is the caloric of our time
that is it will turn out the dark matter
is a feature of space and it is not a
bunch of particles you know at the time
when when people were talking about heat
they knew about fluids and they said
well heat must just be another kind of
fluid because that's what they knew
about yes but now people know about
particles and so they say well what's
dark matter it's not it's not it just
must be particles so what could dark
matter be as a feature of space Oh I
don't know yet all right um I mean I
think the the thing I'm really one of
the things I'm hoping to be able to do
is to find the analog of brown in Motion
in space so in other words Brown in
motion was was seeing down to the level
of an effect from Individual molecules
and so in the case of space you know
most of the things the things we see
about space so far just everything seems
continuous Brownian motion have been
discovered in the 1830s and it was only
identified what it was what it was the
the result of bias uh smallochowski and
Einstein at the beginning of the 20th
century and you know Dark Matter was was
discovered that phenomenon was
discovered 100 years ago
um you know the rotation curves of
galaxies don't follow the Luminous
matter that was discovered 100 years ago
and I think you know that I I wouldn't
be surprised if there isn't an effect
that we already know about that is kind
of the analog of brown in motion that
reveals the discreetness of space and in
fact we're beginning to have some
guesses we have some some evidence that
black hole mergers work differently when
there's discrete space and there may be
things that you can see in gravitational
wave signatures and things associated
with the discreetness of space but this
is kind of uh for me it's kind of it's
kind of interesting to see this sort of
recapitulation of the history of physics
where people you know vehemently say you
know matter is continuous
electromagnetic field is continuous and
turns out it isn't true and then they
say space is continuous But but so you
know entropy is the number of states of
the system consistent with some
constraint yes and the the thing is that
if you have if you know in great detail
the position of every molecule in the
gas
the entropy is is always zero because
there's only one possible State the the
configuration of molecules and the gas
the molecules bounce around they have a
certain rule for bouncing around there's
just one state of the gas evolves to one
state of the gas and so on but it's only
if you don't know in detail where all
the molecules are that you can say well
the entropy increases because the things
we do know about the molecules there are
more possible microscopic states of the
system consistent with what we do know
about where the molecules are and so the
question of whether
um so people this sort of paradox in a
sense of oh if we knew where all the
molecules were the entropy wouldn't
increase there was this idea introduced
by by Gibbs in the early 20th century
well actually the very beginning of the
20th century as a physics Professor an
American physics Professor was sort of
the first distinguished American physics
Professor um at Yale
um and he he introduced reduce this idea
of course graining this idea that well
you know these molecules have a detail
where they're bouncing around but we can
only observe a coarse grained version of
that but the confusion has been nobody
knew what a valid course screening would
be so nobody knew that whether you could
have the score screening that very
carefully was sculpted in just such a
way that it would notice that the
particular configurations that you could
get from the simple initial condition
you know they fit into this coarse
graining and the course graining very
carefully observes that why can't you do
that kind of very detailed precise
course screening the answer is because
if you are a computationally bounded
Observer and the underlying Dynamics is
computationally irreducible that's
that's what defines possible core
screenings is what a computationally
bounded Observer can do and it's the
it's the fact that a computationally
bounded Observer uh is is forced to look
only at this kind of coarse grained
version of what the system is doing
that's why and and because the what's
what's going on underneath is it's kind
of filling out this this the the
different possible you're ending up with
something where
the sort of underlying computational
irreducibility is uh
your if if all you can see is what the
coarse grained result is with copy with
a sort of computationally bounded
observation then inevitably there are
many possible underlying configurations
that are consistent with that just to
clarify basically any Observer that
exists inside the universe
is going to be computationally bounded
no any Observer like us I don't know I
can't say like us what do you mean what
do you mean like us well
humans with finite Minds you're
including the tools of science yeah yeah
I mean and and as we you know we have
more precise and by the way there are
little sort of microscopic violations of
the second order of thermodynamics that
you can start to have when you have more
precise measurements of where precisely
molecules are but for uh for a large
scale when you have enough molecules we
don't have you know we're not tracing
all those molecules and we just don't
have the computational resources to do
that and it wouldn't be uh you know I
think the the to imagine what an
observer who is not computationally
bounded would be like
it's an interesting thing because okay
so what does computational boundedness
mean among other things it means we
conclude that definite things happen we
go we take all this complexity of the
world and we make a decision we're going
to turn left or turn right and that is
kind of reducing all this kind of uh
detail into we're observing it we're
we're sort of crushing it down to this
this one thing yeah and and that if we
didn't do that
uh we wouldn't we wouldn't have all this
sort of symbolic structure that we build
up that lets us think things through
with our finite Minds we'd be instead
you know we'd be just we'd be sort of
one with the universe yeah so content
to not simplify yes if we didn't
simplify then we wouldn't be like us we
would be like the universe like the the
intrinsic universe but not having
experiences like the experiences we have
where we for example conclude that
definite things happen we you know we we
sort of have this this uh uh notion of
being able to make make sort of
narrative statements yeah I wonder if
it's just like you imagined as a thought
experiment what it's like to be a
computer I wonder if it's possible to
try to begin to imagine what it's like
to be an unbounded computational
well okay so here's here's how that I
think plays out vibrations suck yeah so
I mean in this we talk about this
rouliad the space of all possible
computation yes and this idea of you
know being at a certain place in the
rouliad which corresponds to sort of a
certain way of of rep of a certain set
of computations that you're representing
things in terms of okay so as you expand
out in the rouliad as you kind of
Encompass more possible views of the
universe as you Encompass more possible
kinds of computations that you can do
eventually you might say that's a real
win you know we're colonizing the
rouliad we're we're building out more
paradigms about how to think about
things and eventually you might say we
we won all the way we managed to
colonize the whole Riyadh okay here's
the problem with that the problem is
that the notion of existence coherent
existence
requires some kind of specialization by
the time you are the whole rouliad by
the time you cover the whole rouliad in
no useful sense do you coherently exist
so in other words in inches the notion
of existence the notion of what we think
of as as definite existence requires
this kind of specialization requires
this kind of idea that we are we are not
all possible things we are the a
particular set of things and that's kind
of how we uh that that's kind of what
what makes us have a coherent existence
if we were spread throughout the rouliad
we would not there would be no coherence
to the way that we work we would work in
all possible ways and that wouldn't be
kind of a notion of identity we wouldn't
have this notion of kind of uh uh of of
of coherent identity
I am geographically located somewhere
exactly precisely in the rouliad
therefore I am
physical space you're in a certain place
in real space and if if you uh if you
are sufficiently spread out you are no
longer coherent and you no longer have I
mean in in the in our perception of what
it means to exist and to have experience
it doesn't happen now so therefore so to
exist means to be computationally bonded
I think so to exist in the way that we
think of ourselves as existing yes the
very active existence is like operating
in this place that's computationally
reducible so this is just giant mess of
things going on that you can't possibly
predict
but nevertheless because of your
limitations
you have an imperative of like what is
it an imperative or a skill set to
simplify or an ignorance a sufficient
love okay so the thing which is not
obvious is that you are taking a slice
of all this complexity just like we have
all of these molecules bouncing around
in the room but all we notice is you
know the the the the kind of the flow of
the air or the pressure of the air we're
just noticing these particular things
and the the big interesting thing is
that there are rules there are laws that
govern those big things that we we
observe yeah so it's not obvious that's
amazing because it doesn't feel like
it's a slice yeah well right not a slice
well it's like uh it's like an
abstraction yes but I mean the fact that
the gas laws work that we can describe
pressure volume Etc et cetera et cetera
we don't have to go down to the level of
talking about individual molecules that
is a non-trivial fact and and here's the
thing that I sort of exciting thing as
far as I'm concerned the fact that there
are certain aspects of the universe so
you know we think space is made
ultimately these atoms of space and
these typographs and so on and we think
that uh but we nevertheless perceive the
universe at a large scale to be like
continuous space and so on
um we uh in quantum mechanics we think
that there are these many threads of
time these many threads of History yet
we kind of span so so you know in in
quantum mechanics in our models of
physics there are these time is not a
single thread time breaks into many
threads they Branch they merge and but
we we are part of that branching merging
Universe right and so our brains are
also branching and merging and so when
we perceive the universe we are
branching brains perceiving a branching
Universe yeah and so the fact that the
claim that we exp we believe that we are
persistent in time we have this single
thread of experience that's the
statement that somehow we managed to
aggregate together those separate
threads of time that are separated in in
the operation of in the fundamental
operation of the universe so just as in
space we're averaging over some big
region of space and we're looking at
many many of the aggregate effects of
many atoms of space so similarly in what
we call branchial space the space of
these these Quantum branches we are
effectively averaging over many
different branches of possible of
histories of the universe and so in in
thermodynamics we're averaging over many
configurations of you know many many
possible positions of molecules yeah so
what what we see here is so the question
is when you do that averaging for space
What are the aggregate laws of space
when you do that averaging of a
branchial space What are the aggregate
laws a branchial space when you do that
averaging over the molecules and so on
what are the aggregate laws you get and
this is this is the thing that I I think
is just amazingly amazingly neat that
there are aggregate laws at all well yes
but the question is what are those
aggregate laws so the answer is for
space the aggregate laws Einstein's
equations for Gravity for the structure
of space-time for branchial space the
aggregate laws are the laws of quantum
mechanics and for uh the case of of
molecules and things the aggregate laws
are basically the second law of
thermodynamics
and so the um though that's the and the
things that follow from the second world
of thermodynamics and so what that means
is that the three great theories of 20th
century physics which are basically
general relativity the theory of gravity
uh quantum mechanics and statistical
mechanics which is what kind of grows
out of the second row of thermodynamics
all three of the great theories of 20th
century physics are the result of this
interplay between computational
irreducibility and the computational
boundedness of observers and you know
for me this is really neat because it
means that all three of these laws are
derivable so we used to think that for
example Einstein's equations were just
sort of a wheel in feature of our
universe that they could be the universe
might be that way it might not be that
way quantum mechanics is just like well
it just happens to be that way and the
second law people kind of thought well
maybe it is derivable okay what turns
out to be the case is that all three of
the fundamental principles of physics
are derivable but they're not derivable
just from mathematics they require or
just from some kind of logical
computation they require one more thing
they require that the Observer that the
thing that is sampling the way the
universe works is an observer who has
these characteristics of computational
boundedness of belief and persistence
and time and so that that means that it
is the nature of the Observer
you know the rough nature of the
Observer not the details of oh we got
two eyes and we've observed photons of
this frequency and so on uh but the the
the the the kind of the very coarse
features of the Observer
um then imply these very precise facts
about physics and it's it's I think it's
amazing so if we just look at the actual
experience of the Observer that we
experience this reality it seems real to
us
and you're saying because of our bonded
nature it's actually all an illusion
it's a simplification yeah it's a
simplification right what's what's you
don't think a simplification is an
illusion
no I mean it's it's well I don't know I
mean underneath
uh okay that's an interesting question
um what's real and that relates to the
whole question of why does the universe
exist and um you know what is the
difference between reality and a mere
representation of what's going on yes we
experience the representation yes but
the the question of so so one question
is uh you know why is there a thing
which we can experience that way
and the answer is because this rouliad
object which is this entangled limit of
all possible computations there is no
choice about it it has to exist it has
to there has to be such a thing it is in
in the same sense that you know two plus
two if you define what two is and you
plot pluses and so on two plus two has
to equal four similarly this really add
this limit of all possible computations
just has to be a thing you that is once
you have the idea of computation you
inevitably have the rule yeah you're
gonna have to have a rule yeah yeah
right and and what's important about it
there's just one of it it's it's it's
just this unique object and that unique
object necessarily exists and then the
question is what uh and then we
uh once once you know that we are sort
of embedded in that and taking samples
of it that it's sort of inevitable that
there is this thing that we can perceive
that is you know the the our perception
of kind of physical reality
necessarily is that way given that we
are observers with the characteristics
we have
so in other words the fact that the fact
that the Universe exists is it's
actually it's almost like it's you know
to think about it almost theologically
so to speak and I I really it's it's
funny because a lot of the questions
about the existence of the universe and
so on they they transcend what kind of
the science of the last few hundred
years has really been concerned with the
science of the last few hundred years
hasn't thought it could talk about
questions like that yeah
um and uh but I think it's kind of and
so a lot of the kind of arguments of you
know does God exist you know is it
obvious that I think it in some sense in
some representation it's sort of more
more obvious that uh that something sort
of bigger than us exists than that we
exist and we are you know our existence
and as observers the way we are is sort
of a contingent thing about the universe
and it's more inevitable that the whole
the whole universe kind of the whole set
of all possibilities
exists but but this question about you
know is is it real or is it an illusion
you know all we know is our experience
and so the fact that well our experience
is this absolutely microscopic piece of
sample of the rouliad and we're
um and and you know there's this this
point about you know we might sample
more and more of the rouliad we might
learn more and more about we might learn
you know like like different areas of
physics like Quantum Mechanics for
example the fact that it it was
discovered I think is closely related to
the fact that electronic amplifiers were
invented that allowed you to take a
small effect and amplify it up which
hadn't been possible before you know
microscopes have been invented that
magnify things and so on but they're you
know having a very small effect and
being able to magnify it was sort of a
new thing that allowed one to see a
different sort of aspect of the universe
and let one discover this kind of thing
so you know we can expect that in the
rouliad they're an infinite collection
of new things we can discover there's
there's in fact computational energy
ability kind of guarantees that there
will be an infinite collection of kind
of you know pockets of reducibility that
can be discovered
boy would it be fun to take a walk down
the woolly ad and see what kind of stuff
we find there you write about alien
intelligences
yes I mean just these worlds yes well
computation problem with these worlds is
that we can't talk to them yes and and
you know the thing is what I've kind of
spent a lot of time doing is just
studying computational systems seeing
what they do what I now call ruleology
kind of just the study of rules yeah and
what they do you know you can kind of
easily jump somewhere else in the
rouliad and start seeing what do these
rules do yeah and what you says they
they just they do what they do and
there's no human connection so to speak
because you think you know some people
are able to uh
uh communicate with animals do you think
you can become a Whisperer of these
trying that's what I've spent some part
of my life have you have you heard and
well are you at the risk of losing your
mind sort of my favorite science
discovery
is this fact that these very simple
programs can produce very complicated
Behavior yeah and that and that fact is
kind of in a sense a whispering of
something out in the computational
universe that we didn't really know was
there before I mean it's you know I it's
like you know back in the 1980s I was
doing a bunch of work with some very
very good mathematicians and they were
like trying to pick away you know can we
figure out what's going on in these
computational systems and they they
basically said look the math we have
just doesn't get anywhere with this
we're stuck there's nothing to say we
have nothing to say and you know in a
sense perhaps my main achievement at
that time was to realize that the very
fact that the the good mathematicians
had nothing to say was itself a very
interesting thing that was kind of a
sort of in some sense a whispering of a
different part of the rouliad that one
hadn't you know one wasn't was not
accessible from what we knew in MA and
so on
does it make you sad that you're
exploring some these gigantic ideas and
it feels like we're on the verge of
breaking through to some very
interesting discoveries
and yet you're just a finite being
that's going to die way too soon and
that scan of your brain or your full
body kind of shows that you're yeah it's
just a bunch of meat it's just a bunch
of meat
um yeah does that make you make you a
little sad kind of shy I mean I kind of
like to see how all this stuff works out
but I think the thing to realize you
know it's an interesting sort of thought
experiment you know you you say okay you
know let's assume we can get cryonics to
work and one day it will that will be
one of these things that's kind of like
chat gbt one day somebody will figure
out you know how to get water from zero
degrees Centigrade down to you know
minus 44 or something without it
expanding and you know cryonics will be
solved and you'll be able to like just
uh you know put a pause in so to speak
and you know uh kind of reappear 100
years later or something and the thing
though that I've kind of increasingly
realized is that in a sense this this
whole question of kind of the the um
sort of one is embedded in a certain
moment in in time and you know kind of
the things we care about now the things
I care about now for example had I lived
you know 500 years ago many of the
things I care about now it's like that's
totally bizarre I mean nobody would care
about that it's not even the thing one
thinks about
in the future the things that most
people will think about you know one
will be a strange relic of thinking
about you know the kind of you know it
might be or might have been a theologian
thinking about you know how many angels
fit on the head of a pin or something
and that might have been the you know
the big intellectual thing so I think
it's a it's a um uh but yeah it's a it's
a you know it's one of these things
where particularly you know I've had the
I don't know good or bad fortune I'm not
sure I think it's it's a mixed thing
that I've you know I've invented a bunch
of things which I kind of can I think
see well enough what's going to happen
that you know in 50 years 100 years
whatever assuming the world doesn't
exterminate itself so to speak you know
these are things that will be sort of
centrally important to what's going on
and it's kind of both it's both a good
thing and a bad thing in terms of the
passage of one's life I mean it's kind
of like if everything I'd figured out
was like okay I figured it out when I
was 25 years old and everybody says it's
great and we're done and it's like okay
but I'm gonna live another how many
years and that's kind of it's all
downhill from there in a sense it's it's
better in some sense to to be able to
you know there's there's it sort of
keeps things interesting that you know
why I can see you know a lot of these
things I mean it's kind of I I didn't
expect you know chat GPT I didn't expect
the kind of uh the sort of opening up of
this idea of computation and
computational language that's been made
possible by this I didn't expect that
this is this is ahead of schedule so to
speak
um you know even though the sort of the
the big kind of flowering of that stuff
I'd sort of been assuming was another 50
years away so if it turns out it's a lot
less time that's pretty cool because you
know I'll hopefully get to see it so to
speak rather than
well I I think I speak for a very very
large number of people in saying that I
hope you stick around for a long time to
come you've had so many interesting
ideas you've created so many interesting
systems over the years and I can't see
now that GPT and language models broke
up in the world even more I can't wait
to see uh you at the Forefront of this
development what you what you do
and uh yeah I've been a fan of yours
like I've told you many many times since
the very beginning I'm deeply grateful
that you wrote a new kind of science
that you explored this mystery of
cellular automata and inspired this one
little kid in me uh to to pursue
artificial intelligence and all this
beautiful world so soon thank you so
much it's a huge honor to talk to you to
to just be able to pick your mind and to
explore all these ideas with you and
please keep going and I can't wait to
see what you come up with next and thank
you for talking today we hit it thanks
we went past midnight we only did uh
four and a half hours I mean we could
probably go for four more but we'll save
that till next time to uh this is round
number four well I'm sure talk many more
times thank you so much my pleasure
thanks for listening to this
conversation with Stephen Wolfram to
support this podcast please check out
our sponsors in the description and now
let me leave you some words from George
Cantor
the essence of mathematics lies in its
freedom
thank you for listening and hope to see
you next time