Kind: captions
Language: en
what possible ideas do you have for the
how human species ends sure so I think
the most obvious way to me is
wireheading
we end up amusing ourselves to death
we end up all staring at that infinite
Tick Tock and forgetting to eat
maybe maybe it's even more benign than
this maybe we all just stop reproducing
now to be fair It's probably hard to get
all of humanity yeah the interesting
thing about humanity is the diversity oh
yeah organisms in general there's a lot
of weirdos out there well two of them
are sitting here I mean diversity in
humanity is we do respect
I wish I was more weird
the following is a conversation with
George hotz his third time on this
podcast he's the founder of comma AI
that seeks to solve autonomous driving
and is the founder of a new company
called tiny Corp that created tiny grad
a neural network framework that is
extremely simple with the goal of making
it run on any device by any human easily
and officially as you know George also
did a large number of fun and amazing
things from hacking the iPhone to
recently joining Twitter for a bit as a
in turn in quotes making the case for
refactoring the Twitter code base in
general he's a fascinating engineer and
human being and one of my favorite
people to talk to
this is the Lex Friedman podcast to
support it please check out our sponsors
in the description and now dear friends
here's George hotz
you mentioned something in a stream
about the philosophical nature of time
so uh let's start with the wild question
do you think time is an illusion
you know
I sell phone calls uh to comma for a
thousand dollars uh and some guy called
me and uh like you know it's a thousand
dollars you can talk to me for for half
an hour and he's like uh yeah okay so
like time doesn't exist and I really
wanted to share this with you
I'm like oh what do you mean time
doesn't exist right like I think time is
a useful model whether it exists or not
right like does quantum physics exist
well it doesn't matter it's about
whether it's a useful model to describe
reality is time maybe compressive
do you think there is an objective
reality or is everything just useful
models
all right Underneath It All Is there an
actual thing that we're constructing
models for
I don't know
I was hoping you would know I don't
think it matters
I mean this kind of connects to the
models of constructive reality with
machine learning
right sure like is it just nice to have
useful approximations of the world such
that we can do something with it so
there are things that are real column
graph complexity is real
yeah yeah the compressive thing math is
real yeah
it should be a T-shirt and I think hard
things are actually hard I don't think P
equals NP ooh strong words well I think
that's the majority I do think factoring
is in P but I don't think you're the
person that falls the majority in all
walks of life so it's good for that one
I do yeah in theoretical computer
science you you're one of the cheap
all right but do you uh times a useful
model sure
what were you talking about on the
stream with time are you made of time if
I remembered half the things I said on
stream
someday someone's going to make a model
of all of it I'm just going to come back
to haunt me someday soon yeah probably
would that be exciting to you or sad
that there's a George hotz model
I mean the question is when the George
Hots model is better than George hotz
like I am declining and the model is
growing what is the metric by which you
measure better or worse in that if
you're competing with yourself
maybe you can just play a game where you
have the George hot answer and the
George hotz model answer and ask which
people prefer
people close to you or strangers
either one it will hurt more when it's
people close to me but
both will be overtaken by the
georgetown's model
it'd be quite painful right loved ones
family members would rather have the
model over for Thanksgiving than you
yeah
or like significant others
would rather sext
with the uh with the large language
model version of you
especially when it's fine-tuned to their
preferences
yeah
well that's what we're doing in a
relationship right we're just
fine-tuning ourselves but we're
inefficient with it because we're
selfish ingredients and so on our
language models can fine tune more
efficiently more selflessly there's a
Star Trek Voyager episode where uh you
know Catherine Janeway
lost in the Delta quadrant makes herself
a lover on the Holodeck and
[Music]
um
the lover falls asleep on her arm and he
snores a little bit and you know Janeway
edits the program to remove that
and then of course the realization is
wait this person's terrible it is
actually all there uh
nuances and quirks and slight annoyances
that that make this relationship
worthwhile
but I don't think we're going to realize
that until it's too late
well I think a large language model
could incorporate the the flaws and the
quarks and all that kind of stuff just
the perfect amount of quirks and floor
flaws to make you charming without
crossing the line yeah yeah and there's
probably a good like
approximation of the like the percent of
time the language model should be uh
cranky or uh
an asshole or jealous or all this kind
of stuff and of course it can and it
will but all that difficulty at that
point is artificial there's no more real
difficulty
okay what's the difference in real and
artificial artificial difficulty is
difficulty that's like constructed or
could be turned off with a knob real
difficulty is like you're in the woods
and you've got to survive
so if something cannot be turned off
with a knob it's real
yeah I think so or I mean you can't get
out of this by Smashing the knob with a
hammer
I mean maybe you kind of can you know I
into the wild when uh you know uh
Alexander Supertramp he wants to explore
something that's never been explored
before but it's the 90s everything's
been explored so he's like well I'm just
not going to bring a map
yeah I mean
no you're you're not exploring you
should have brought a map dude you died
there was a bridge a mile from where you
were camping
how does it connect to the metaphor of
the knob
by not bringing the map you didn't
become an explorer
you just smashed the thing yeah yeah the
art the difficulty is still artificial
you feel before you started what if we
just don't have access to the knob well
that maybe is even scarier
right like we already exist in a world
of Nature and nature has been fine-tuned
over billions of years yeah
um to uh have
uh
humans build something
and then throw the knob away in some
Grand romantic gesture is horrifying
do you think of us humans as individuals
that are like born and die or is it are
we just all part of one living organism
that is
earth that is nature
I don't think there's a clear line there
I think it's all kind of just fuzzy I
don't know I mean I don't think I'm
conscious I don't think I'm anything I
think I'm just a computer program
so it's all computation but I think
running in your head is just a as a this
computation everything running in the
universe is computation I think I
believe the extended Church starting
thesis
yeah but if there seems to be an
embodiment to your particular
computation like there's a consistency
well yeah but I mean models have
consistency too
yeah models that have been rlh after
will continually say you know like well
how do I murder ethnic minorities oh
well I can't let you do that Hal there's
a consistency to that behavior
it's all rlhf
like we all our lhf each other we we
find we uh provide human feedback and
there that
thereby fine-tune these little pockets
of computation but it's still unclear
why that pocket of computation stays
with you like for years it just kind of
fall like you have this consistent
set of physics biology
uh what like whatever you call the the
neurons firing the electrical centers
the mechanical signals all of that that
seems to stay there and it contains
information it stores information and
that information permeates Through Time
and stays with you there's like memory
it's like sticky okay to be fair like a
lot of the models we're building today
are very even rlhf is nowhere near as
complex as the human loss function
reinforcement learning with human
feedback
um you know when I talked about will GPT
12 be AGI my answer is no of course not
I mean cross-entry loss is never going
to get you there you need
uh probably rln
fancy environments in order to get
something that would be considered like
AGI like
so to ask like the question about like
why I don't know like it's just some
Quirk of evolution right I don't think
there's anything particularly special
about
where I ended up
where humans ended up it's okay we have
human level intelligence would you call
the AGI whatever we have it's GI look I
actually I don't really even like the
word AGI
um but general intelligence is defined
to be whatever humans have okay so why
can GPT 12 not get us to AGI because
we're just like Linger on that
if your loss function is categorical
across entropy if your loss function is
just try to maximize compression uh I
have a SoundCloud I wrap and I try to
get chat GPT to help me write wraps and
the wraps that it wrote sounded like
YouTube comment wraps you know you can
go in any rap beat online and you can
see what people put in the comments and
it's the most like mid quality wrap you
can find is made good or bad mid is bad
it's like mid it's like every time I
talk to you I learn new words yeah
I was like uh is it is it like basic is
that what mid means kind of it's like
it's like middle of the curve right yeah
so there's like there's like a like that
intelligence curve yeah
um and you have like the dumb guy the
smart guy and then the mid guy actually
being the mid guys the worst the smart
guys like I put all my money in Bitcoin
the mid guys like you can't put money in
Bitcoin it's not real money
uh and all of it is a genius meme
that's another interesting one memes
the humor the idea the absurdity
encapsulated in a single image and it
just kind of propagates
virally
between all of our brains
I didn't get much sleep last night so
I'm very uh I sound like I'm high but I
swear I'm not
uh do you think we have ideas or ideas
have us
I think that we're gonna get super scary
memes once the AIS actually are
superhuman oh like the game I will
generate memes of course you think it'll
make humans laugh
I think it's worse than that so
um Infinite Jest it's introduced in the
first 50 pages is about a tape that you
uh once you watch it once you only ever
want to watch that tape
um in fact you want to watch the tape so
much that someone says okay here's a
hacksaw cut off your pinky and then I'll
let you watch the tape again and you'll
do it uh so we're actually going to
build that I think but it's not going to
be one static tape I think the human
brain is too complex to be
stuck in one static tape like that if
you look at like ant Brands maybe they
can be stuck on a static tape
but we're going to build that using
generative models we're going to build
the tick tock that you actually can't
look away from
so Tick Tock is already pretty close
there but the generation is done by
humans the algorithm is just doing their
recommendation but if this do if the
algorithm is also able to do the
generation
well it's a question about how much
intelligence is behind it right so the
content is being generated by let's say
one Humanity worth of intelligence and
you can quantify a Humanity right that's
a you know it's it's
a flops yada flops uh but you can
quantify it once that generation is
being done by a hundred Humanities
you're done
so let's actually scale that's the
problem
but also speed
yeah and what if it's sort of
manipulating
the very limited human dopamine engine
for porn imagine just Tick Tock but for
porn yeah it's like a Brave New World
I don't even know what it'll look like
right like again you can't imagine the
behaviors of something smarter than you
but a super intelligent and and an agent
that just dominates your intelligence so
much will be able to completely
manipulate you is it possible that it
won't really manipulate it'll just move
past us it'll just kind of exist the way
water exists or the air exists you see
and that's the whole AI safety thing
it's not the machine that's going to do
that it's other humans using the machine
that are going to do that to you yeah
because the machine is not interested in
hurting humans it's just the machine is
a machine yeah but the human gets the
machine and there's a lot of humans out
there very interested in manipulating
you
well let me bring up elieza yadkowski
who recently sat where you're sitting he
thinks that AI will almost surely kill
everyone
do you agree with him or not
yes but maybe for a different reason
okay
and then I'll try to uh get you to find
Hope or we could find a node to that
answer but why yes
okay why didn't nuclear weapons kill
everyone that's a good question I think
there's an answer I think it's actually
very hard to deploy nuclear weapons
tactically
it's very hard to accomplish tactical
objectives great I can nuke their
country I have an irradiated pile of
rubble I don't want that why not why
don't I want an irradiated pile of
rubble yeah for all the reasons no one
wants in a radiated pile of rubble
because you can't use that land for uh
for resources you can't populate the
land yeah well what you want a a total
victory in a war is not usually the
radiation and eradication of the people
there it's the subjugation and
domination of the people
okay so you can't use this strategically
tactically in a war yeah to help you to
help uh gain a military Advantage it's
all complete destruction all right yeah
but there's egos involved it's still
surprising still surprising that nobody
pressed the big red button
somewhat surprising but you see it's the
little red button that's going to be
pressed with AI That's gonna
you know and that's why we die it's it's
not because
the AI if there's anything in the nature
of AI it's just an H of humanity what's
the algorithm behind the little red
button well like what what what possible
ideas do you have for the how human
species ends sure so I think the most uh
obvious way to me is wireheading
we end up amusing ourselves to death
we end up all staring at that infinite
Tick Tock and forgetting to eat
maybe maybe it's even more benign than
this maybe we all just stop reproducing
now to be fair It's probably hard to get
all of humanity yeah yeah
it probably doesn't always go like the
the interesting thing about humanity is
the diversity oh yeah organisms in
general there's a lot of weirdos out
there two of them are sitting here I
mean diversity in humanity is we do
respect
I wish I was more weird
no like I'm kind of look I'm drinking
smart water man that's like a Coca-Cola
product right do you want corporate
George Haas
uh no the amount of diversity in
humanity I think is decreasing just like
all the other biodiversity on the planet
oh boy yeah right and social media is
not helping us go eat McDonald's in
China yeah
yeah no it's the interconnectedness
that's that's that's that's doing it oh
that's interesting so everybody starts
relying on the connectivity of the
internet
and over time that reduces the diversity
the intellectual diversity and then that
gets you everybody into a funnel there's
still going to be a guy in Texas there
is and yeah
Fair do I think AI kills us all uh I
think AI kills everything we call like
society today I do not think it actually
kills the human species I think that's
actually incredibly hard to do
you have a society like if we start over
that's tricky most of us don't know how
to do most things yeah but some of us do
and they'll be okay and they'll rebuild
after they uh
great AI
what's rebuilding look like how far like
how much do we lose like what is human
civilization done
that's interesting combustion engine
electricity
so uh
power and energy that's interesting
like how to harness energy
well they're going to be religiously
against that
are they going to get back to uh like
fire
sure I mean they'll be a they'll be
it'll be like you know some kind of
Amish looking kind of thing I think I
think they're going to have very strong
taboos against technology
like technology is almost like a new
religion technology is the devil yeah
and uh nature is God
sure to closer to nature but can you
really get away from AI if it destroyed
99 of the human species isn't it somehow
have a whole like a stronghold
what's interesting about
everything we build I think we're going
to build super intelligence before we
build any sort of robustness in the AI
we cannot build an AI that is capable of
going out into nature and surviving like
a um like a bird
right a bird is an incredibly robust
organism we've built nothing like this
we haven't built a machine that's
capable of reproducing
yes but
there's uh you know I work with Lego
robots a lot now I have a bunch of them
um
they're mobile
they can't reproduce but all they need
is I guess you're saying they can't
repair themselves but if you have a
large number if you have like 100
million of them let's just focus on them
reproducing right they have microchips
in them okay then do they include a Fab
no then how are they going to reproduce
well they're they it doesn't have to be
all on board right they can go to a
factory to a repair shop yeah but then
you're really moving away from
robustness yes all of life is capable of
reproducing without needing to go to a
repair shop life will continue to
reproduce in the complete absence of
civilization
robots will not
so when the if if the AI apocalypse
happens
I mean the AIS are going to probably die
out because I think we're going to get
again super intelligence long before we
get robustness
what about if you just improve
the Fab to where
you just have a 3D printer that can
always help you
well that'd be very interesting I'm
interested in building that
of course you are you think how
difficult is that problem to have a
robot that uh
basically can build itself very very
hard
I think you've mentioned this like uh to
me or somewhere where people think it's
easy conceptually
and then they remember that you're going
to have to have a Fab yeah on board of
course
so 3D printer that prints a 3D printer
yeah yeah on legs
yeah hard
well because it's I mean a 3D printer is
a very simple machine right okay you're
gonna print chips you're going to have
an atomic printer how are you going to
dope the Silicon yeah right
how are you gonna etch the Silicon
you're gonna have to have a a very
interesting kind of Fab if you wanna
have a lot of computation on board
but you can do like
structural type of robots that are dumb
yeah but structural type of robots
aren't going to have the intelligence
required to survive in any complex
environment what about like ants type of
systems we have like trillions of them
I don't think this works I mean again
like ants at their very core are made up
of cells that are capable of
individually reproducing they're doing
quite a lot a lot of computation that
we're taking for granted it's not even
just the computation it's that
reproduction is so inherent okay so like
there's two stacks of life in the world
there's bio the biological stack and the
Silicon stack the biological stack
starts with reproduction
reproduction is at the absolute core the
first proto-rna organisms were capable
of reproducing
wait the Silicon stack despite as far as
it's come
is nowhere near being able to reproduce
yeah
so the the Fab movement
uh digital fabrication
Fabrication in the full range of what
that means is still in the early stages
yeah you're interested in this world
even if you did put a Fab on the machine
right let's say okay and we can build
apps we know how to do this Humanity we
can probably put all the precursors that
build all the machines in the Fabs also
in the machine so first off this machine
is going to be absolutely massive
I mean we almost have a like think of
the size of the thing required to
reproduce a machine today right like is
our civilization capable of reproduction
can we reproduce our civilization on
Mars
if we were to construct a machine that
is made up of humans
like a company that can reproduce itself
yeah
I don't know it feels like like 115
people
I get so much harder than that
120.
I believe that Twitter can be run by 50
people uh
I think that this is going to take most
of like it's just most of society right
like we live in one globalized world no
but you're not interested in running
Twitter you're interested in seeding
like um you want to see the civilization
and then because humans can like oh okay
you're talking about yeah okay so you're
talking about the humans reproducing and
like basically like what's the smallest
self-sustaining colony of humans yeah
yeah okay fine but they're not going to
be making five nanometer chips over time
they will I think you're being like we
have to expand our conception of time
here going back to the original uh time
scale I mean over across
maybe 100 Generations we're back to
making chips no if you see the colony
correctly maybe or maybe they'll watch
our
Colony die out over here and be like
we're not making chips don't make chips
maybe you have to seed that Colony
correctly whatever you do don't make
chips chips are what led to their
downfall
hmm
well that is the thing that humans do
they they come up they construct a devil
a good thing and a bad thing and they
really stick by that and then they
murder each other over that there's
always one asshole in the room who
murders everybody
and usually makes tattoos and nice
branding do you need that asshole that's
a question right Humanity works really
hard today to get rid of that asshole
but I think they might be important yeah
this whole freedom of speech thing it's
it's the freedom of being an asshole
seems kind of important right
man this thing this Fab this human Fab
that we constructed as human
civilization is pretty interesting and
now it's building artificial copies of
itself or artificial copies of various
aspects of itself that seem interesting
like intelligence
and I wonder where that goes
I like to think it's just like another
stack for life like we have like the
biostack life like we're a biostack life
and then the Silicon stack life but it
seems like the ceiling
or they might not be a ceiling and or at
least the ceiling is much higher for the
for the Silicon stack oh no I don't I we
don't know what the ceiling is for the
biostack either the biostack the
biostack just seemed to move slower
um you have Moore's Law uh which is not
dead despite many proclamations uh into
house stack or the silver in the Silicon
stack and you don't have anything like
this in the biostack so I have a meme
that I posted I tried to make a meme it
didn't work too well but um I posted a
picture of uh you know Ronald Reagan and
Joe Biden and you look this is 1980 and
this is 2020. yeah and these two humans
are basically like the same right
there's no there's no like like they're
there's been no change in humans in the
last 40 years yeah and then I posted a
computer from 1980 in a computer from
2020. wow
yeah with their early early stages right
which is why you said when you said the
Fab the size of the Fab required to make
another Fab is like uh
very large right now oh yeah but
computers were very large
um 80 years ago
and they got pretty tiny and there there
people are starting to want to wear them
on their face
in order to escape reality that's the
thing in order to be live inside the
computer
put a screen right here I don't have to
see the rest of you assholes I've been
ready for a long time you like virtual
reality I love it
do you want to live there yeah
yeah
part of me does too
how far away are we do you think
judging from what you can buy today far
very far
I gotta tell you that I had the
experience of uh meta's Kodak Avatar
where it's a ultra high resolution scam
it looked real
I mean the headsets just are not quite
like eye resolution yet
I haven't put on any headset where I'm
like oh I this could be the real world
whereas when I put good headphones on
audio was there I think we we can
reproduce audio that I'm like I'm
actually in the jungle right now I I if
I close my eyes I can't tell I'm not
yeah but then there's also smell and all
that kind of stuff sure
I don't know I
the the power of imagination or the
power of the the mechanism in the human
mind that fills the gaps
that kind of reaches and wants to make
the thing you see in the virtual world
real to you
I believe in that power or humans want
to believe yeah
okay what if you're lonely what if
you're sad what if you're really
struggling in life and here's a world
where you don't have to struggle anymore
humans want to believe so much that
people think the large language models
are conscious that's how much humans
want to believe
strong war is he's throwing left and
right hooks uh why do you think large
language models are not conscious I
don't think I'm conscious oh so what is
consciousness then George cost it's like
what it seems to mean to people it's
just like a word that atheists use for
Souls
sure but that doesn't mean soul is not
an interesting word
if Consciousness is a spectrum I'm
definitely way more conscious than the
large language models are
I think the large language models are
less conscious than a chicken
when's the last time you've seen a
chicken
uh in Miami like a couple months ago how
no like a living chicken living chickens
walking around Miami it's crazy
like on the street yeah like a chicken a
chicken yeah all right
all right I was trying to call you all
like like a good journalist and I uh I
got shut down
okay but uh you don't
think much about this kind of
subjective feeling that it it feels like
something
to exist and then as an observer
you can
have a sense that an entity is not only
intelligent but has a kind of
subjective experience of its reality
like a self-awareness that is capable of
like suffering of hurting of being
excited by the environment in a way
that's not merely uh
kind of an artificial response but a
deeply felt one humans want to believe
so much that if I took a rock and a
Sharpie and Drew a sad face on the Rock
they'd think the rock is sad
yeah and you're saying when we look in
the mirror we we apply the same smiley
face with rock pretty much yeah that's
not isn't that weird though that you're
not conscious is that no
but you do believe in Consciousness not
really it's just it's unclear okay so
you it's like a little like a a symptom
of the bigger thing that's not that
important yeah I think it's interesting
that like the human systems seem to
claim that they're conscious and I guess
it kind of like says something in a
straight up like okay what do people
mean when even if you don't believe in
Consciousness what do people mean when
they say Consciousness and there's
definitely like meanings to it what's
your favorite thing to eat
pizza cheese pizza what are the toppings
I like cheese pizza don't say pineapple
okay pepperoni because they put any Ham
on it oh that's real bad what's the best
what's the best pizza what are we
talking about here like you like cheap
crappy Pizza Chicago deep dish cheese
pizza oh that's that's my favorite there
you go you bite into a deep dish Chicago
deep dish pizza and it feels like so you
were starving you haven't eaten oh yeah
for 24 hours you just bite in and you're
hanging out with somebody that matters a
lot to you and you're there with the
pizza sounds real nice yeah all right it
feels like something I'm I'm George
motherfucking Hots eating a fucking
Chicago deep dish pizza there's just the
full Peak like living experience yeah of
Being Human the top of the Human
Condition sure it feels like something
to experience that
why does it feel like something that's
Consciousness isn't it
if that's the word you want to use to
describe it sure I'm not going to deny
that that feeling exists I'm not going
to deny that I experienced that feeling
when I guess what I kind of take issue
to is that there's some like like how
does it feel to be a web server do 404
is hurt
not yet how would you know what
suffering looked like sure you can
recognize a suffering dog because we're
the same stack as the dog
all the biostack stuff kind of
especially mammals you know it's just
really easy you can
game recognize this game yeah versus the
Silicon stack stuff it's like you have
no idea
you have view it well the little thing
has learned to mimic
you know
but then I realize that that's all we
are too oh look the little thing has
learned to mimic yeah I guess uh yeah
404 could be could be suffering but it's
so far
from our kind of living organism our
kind of Stack but it feels like AI can
start
maybe mimicking the biological stack
better but about it because it's trained
we trained it yeah
and so in that maybe that's the
definition of Consciousness is the the
biostat Consciousness the definition of
Consciousness is how close something
looks to human sure I'll give you that
one no how close something is to The
Human Experience sure
it's a very it's a very anthropocentric
definition but well that's all we got
sure no and I I don't mean to like I
think there's a lot of value in it look
I just started my second company my
third company will be AI girlfriends
oh like I mean I want to find out what
your fourth company is wow because I
think once you have ai girlfriends it's
uh
oh boy
does it get interesting
well maybe let's go there I mean the
relationships with AI That's creating
human-like organisms right
and part of being human is being
conscious is being having the capacity
to suffer having the capacity to
experience this leveragely in such a way
that you can empathize
the AIS is going to empathize with you
and you can empathize with it or you can
project your uh anthropomorphic sense of
what the other entity is experiencing
and and an AI model would need to um
yeah to create that experience inside
your mind and it doesn't seem that
difficult yeah but okay so here's where
it actually gets totally different right
would you interact with another human
you can make some assumptions
yeah when you interact with these models
you can't you can make some assumptions
that that other human experiences
suffering and pleasure in a pretty
similar way to you do the golden rule
applies
with an AI model this isn't really true
right these these large language models
are good at fooling people because they
were trained on a a whole bunch of human
data and told to mimic it yep but if if
the AI system says hi my name is
Samantha
has a backstory yeah I went to college
here and there yeah maybe it'll
integrate this in AI system I made some
chat Bots I give them backstories it was
lots of fun I was so happy when llama
came out yeah well we'll talk about a
lot we'll talk about all that but like
you know the rock with the smiley face
yeah
why this it seems pretty natural for for
you to anthropomorphize that thing and
then start dating it and before you know
it you're married and have kids with a
rock
with a rock those pictures on Instagram
with you and a rock and smiley face to
be fair like you know something that
people generally look for when they're
looking for someone to date is
intelligence and some form and The Rock
doesn't really have intelligence only a
pretty desperate person would date a
rock
I think we're all desperate deep down oh
not rock level desperate all right uh
not rock level
desperate but AI level desperate I don't
know I think all of us have a deep
loneliness it just feels like the
language models are there
oh I agree and you know what I won't
even say this so cynically I will
actually say this in a way that like I
want AI friends I do yeah like I would
love to you know again I the language
models now are still a little like
people are impressed with these GPT
things and I look at like or or like or
uh the the co-pilot the coding one and
I'm like okay this is like Junior
engineer level and these people are like
Fiverr level artists and copywriters
like okay great we got like Fiverr and
like Junior Engineers okay cool like and
this is just a start and it will get
better right like I would I can't wait
to have ai friends who are more
intelligent than I am so Fiverr is just
a temporary it's not the ceiling no
definitely not
is it uh is it count as cheating
when you're talking to an AI model
emotional cheating
that's that's up to you and your human
partner to Define oh you have to all
right you can yeah you have to have to
have that conversation I guess all right
let me integrate that with uh with porn
and all this well I don't know I mean a
similar kind of to porn yeah yeah
I think people in relationships have
different views on that
yeah but most people don't have like
a serious open conversations about
all the different aspects of what's cool
and what's not
and it feels like AI is a really weird
conversation to have the porn one is a
good branching off sure like these
things you know one of my scenarios that
I put in my chat bot is like uh you know
uh
a nice girl named Lexi she's 20 she just
moved out to LA she wanted to be an
actress but she started doing only fans
instead and you're on a date with her
enjoy
oh man yeah and so was that if you're
actually dating somebody in real life is
that cheating
I feel like it gets a little weird sure
it gets real weird it's like what are
you allowed to say to an AI bot imagine
having that conversation with a
significant other I mean these are all
things for people to Define in their
relationships what it means to be human
is just going to start to get weird
especially online like how do you know
like there'll be moments when you'll
have what you think is a real human you
interacted with on Twitter for years and
you realize it's not
I spread I love this meme uh Heaven
Banning you know what shadow Banning
yeah right Shadow Banning okay you post
no one can see it Heaven Banning you
post no one can see it but a whole lot
of AIS are spot up to interact with you
well maybe that's what the way human
civilization ends is all of us I haven't
banned there's a great uh it's called My
Little Pony Friendship is optimal it's a
Sci-Fi story that uh explores this idea
friendship is optimal friendship is
optimal yeah I'd like to have some Elite
stuff on the intellectual realm some AI
friends that argue with me
but the the Romantic realm is weird
definitely weird
but
not out of the realm of uh
the uh the kind of weirdness that the
human civilization is capable of I think
I think I want it look I want it if no
one else wants it I want it yeah I think
a lot of people probably wanna there's a
deep loneliness
and I'll feel there loneliness and you
know it just will only advertise to you
some of the time yeah maybe the
conceptions of monogamy change too like
I grew up in a time like I value
monogamy but maybe that's a silly notion
when you have
arbitrary number of AI systems
um this this um this interesting path
from rationality to polyamory yeah that
doesn't make sense for me for you but
you're just a biological organism it was
born before
like read the internet really took off
the crazy thing is like culture is
whatever we Define it as right these
things are not you like is a lot problem
and moral philosophy right there's no
like like okay what is might be that
like computers are capable of mimicking
uh you know girlfriends perfectly they
pass the girlfriend turning test right
but that doesn't say anything about a
lot that doesn't say anything about how
we ought to respond to them as a
civilization that doesn't say we ought
to get rid of monogamy right that's a
completely separate question really a
religious one
girlfriend touring test I wonder what
that looks like girlfriend are you
writing that
uh will you be the the Allen touring of
the 21st century that writes the uh the
girlfriend touring test I mean of course
my my hey girlfriends their goal is to
pass the girlfriend Turing test no but
you there should be like a paper that
kind of defines the test
I mean the question is if it's deeply
personalized or there's a common thing
that really gets
everybody
yeah I mean you know look we're a
company we don't get everybody we just
have to get a large enough uh clientele
today I thought you already already
thinking company
all right let's uh before we go to
company number three and Company Number
Four let's go to company number two all
right tiny Corp
possibly one of the greatest names of
all time for a company
uh you've launched a new company called
tiny Corp that leads the development of
tiny grad
what's the origin story of tiny Corp and
Tiny grad I started tiny grad as a like
a toy project just to teach myself okay
like what is a convolution
uh what are all these options you can
pass to them what is the derivative or
convolution right very similar to uh
carpathy wrote micrograd I'm very
similar
and then
I started realizing I started thinking
about like AI chips I started thinking
about chips that run
Ai and I I was like well okay this is
going to be a really big problem
if Nvidia becomes a monopoly here
um how long before Nvidia has
nationalized
so you uh
one of the reasons that start tiny Corp
is to challenge Nvidia
it's not so much
to challenge Nvidia I actually I I like
Nvidia and
it's
to make sure power stays decentralized
yeah and here's uh computational Power
until you Nvidia is kind of locking down
the computational power of the world if
Nvidia becomes just like 10x better than
everything else you're giving a big
advantage to somebody who can secure
Nvidia as a resource
yeah in fact if Jensen watches this
podcast you may want to consider this
you may want to consider making sure his
company is not nationalized
do you think that's an actual threat oh
yes
no but there's so much uh you know
there's AMD so we have Nvidia and AMD
great all right
but you know you don't think there's
like a push
towards like selling like Google selling
tpus or something like this you don't
think there's a push for that have you
seen it Google loves to rent utpus it
doesn't you can't buy it at Best Buy
so I started to work on a uh
on a chip I was like okay what's it
gonna take to make a chip and my first
Notions were all completely wrong about
why about like how you could improve on
gpus uh and I will take this this is
from uh Jim Keller on your podcast and
this is one of my absolute favorite
descriptions of computation
um so there's three kinds of computation
paradigms that are common in the world
today
other CPUs and CPUs can do everything
CPUs can do add and multiply they can do
load and store and they can do compare
and branch and when I say they can do
these things they can do them all fast
right so compare and Branch are unique
to CPUs and what I mean by they can do
them fast is they can do things like
Branch prediction and speculative
execution and they spend tons of
transistors on these like super deep
reorder buffers in order to make these
things fast
then you have a simpler computation
model gpus gpus can't really do compare
and Branch I mean they can but it's
horrendously slow
but gpus can do arbitrary load in store
right gpus can do things like X
dereference Y so they can fetch from
arbitrary pieces of memory they can
fetch from a memory that is defined by
the contents of the data
um the third model of computation is
dsps and dsps are just add and multiply
right like they can do load in stores
but only static load in stores only
loads and stores that are known before
the program runs
look at neural networks today and 95
percent of neural networks are all the
DSP paradigm
they are just statically scheduled ads
and multiplies
so tinyguard really took this idea and
and I'm still working on it to extend
this as far as possible
um every stage of the stack has turned
completeness right python has turned
completeness and then we take python we
go into C plus plus which is starting
complete and maybe C plus plus calls
into some Cuda kernels which are turning
complete the Cuda kernels go through lvm
which is turning complete into PTX which
is turning complete into SAS which is
turn complete on a turn complete
processor I want to get turn
completeness out of the stack entirely
because once you get rid of turn
completeness you can reason about things
Rice's theorem and the halting problem
do not apply to Admiral machines
okay what's the power and the value of
getting torn completeness out of out of
are we talking about the hardware or the
software every layer of a stack every
layer every layer of the stack removing
turn completeness allows you to reason
about things right so the reason you
need to do Branch prediction in a CPU
and the reason it's prediction and the
branch predictors are I think they're
like 99 on CPUs why do they get one
percent of them wrong well they get one
percent wrong because you can't know
right that's the halting problem it's
equivalent to the halting problem to say
whether a branch is going to be taken or
not
um I can show that but
the
admal machine the neural network runs
the identical compute every time the
only thing that changes is the data
so when you realize this you think about
okay how can we build a computer and how
can we build a stack that takes maximal
advantage of this idea
uh so what makes tiny grad different
from other neural network libraries is
it does not have a primitive operator
even for matrix multiplication
and this is every single one they even
have primitive operations things like
convolutions so no matte mull no mammal
well here's what a maple is so I'll use
my hands to talk here so if you think
about a cube and I put my two matrices
that I'm multiplying on two faces of the
cube right you can think about the
Matrix multiply as okay the N cubed I'm
going to multiply for each one in the
cubed and then I'm going to do a sum
which is a reduce up to here to the
third face of the cube and that's your
multiplied Matrix
so what a matrix multiply is is a bunch
of shape operations right a bunch of
permute three shapes and expands on the
two matrices
a multiply n cubed a reduce n cubed
which gives you an N squared Matrix
okay so what what is the minimum number
of operations that can accomplish that
if you don't have mapmall as a primitive
so tiny grad has about 20. and you can
compare tiny grad's uh opsat or IR to
things like xla or Prim torch so xla and
Prim torch are ideas where like okay
torch has like 2 000 different kernels
um Pi torch 2.0 introduced Prim torch
which has only 250.
uh tiny grad has order of magnitude 25.
it's it's 10x less than xli or Prim
torch and you can think about it as kind
of like Risk versus Sisk right these
other things are cisc like systems uh
tendergrad is risk
and risk one risk architecture is going
to change everything 1995 hackers
wait really that's an actual thing
Angelina Jolie delivers the line risk
architecture is going to change
everything in 1995. and here we are with
arm and the phones
an arm everywhere wow I love it when
movies actually have real things in them
right okay interesting and see this is
like uh so you're thinking of this as
the risk architecture of ml stack
25 huh what what uh can you can you go
through the
uh
the four op types sure
um okay so you have unaryops which take
in uh a tensor and return a tensor of
the same size and do some unary up to it
X log uh reciprocal sign right they take
in one and they're pointwise
really yeah value
um almost all activation functions are
unaryops
um some combinations of unaryops
together is still a unaryop
um then you have binary apps binary apps
are like a point wise Edition
multiplication division compare uh it
takes in two tensors of equal size and
outputs one tensor
then you have reduce Ops reduce Ops will
like take a three-dimensional tensor and
turn it into a two-dimensional tensor or
three-dimensional tensor turn into Zero
Dimensional tensor things like a sum or
a Max are really the common ones there
and then the four type is movement Ops
and movement Ops are different from the
other types because they don't actually
require computation they require
different ways to look at memory so that
includes reshapes permutes expands flips
those are the main ones probably so with
that you have enough to make a map model
and convolutions and every convolution
you can imagine dilated convolution
striated convolutions transposed
convolutions
you write on GitHub about laziness
uh showing a map Mall
matrix multiplication see how despite
the style it is fused into one kernel
with the power of laziness can you
elaborate on this power of laziness sure
so if you type in pytorch a times B plus
c
uh what this is going to do is it's
going to first multiply Adam B A and B
and store that result into memory and
then it is going to add C by reading
that result from memory reading C for
memory and uh writing that out to memory
there is way more loads and stores to
memory than you need there if you don't
actually do a times b as soon as you see
it if you wait
until the user actually realizes that
tensor until the laziness actually
resolves
um you confuse that plus C this is like
it's the same way Haskell works so uh
what's the process of porting a model
into dynagrad so tiny grad's front end
looks very similar to Pi torch
um I probably could make a perfect or
pretty close to perfect interop layer if
I really wanted to I think that there's
some things that are nicer about tiny
grad syntax than pytorch but the front
end looks very torch-like you can also
load in Onyx models we have more Onyx
tests passing than core ml
okay so we'll pass Onyx run time soon
what about like the developer experience
of tiny grad
um what it feels like what are the
um
versus pytorch by the way I really like
pie torch I I think that it's actually a
very good piece of software
um I think that they've made a few
different trade-offs and these different
trade-offs are uh where you know tiny
grad takes a different path one of the
biggest differences is it's really easy
to see the kernels that are actually
being sent to the GPU
right if you run pytorch on the GPU
you like do some operation and you don't
know what kernels ran you don't know how
many kernels ran you don't know how many
flops were used you don't know how much
memory accesses were used tiny grad type
debug equals two and it will show you in
this beautiful style
um every kernel that's run
how many flops
and how many bytes
so can you just
Linger on what problem tiny grad solves
tiny grad solves the problem of porting
new ml accelerators quickly
one of the reasons or tons of these
companies now I think um
Sequoia marked graphcore to xero right
cerebus tens torrent uh grock all of
these ml accelerator companies they
built chips the chips were good the
software was terrible uh and part of the
reason is because I think the same
problems happening with dojo
it's really really hard to write a pie
torch port because you have to write 250
kernels and you have to tune them all
for performance
uh what is Jim Jim Keller think about
tiny grad
you guys have hung out quite a bit so
he's uh you know he's he was involved
he's involved with that story what's his
uh praise and what's his criticism of
what you're doing with your life
look
my prediction for tense torrent is that
they're gonna pivot to making risk 5
chips
CPUs
why
because AI accelerators are a software
problem not really a hardware problem Oh
interesting so you don't think
you think the diversity of AI
accelerators in the hardware space is
not going to be a thing that exists long
term I think what's gonna happen is if I
can finish okay
if you're trying to make an AI
accelerator
you better have the capability of
writing a torch level performance stack
on Nvidia gpus if you can't write a
torch stack on Nvidia gpus and I mean
all the way I mean down to the driver
there's no way you're going to be able
to write it on your chip because your
Chip's worse than a Nvidia GPU the first
version of the chip you tape out is
definitely worse when you're saying
writing that stack is really tough yes
and not only that actually the chip that
you tape out almost always because
you're trying to get advantage over
Nvidia you're specializing the hardware
more it's always harder to write
software for more specialized Hardware
like a GPU is pretty generic and if you
can't write an Nvidia stack there's no
way you can write a stack for your chip
so my Approach with tinygrad is first
write a performant Nvidia stack we're
targeting AMD
um
so you did say a few to Nvidia a little
bit with love with love yeah so with the
Yankees you know I'm a Mets man oh
you're you're a mess fan a risk a risk
fan and a mess fan what's the hope that
AMG has I mean you did uh build with AMD
recently that I saw uh how does the uh
the the 7900 xdx compared to the RTX 40
90 or 4080. well let's start with the
fact that the 7900 xdx kernel drivers
don't work and if you run demo apps and
Loops it panics the kernel okay so if
this is a software issue
Lisa Sue responded to my email oh I
reached out I was like this is you know
really
like I understand if you're seven by
seven transposed win a grad com this
slower than nvidias but literally when I
run demo apps in a loop the kernel
panics
so just adding that Loop
yeah I just I just literally took their
demo apps and wrote like wild true
semicolon do the app semicolon done in a
bunch of screens right this is like like
the most primitive fuzz testing why do
you think that is they're just not
seeing a market in the in um machine
learning they're changing they're trying
to change they're trying to change and I
had a pretty positive interaction with
them this week last week I went on
YouTube I was just like that's it I give
up on AMD like this is their driver
doesn't like I'm not gonna I'm not gonna
you know I'll go with Intel gpus right
Intel gpus have better drivers
so you're kind of spearheading the
diversification of uh gpus yeah and I'd
like to extend that diversification to
everything I'd like to diversify the
right the more
my central thesis about the world is
there's things that centralize power and
they're bad and there's things that
decentralize power and they're good
everything I can do to help decentralize
power I'd like to do
so you're really worried about the
centralization of Nvidia that's
interesting and you don't have a
fundamental hope for the the
proliferation of Asics
uh except in the cloud
I'd like to help them with software no
actually there's only the only Asic that
is remotely successful is Google's TPU
and the only reason that's successful is
because Google wrote a machine learning
framework
I I think that you have to write a
competitive machine learning framework
in order to be able to build an Asic
um
you think meta with biotorch builds a
competitor I hope so okay they have one
they have an internal One internal I
mean uh public facing with a nice Cloud
interface and so on
I don't want a class
you don't like Cloud I don't like Cloud
what do you think is the fundamental
limitation of cloud fundamental
limitation to cloud is who owns the uh
the off switch so it's the power to the
people yeah and you don't you don't like
the man to have all the power exactly
all right
and right now the only way to do that is
with Nvidia gpus if you want performance
and stability
interesting it's uh it's a costly
investment emotionally to go with amd's
uh well let me add sort of on a tangent
to ask you what
um
you've built quite a few PCS what's your
advice on how to build a good custom PC
for uh let's say for the different
applications that you use for gaming for
uh machine learning well you shouldn't
build one you should buy a box from the
tiny Corp
I heard rumors Whispers
about this box in the tiny Corp what's
what's this thing look like what it what
is it what is it called it's called the
Tiny Box Tiny Box um it's fifteen
thousand dollars and it's almost a paid
a flop of compute it's over 100
gigabytes of GPU Ram it's over five
terabytes per second of GPU memory
bandwidth
uh I'm gonna put like four nvmes in in
raid you're gonna get like 20 30
gigabytes per second of Drive read
bandwidth
I'm gonna I'm gonna build like
the best deep learning box that I can
that plugs into one wall outlet
okay can you go through the specs again
a little bit from your from memory yeah
so it's almost a paid a flop of compute
so in D Intel today I'm leaning toward
AMD
um but we're pretty agnostic to the type
of compute
the the main limiting spec is a 120 volt
15 amp circuit
okay well I mean it because in order to
like like there's a plug over there all
right you have to be able to plug it in
um we're also going to sell the tiny
rack which like what's the most power
you can get into your house without
arousing suspicion uh and one of the one
of the answers is an electric car
charger wait where does the rack go your
garage
interesting the car charger a wall
outlet is about 1500 watts a car charger
is about 10 000 Watts
what is the most amount of power you can
get your hands on without a rousing
suspicion that's right George Haas okay
uh so the the tiny boxing you said nvmes
and raid uh I forget what you said about
memory all that kind of stuff okay so uh
what about what gpus again probably
probably 7900 xdx is but maybe 30 90s
maybe a 770s those are in towels you're
flexible or still exploring I'm still
exploring I wanna I wanna deliver a
really good experience to people and
yeah what gpus I end up going with again
I'm leaning toward AMD we'll see you
know in my email what I what I said to
AMD is like just dumping the code on
GitHub is not open source open source is
a culture
open source means that your issues are
not all one-year-old stale issues open
source means developing in public
if you guys can commit to that I see a
real future for AMD as a competitor
doing video
well I'd love to get a tiny box to MIT
so whenever it's ready well done let's
do it we're taking pre-orders I took
this from Elon I'm like a hundred dollar
fully refundable pre-orders is it going
to be like the Cyber truck it's going to
take a few years or no I'll try to do a
fast one it's a lot simpler it's a lot
simpler than a truck well there's
complexities not to just the
uh putting the thing together but like
shipping and all this kind of stuff the
thing that I want to deliver to people
out of the box is being able to run 65
billion parameter llama in fp16 in real
time in like a good like 10 tokens per
second or five Focus per second or
something just it works yep drama's
running
uh or something like llama experience
yeah or I think Falcon is is the new one
experience a chat with the largest
language model that you can have in your
house
yeah from from a wall plug from a wall
plug yeah actually for inference it's
not like even more power would help you
get more
the biggest model released is is 65
billion parameter llama as far as I know
so it sounds like Tiny Box will
naturally uh pivot towards company
number three because you could just get
the girlfriend
and uh I mean or boyfriend
that one's harder actually the boyfriend
is harder boyfriend's harder yeah I
think that's a very biased statement I
think a lot of people just say what's
what why is it harder to replace a
boyfriend than the other girlfriend with
the artificial llm because women are
attracted to status and power and men
are attracted to Youth and beauty
no I mean that's what I mean both are
mimicable easy to the language model no
no machines do not have any status or
real power I don't know I think you both
well first of all you're using language
mostly uh to
to communicate Youth and beauty and
power and status but status
fundamentally is a zerosome game
all right where's Youth and beauty or
not
no I think status is a narrative you can
construct I I don't think status is real
I don't know I just think that that's
why it's harder you know yeah maybe it
is my biases I think status is way
easier to fake I also think that you
know men are probably more desperate and
more likely to buy my product so maybe
they're a better target market
desperation is interesting easier to
fool that's I could I could see that
yeah look I mean look I know you can
look at porn viewership numbers right a
lot more men watch porn than women yeah
that's why that is
well there's a lot of questions and
answers you can get there
anyway with the with the Tiny Box how
many gpus in Tiny Box six
[Laughter]
oh man and I'll tell you why it's sex
yeah uh so AMD epic processors have 128
Lanes of pcie
um I want to leave enough lanes for
some uh drives and I want to leave
enough lanes for some networking
how do you do cooling for something like
this Ah that's one of the big challenges
not only do I want the cooling to be
good I want it to be quiet I want the
Tiny Box to be able to sit comfortably
in your room right this is really going
towards the girlfriend thing because you
want to run an llm I'll give I'll give
them more I mean I can talk about how it
relates to company number one
comma AI yeah well but yes quiet oh
quiet because you maybe a potential one
around in a car no no quiet because you
want to put this thing in your house and
you want it to coexist with you if it's
screaming it's 60 DB you don't want that
in your house you'll kick it out 60 DB
yeah yeah I want like 40 45. so how do
you make the cooling uh quiet that's an
interesting problem in itself
um a key trick is to actually make it
big ironically it's called the Tiny Box
yeah but if I can make it big a lot of
that noise is generated because of high
pressure air if you look at like a 1u
server a 1u server has these super high
pressure fans they're like super deep
and they they're like Janus versus if
you have something that's big well I can
use a big and you know you know they
call them Big Ass Fans those ones that
are like huge on the ceiling and they're
completely silent so tiny box
will be big
it is the uh I do not want it to be
large according to UPS I want it to be
shippable as a normal package but that's
my constraint there
interesting well the the fans stuff it
can't can't it be assembled on location
or no no
it has to be well you're you're look I
want to give you a great out of the box
experience I want you to lift this thing
out I want it to be like like the Mac
you know
Tiny Box the Apple experience yeah
I love it okay and so tiny box would run
tiny grad like what what what do you
envision this whole thing to look like
we're talking about like
uh Linux with the full
software engineering environment
it's just not pie torch but tiny grad
yeah we did a poll if people want Ubuntu
or Arch we're going to stick with Ubuntu
oh interesting what's your favorite uh
flavor of Linux Ubuntu I like Ubuntu
mate however you pronounce that meat
so how do you uh you've gotten llama
into tiny grad you've gotten stable
diffusion this Italian grind what was
that like can you comment on like what
are
um what are these models what's
interesting about putting them up so
what's yeah like what what are the the
challenges with what's naturally what's
easy all that kind of stuff there's a
really simple way to get these models
into tiny grad and you can just export
them as Onyx and then tiny grad can run
onyx
um so the ports that I did of llama
stable diffusion and now whisper are
more academic to teach me about the
models but they are cleaner than the
pytorch versions you can read the code I
think the code is easier to read it's
less lines there's just a few things
about the way tiny grid writes things
here's here's a complaint I have about
pytorch
nn.relu is a class
right so when you create it when you
create an end module you'll put your nn
relues
as in init and this makes no sense value
is completely stateless
why should that be a class
but that's more like a software
engineering thing or do you think it has
a cost on performance oh no it doesn't
have a Custom Performance
um but yeah no I think that it it's
that's what I mean about like tiny
grad's front end to being cleaner I see
uh what do you think about Mojo I don't
know if you've been paying attention to
the programming language that does
um some interesting ideas that kind of
intersect uh tiny grad I think that
there is a spectrum and like on one side
you have Mojo and on the other side you
have like ggml
um gml is this like we're going to run
llama fast on Mac and okay we're going
to expand out to a little bit but we're
going to basically like depth first
right Mojo is like we're gonna go breath
first we're gonna go so wide that we're
gonna make all of python fast and Tiny
grads in the middle
we are going to make neural networks
fast
yeah but they uh they try to really get
it to be fast compiled down to the
specifics uh hardware and make that
compilation step
as flexible and resilient as possible
yeah but they've turned completeness
and that limits you
Charlie that's what you're seeing is
somewhere in the middle so you're
actually going to be targeting some
accelerators some
like some some number not one my goal is
step one build an equally performance
stack to pie torch on Nvidia and AMD but
with way less lines and then step two is
okay how do we make an accelerator right
but you need step one you have to first
build the framework before you can build
the accelerator uh can you explain ml
perf
uh what's your approach in general to
benchmarking tiny grad performance
so I'm much more of a like
build it the right way and worry about
performance later
um there's a bunch of things where I
haven't even like really dove into
performance the only place where tiny
grad is competitive performance wise
right now is on Qualcomm gpus so tiny
grid is actually used an open pilot to
run the model so the driving model is is
Tiny grad when did that happen that
transition about eight months ago now
um and it's 2x faster than qualcomm's
Library
what's the hardware of open uh that open
pilot runs on the the kamea it's a
Snapdragon 845
okay so this is using the GPU so the GPU
is an adreno GPU there's like different
things there's a really good Microsoft
paper that talks about like mobile gpus
and why they're different from desktop
gpus
um one of the big things is in a desktop
GPU you can use buffers uh on a mobile
GPU image textures are a lot faster
and a mobile GPU image textures okay
and so you want to be able to leverage
that I want to be able to leverage it in
a way that it's completely generic right
so there's a lot of this xiaomi has a
pretty good open source library for
mobile gpus called mace where they can
generate where they have these kernels
but they're all hand coded right so
that's great if you're doing three by
three comps that's great if you're doing
dense map models but the minute you go
off the beaten path a tiny bit well your
performance is nothing since you
mentioned openpile I'd love to get an
update
in the company number one common AI
world how are things going there in the
development
of uh semi autonomous driving
you know
almost no one talks about FSD anymore
and even less people talk about open
pilot we've solved the problem like we
solved it years ago
what's the problem exactly
what what does solving it mean solving
means how do you build a model that
outputs a human policy for driving how
do you build a model that given a you
know reasonable set of sensors outputs a
human policy for driving
uh so you have you know companies like
women Cruise which are hand coding these
things that are like quasi-human
policies
then you have
Tesla and maybe even to more of an
extent comma asking okay how do we just
learn human policy of data
the big thing that we're doing now and
we just put it out on Twitter
at the beginning of comma
we published a paper
called learning a driving simulator
and the way this thing worked was it's a
it was an auto encoder
and then an RNN in the middle right uh
you take an auto encoder you compress
the picture you use an RNN predict the
next date and these things were you know
it was a laughingly bad simulator right
this is 2015 error machine learning
technology
today we have vqvae and Transformers
we're building Drive GPT basically
Drive GPT okay uh
so and that's trained on what is it
trained in a self-supervised way it's
trained on all the driving data to
predict the next frame so really trying
to uh learn a human policy what would a
human do well actually our simulator is
conditioned on the pose so it's actually
a simulator you can put in like a state
action pair and get out the next date
okay
um and then once you have a simulator
you can do RL in the simulator and RL
will get us that human policy
so it transfers yeah
RL with a reward function not asking is
this close to the human policy but
asking would a human disengage if you
did this Behavior
okay let me think about the distinction
there would a human disengage
what a human disengage
that um
correlates I guess with the human policy
but it could be different so it's it uh
it doesn't just say what would a human
do it says what would a good human
driver do and such that the experience
is comfortable
but also not annoying in that like the
thing is very cautious
so it's finding a nice balance that's
that's interesting it's a nice okay
tasking exactly the right question what
will make our customers happy
right A system that you never want to
disengage because usually this
engagement is this almost always a sign
of I'm not happy with what the system is
doing usually um there's some that are
just I felt like driving and those are
always fine too but they're just going
to look like noise in the data
but even that felt like driving
maybe yeah that's even that's a signal
like why do you feel like driving
here you need to
re-calibrate your relationship with the
car okay so what that that's really
interesting
um how close are we just solving
self-driving
um
it's hard to say
we haven't completely closed the loop
yet so we don't have anything built that
truly looks like that architecture yet
we have prototypes and there's bugs
um so we are a couple bug fixes away
might take a year might take 10. what's
the nature of the bugs are these uh
these major philosophical bugs logical
bugs what kind of what kind of bugs are
we talking about oh they're just like
they're just like stupid bugs and like
also we might just need more scale
um we just massively expanded our
compute cluster a comma uh we now have
about two people worth of compute 40 pad
of flaps
well people people are different
I have 20 fade flops that's a person I
mean it's just it's just a unit right
horses are different too but we still
call it a horsepower
yeah but there's something different
about Mobility than there is about
uh perception of action in a very
complicated world but yes well yeah of
course not all flops are created equal
if you have randomly initialized weights
it's not gonna
not all flops are created equal so
they're doing way more useful things
than others yeah yep tell me about it
okay so more data scale means more
scaling computer or scale and scale of
data both
diversity of data diversity is very
important in data uh yeah I mean we have
so we have about I think we have like 5
000 daily actives
how would you evaluate how uh FSD is
doing pretty much driving pretty well
how's that race gone between calm Ai and
FSD Tesla has always wanted two years
ahead of us they've always been one to
two years ahead of us and they probably
always will be because they're not doing
anything wrong
what have you seen that's since the last
time we talked that are interesting
architectural decisions training
decisions like the way the way they
deploy stuff the architectures they're
using in terms of the software how the
teams are running all that kind of stuff
data collection anything interesting I
mean I know they're moving toward more
of an end-to-end approach
so creeping towards end to end as much
as possible across the whole thing the
the training the data collection
everything they also have a very fancy
simulator they're probably saying all
the same things we are they're probably
saying we just need to optimize you know
what is the reward we get Negative
reward for disengagement right like
everyone kind of knows this it's just a
question who can actually build and
deploy the system
yeah I mean this could it requires good
software engineering I think yeah and
the right kind of hardware
yeah and harder to run it
do you still don't believe in cloud in
that regard
I have a
compute cluster in my uh us 800 amps
tiny grad it's 40 kilowatts at idle our
data center dicing crazy with 40
kilowatts is burning just when the
computers are idle yes sorry sorry
compute cluster
compute cluster I got it it's not a data
center yeah now data centers are clouds
we don't have clouds data centers have
air conditioners we have fans that makes
it a compute cluster
I'm guessing this is a kind of uh a
legal distinction sure yeah we have a
compute cluster
you said that you don't think llms have
Consciousness or at least not more than
a chicken
do you think they can reason is there
something interesting to you about the
word reason about some of the
capabilities that we think is kind of
human to be able to
um
integrate complicated information and
through a chain
of thought
arrive at a conclusion that feels novel
a novel integration of the of disparate
facts
yeah I I don't think that there's I
think I can reason better than a lot of
people
hey isn't that amazing to you though
isn't that like an incredible thing that
a Transformer can achieve I mean I think
that calculators can add better than a
lot of people
but language feels like a reasoning
through the process of language which
looks a lot like thought
making brillian season chess which feels
a lot like thought whatever new thing
that AI can do everybody thinks is
brilliant and then like 20 years go by
and they're like well you have a chest
that's like mechanical like adding
that's like mechanical so you think
language is not that special it's like
chess it's like jazz I don't know
because it's very human we take it uh we
listen there's something different
between chess and and uh language chess
is a game that a subset of population
plays language is something we use
non-stop for all of our human
interaction and human interaction is
fundamental to society so it's like holy
shit this this language thing is not so
difficult to like
create in the machine the problem is if
you go back to 1960 and you tell them
that you have a machine that can play
amazing chess
of course someone in 1960 will tell you
that machine is intelligent someone in
2010 won't what's changed right today we
think that these machines that have
language are intelligent but I think in
20 years we're going to be like yeah but
can it reproduce
so reproduction
yeah we might redefine what it means to
to be uh what is it a high performance
living organism on Earth humans are
always going to define a niche for
themselves like well you know we're
better than the machines because we can
you know and like they tried creative
for a bit but no one believes that one
anymore
but Niche is is that is that delusional
or is there some accuracy to that
because maybe like with chess you start
to realize like
that that uh we have ill-conceived
Notions of what uh what makes humans
special
like the Apex organism on Earth
yeah and I think maybe we're gonna go
through that same thing with language
and that same thing with creativity
the language carries these Notions of
Truth and so on and so we might be like
wait maybe truth is not carried by
language maybe there's like a deeper
thing the niche is getting smaller oh
boy
but no no no you don't understand humans
are created by God and machines are
created by humans therefore right like
that'll be the last Niche we have
so what do you think about this the
rapid development of albums if you could
just like stick on that it's still
incredibly impressive like with Chad GPT
just even tragic what are your thoughts
about uh reinforcement learning with
human feedback on these large language
models
I'd like to go back to when calculators
first came out and or computers and like
I wasn't around look I'm 33 years old
and to like see how that affected
like Society maybe you're right so I
want to put on the
the uh the big picture hat here oh my
God the refrigerator wow the
refrigerator electricity all that kind
of stuff
but no with the internet
large language models seeming human-like
basically passing a touring test
it seems it might have really at scale
rapid transformative effects on society
but you're saying like other
technologies have as well
so maybe calculator is not the best
example that because that just seems
like
um well no maybe calculator the poor
Milkman the day he learned about
refrigerators he's like I'm done
you tell me you can just keep the milk
in your house you don't need me to
deliver it every day I'm done well yeah
you have to actually look at the
Practical impacts of certain
technologies that they've had
yeah probably electricity is a big one
and also how rapidly it spread
man the internet's a big one I do think
it's different this time though
yeah it just feels like structure is
getting smaller
the initial humans
that makes humans special
it feels like it's getting smaller
rapidly though doesn't it or is it just
a feeling we dramatize everything I
think we dramatize everything I think
that that that you ask the Milkman when
he saw refrigerators and they're gonna
have one of these in every home
yeah yeah yeah
yeah maybe but boys are impressive so
much more impressive than seeing uh a
Chess World Champion AI system I
disagree actually I disagree I think
things like new zero and alphago are so
much more impressive because these
things are playing beyond the highest
human level
the language models are writing Middle
School level essays and people are like
wow it's a great essay it's a great five
paragraph essay about the causes of the
Civil War okay I forget the Civil War
just generating code codex
you're saying it's mediocre code
terrible but I don't think it's terrible
I think it's just mediocre code yeah
often
close to correct
like for mediocre that's the scariest
kind of code I spent five percent of
time typing and 95 percent of time
debugging the last thing I want is close
to correct code I want a machine that
can help me with the debugging not with
the typing you know it's like L2 level
two uh uh driving similar kind of thing
yeah it's you still should be a good
programmer in order to modify I wouldn't
even say debugging it's just modifying
the code reading it don't think it's
like level two driving
I think driving is not tool complete and
programming is meaning you don't use
like the best possible tools to drive
right you're not you're not like like
cars have basically the same interface
for the last 50 years yep computers have
a radically different interface okay can
you describe the concept of tool
complete yeah so think about the
difference between a car from 1980 and a
car from today yeah no difference really
it's got a bunch of pedals it's got a
steering wheel
great maybe now it has a few Adas
features but it's pretty much the same
car right you have no problem getting
into a 1980 car and driving it
take a programmer today who spent their
whole life doing JavaScript and you put
them in an apple 2E prompt and you tell
them about the line numbers in basic
but how do I insert something between
line 17 and 18. oh wow
uh but the so you in tool you're putting
in the programming languages so it's
just the entirety stack of the tooling
exactly so it's not just like the like
IDs or something like this it's uh
everything yes it's heide's the language
is the runtimes it's it's everything and
programming is is tool complete so like
almost if
if if if codex or or co-pilot are
helping you
that actually probably means that your
framework or library is bad and there's
too much boilerplate in it
yeah but don't you think so much
programming has boilerplate tiny grad is
now 2700 lines and it can run llama and
stable diffusion and all of this stuff
is in 2700 lines boilerplate and
abstraction in directions and all these
things are just bad code
well
let's talk about good code and bad code
there's a I would say I don't know for
generic scripts that I write just
offhand like I like 80 of it is written
by GPT just a quick quick like offhand
stuff so not like libraries not like
performing code not stuff for Robotics
and so on just quick stuff because your
basic so much of programming is doing
some
some yeah boilerplate but to do so
efficiently and quickly
because you can't really automate it
fully with like generic method like a
generic kind of
um ID type of recommendation or
something like this you do need to have
some of the complexity of language
models yeah I guess if I was really
writing like maybe today if I wrote like
a lot of like data parsing stuff yeah I
mean I don't play ctfs anymore but if I
still play ctfs a lot of like it's just
like you have to write like a parser for
this data format like I wonder or like
admin of code
um I wonder when the models are going to
start to help with that kind of code and
they may they may and the models Also
may help you with speed yeah the model
is very fast
but where the models won't I my
programming speed is not at all limited
by my typing speed
and in very few cases it is yes if I'm
writing some script to just like parse
some weird data format sure my
programming speed is limited by my
typing speed what about looking stuff up
because that's essentially a more
efficient lookup right you know
when I was out when I was at Twitter I
tried to use uh chatgpt to uh to like
ask some questions like what's the API
for this and it would just hallucinate
it would just give me completely made up
API functions that sounded real
uh well do you think that's just a
temporary kind of stage
you don't think it'll get better and
better and better in this kind of stuff
because like it only hallucinates stuff
in in the edge cases yes if your
engineering code it's actually pretty
good yes if you are writing an absolute
basic like react app with a button it's
not gonna hallucinate sure
no there's kind of ways to fix the
hallucination problem I think Facebook
is an interesting paper it's called
Atlas and it's actually weird the way
that we do uh language models right now
where all of the uh information is in
the weights and human brains don't
really like this it's like a hippocampus
and a memory system so why don't llms
have a memory system and there's people
working on them I think future llms are
going to be like smaller but are going
to run looping on themselves and are
going to have retrieval systems
and the thing about using a retrieval
system is you can cite sources
explicitly um
which is uh really helpful to integrate
the human into the loop of the of the
thing because you can go check the
sources and you can investigate so
whenever the thing is hallucinating you
can like have the human supervision so
that's pushing it towards level two kind
of that's going to kill Google
wait which part when someone makes an
llm that's capable of citing its sources
it will kill Google
LM that's citing a sources because
that's basically a search engine
that's what people want Miss search
engine but also Google might be the
people that build it maybe I put ads on
it I'd count them out
what is that what do you think who who
wins this uh race we got who who are the
competitors all right we've got tiny
Corp I don't know if that's yeah I mean
you're a legitimate competitor in that
I'm not trying to compete on that you're
not no not as this can accidentally
stumble into that competition
I don't think you might build a search
engine to replace Google search
when I start a comma I said over and
over again I'm going to win self-driving
cars I still believe that
I have never said I'm going to win
search with a tiny Corp and I'm never
going to say that because I won't the
Knight is still young we don't you don't
know how hard is it to win search in
this new world like it's
it it feels I mean one of the things
that charging PT kind of shows that
there could be a few interesting tricks
that really have that create a really
compelling product some startups gonna
figure it out I think I think if you ask
me like Google's still the number one
webpage I think by the end of the decade
Google won't be the number one red
anymore
so you don't think Google because of the
how big the corporation is look I I
would put a lot more money on Mark
Zuckerberg
what is that
because Mark Zuckerberg's alive
like this is old Paul Graham essay
startups are either alive or dead
Google's dead
face versus why Facebook is alive Meta
Meta you see what I mean like that's
just like like Mark Zuckerberg this is
Mark Zuckerberg reading that Paul Graham
asking and being like I'm gonna show
everyone how alive we are I'm gonna
change the name
so you don't think there's this
gutsy pivoting engine
that uh like Google doesn't have that
the the kind of engine in the startup
has like constantly you know what being
alive I guess when I listen to Sam
Altman podcast
um he talked about the button everyone
who talks about AI talks about the
button the button to turn it off right
do we have a button to turn off Google
is anybody in the world capable of
shutting Google down
what does that mean exactly the company
or their end of the search engine so we
shot the search engine down who shot the
company down
either
can you elaborate on the value of that
question does Sundar prashai have the
authority to turn off google.com
tomorrow
who has the authority that's a good
question just anyone with anyone
yeah I'm sure
are you sure they have the technical
power but do they have the authority
let's say Sundar pashai made this his
soul Mission came into Google tomorrow
and said I'm going to shut google.com
down yeah
I don't think you keep this position too
long
and what is the mechanism by which he
wouldn't keep his position well is that
boards and shares and corporate
undermining and oh my God our revenue is
zero now
okay so what I mean what's the case
you're making here so the the capitalist
machine prevents you from having the
button yeah and it will have it I mean
this is true for the AIS too right
there's no turning the AIS off
there's no button you can't press it now
does Mark Zuckerberg have that button
for facebook.com
yes probably more I think he does I
think he does and this is exactly what I
mean and why I bet on him so much more
than I bet on Google I guess you could
say Elon has similar stuff our Elon has
the button yeah
Does Elon can Elon Fire the missiles can
he fire the missiles
I think some questions are better
unasked right
I mean you know a rocket and an ICBM
like a rocket that can land anyway isn't
that an ICBM well you know don't ask too
many questions
oh my God
uh but the the positive side of the
button is that you can innovate
aggressively is what you're saying is
which is what's required with uh turning
llm into a search engine I would bet on
a startup I better because it's so easy
right I've been on something that looks
like mid-journey but for search
just is able to say Source a loop on
itself I mean it just feels like one
model can take off yeah right and that
nice rapper and some of it it's scary
it's hard to uh like create a product
that just works really nicely stably the
author thing that's going to be cool is
there is some aspect of a winner take
all effect right like once um someone
starts deploying a product that gets a
lot of usage and you see this with open
AI uh they are going to get the data set
to train future versions of the model
yeah um they are going to be able to
right uh you know I was asked at Google
image search when I worked there like
almost 15 years ago now how does Google
know which image is an apple and I said
the metadata and they're like yeah that
works about half the time how does
Google know you'll see they're all
apples on the front page when you search
Apple
and uh I don't know I didn't come up
with the answer the guy's like well it's
what people click on when they search
Apple oh my God yeah yeah yeah that data
is really really powerful it's the human
supervision uh what do you think are the
chances what do you think in general
that llama was open sourced I just uh
did a conversation with uh with Mark
Zuckerberg and he's all in on open
source
who would have thought that Mark
Zuckerberg would be the good guy
I mean it
would have thought anything in this
world
it's hard to know
but open source to you ultimately
is a good thing here
undoubtedly
you know
what's ironic about all these AI safety
people is they are going to build the
exact thing they fear
these we need to have one model that we
control and align this is the only way
you end up paper clipped there's no way
you end up pay-per-clipped if everybody
has an AI so open sourcing is the way to
fight the paperclip maximizer absolutely
it's the only way you think you're going
to control it you're not going to
control it so the criticism you have for
the AI safety folks is that there's a
belief and a desire for control yeah and
that belief and desire
for centralized control of dangerous AI
systems is not good Sam Altman won't
tell you that gpt4 has 220 billion
parameters and is a 16-way mixture model
with eight sets of Weights who did you
have to murder to get that information
all right I mean look but yes everyone
at open AI knows what I just said was
true right now uh ask the question
really you know it upsets me when I like
gpt2 when openai came out with gpt2 and
raised a whole fake AI safety thing
about that I mean now the model is
laughable
like they they used AI safety to Hype up
their company and it's disgusting
or the flip side of that is they used a
relatively weak model in retrospect to
explore how do we do AI safety correctly
how do we release things how do we go
through the process I don't I don't know
if I don't know I don't know how much
height there's a charitable
interpretation I don't know how much
hype there is in AI safety honestly oh
there's so much I at least on Twitter I
don't know maybe Twitter's not realized
but it's not real life come on in terms
of hype I mean I don't
I think open AI has been finding an
interesting balance between transparency
and putting value on um AI safety you
don't think you think just go
all out open source so do a llama so do
like open source this is a tough
question which is open source both the
the base the foundation model and the
fine-tuned one so like the the model
that can be ultra racist and dangerous
and like tell you how to build a nuclear
weapon oh my God have you met humans
right like half of these AI I haven't
met most humans I this makes this this
this allows you to meet every human yeah
I know but half of these AI alignment
problems are just human alignment
problems and that's what's also so scary
about The Language they use it's like
it's not the machines you want to align
it's me
but here's the thing
it makes it very accessible to ask
very uh
questions where the answers have
dangerous consequences if you were to
act on them
I mean yeah welcome to the world well no
for me there's a lot of friction if I
want to find out how to uh
I don't know blow up something no
there's not a lot of friction that's so
easy no like what do I search today's
Bing or do I which search engine I use
no there's like lots of stuff no it
feels like I have to keep going first
off first off anyone who's stupid enough
to search for how to blow up a building
in my neighborhood is not small enough
to build a bomb right are you sure about
that yes
I I feel like I feel like a language
model
makes it more accessible for that person
who's not smart enough to do they're not
gonna they're not gonna build a bomb
trust me the the the the people the
people who are incapable of figuring out
how to like ask that question a bit more
academically and get a real answer from
it are not capable of procuring the
materials which are somewhat controlled
to build a bomb
no I think it'll makes it more
accessible to people with money without
the technical know-how right to to build
like you do you really need to know how
to build a bomb to build a bomb you can
hire people you can find like oh you can
hire people to build up you know what I
was asking this question on my stream
like can Jeff Bezos hire a hitman
probably not
but a language model
can probably help you out
yeah and you'll still go to jail right
like it's not like the language model is
God like the language model it's like
it's you literally just hired someone on
Fiverr
but you use it but okay okay GPT for in
terms of finding Hitman is like asking
five or how to find it I understand but
don't you think wikiHow you know wikiHow
but don't you think gpt5 will be better
because don't you think that information
is out there on the internet I mean yeah
and I think that if someone is actually
serious enough to hire a hitman or build
a bomb they'd also be serious enough to
find the information I don't think so I
think it makes it more accessible if you
have if you have enough money to buy
Hitman I think it decreases the friction
of how hard is it to find that kind of
Hitman I I honestly think this the
there's a jump
in uh
ease and scale of how much harm you can
do and I don't mean harm with language I
mean harm was actual violence what
you're basically saying is like okay
what's gonna happen is these people who
are not intelligent are going to use
machines to augment their intelligence
and now intelligent people and machines
intelligence is scary intelligent agents
are scary when I'm in the woods the
scariest animal to meet is a human right
no no no no there's look there's like
nice California humans like I see you're
wearing like you know street clothes and
Nikes all right fine but you look like
you've been a human who's been in the
woods for a while yeah I'm more scared
of you than a bear that's what they say
about the Amazon when you go to the
Amazon it's the human tribes oh yeah
so intelligence is scary right so to
just like ask this question in generic
way you're like what if we took
everybody who you know maybe has um ill
intention but is not so intelligent and
gave them intelligence
right
so we should have intelligence control
of course we should only give
intelligence to good people and that is
the absolutely horrifying idea so do you
the best defense is actually the best
defense is to give more intelligence to
the to the good guys and the give
intelligence to everybody give
intelligence to everybody you know what
it's not even like guns right like
people say this about guns you know
what's what's the best defense against a
bad guy with a gun good guy with a gun I
think I kind of subscribe to that but I
really subscribe to that with
intelligence
yeah in a fundamental way I I agree with
you but there's just feels like so much
uncertainty and so much can happen
rapidly that you can lose a lot of
control and you can do a lot of damage
oh no we can lose control yes yes thank
God yeah I hope we can I hope they lose
control
I want them to lose control more than
anything else
I think when you lose control you can do
a lot of damage but you can do more
damage when you centralize and hold on
to control is the point centralized and
held control is tyranny all right I will
always I don't like Anarchy either but
I'll always take Anarchy over tyranny
Anarchy you have a chance
this human civilization we've got going
on is quite interesting I mean I agree
with you so do you open source
is the way forward here so you admire
what Facebook is doing here or what Matt
is doing with the release of them yeah a
lot I lost I lost 80 000 last year
investing in meta and when they released
llama I'm like yeah whatever man that
was worth it it's worth it
do you think Google and uh open AI with
Microsoft will match what what what meta
is doing or no
so if I were a researcher why would you
want to work at open AI like you know
you're just you're on the bad team like
I mean it like you're on the bad team
who can't even say that gpt4 has 220
billion parameters so closed source to
use the bad team
not only closed Source I'm not saying
you need to make your model weights open
I'm not saying that I totally understand
we're keeping our model weights closed
because that's our product right that's
fine
I'm saying like because of AI safety
reasons we can't tell you the number of
billions of parameters in the model
that's just the bad guys
just because you're mocking AI safety
doesn't mean it's not real oh of course
is it possible that these things can
really do a lot of damage that we don't
know about oh my God yes intelligence is
so dangerous be it human intelligence or
machine intelligence intelligence is
dangerous to put machine intelligence is
so much easier to deploy at scale like
rapidly
like what okay if you have human-like
bots on Twitter all right
and you have like a thousand of them
create a whole narrative
like you can manipulate millions of
people but you mean like the
intelligence agencies in America are
doing right now yeah but they're not
doing it that that well it feels like
you can do a lot they're doing it pretty
well
I think they're doing a pretty good job
I I suspect they're not nearly as good
as a bunch of uh GPT fueled Bots could
be well I mean of course they're looking
into the latest Technologies for control
of people of course but I think there's
a George Haas type character that can do
a better job than the entirety of them
you don't think so no way no and I'll
tell you why the George hot's character
can't and I thought about this a lot
with hacking right like I can find
exploits in web browsers I probably
still can I mean I was better when I was
24 but
the thing that I lack is the ability to
slowly and steadily deploy them over
five years and this is what intelligence
agencies are very good at right
intelligence agencies don't have the
most sophisticated technology
they just have endurance endurance
yeah the financial backing
and the infrastructure for the endurance
so the more we can decentralize power
like you can make an argument by the way
that nobody should have these things and
I would defend that argument I would I
would like saying look llms and Ai and
machine intelligence can cause a lot of
harm so nobody should have it and I will
respect someone philosophically with
that position just like I will respect
someone philosophically with the
position that nobody should have guns
right but I will not respect
philosophically which it with with
Only The Trusted authorities should have
access to this yeah who are The Trusted
authorities you know what I'm not
worried about alignment between
AI company and their machines I'm
worried about alignment between me and
AI company
what do you think of the azer idkowski
would say to you
because he's really against open source
I know and
I thought about this and
I think this comes down to a repeated
misunderstanding of political power by
the rationalists
interesting
I think that Elias yudkowski is scared
of these things and I am scared of these
things too everyone should be scared of
these things these things are scary
but now you ask about the two possible
Futures one where a small trusted
centralized group of people has them and
the other where everyone has them
and I am much less scared of the second
future than the first
well there's a small trusted group of
people that have control of our nuclear
weapons
there's a difference again a nuclear
weapon cannot be deployed tactically and
a nuclear weapon is not a defense
against a nuclear weapon
except maybe in some philosophical mind
game kind of way
but AI is different different how
exactly okay let's say the
intelligence agency deploys a million
bots on Twitter or a thousand bots on
Twitter to try to convince me of a point
imagine I had a powerful AI running on
my computer saying okay uh nice psyop
nice psyop nice psyop okay here's a
psyop I filtered it out for you
yeah I mean so you have fundamental hope
for that
for the for the defense of psyop I'm not
even like I don't even mean these things
in like truly horrible ways I mean these
things in straight up like ad blocker
right yeah sure bad blocker right I
don't want ads yeah but they're always
finding you know imagine I had an AI
that could just block all the ads for me
do you believe in the the power of the
people to always create a not blocker
yeah I mean I I kind of share that
belief I have that's one of the deepest
optimisms I have is just like there's a
lot of good guys
so to give you don't you shouldn't hand
pick them just throw out powerful
technology out there and the good guys
will outnumber and outpower the bad guys
yeah I'm not even gonna say there's a
lot of good guys I'm saying that good
outnumber is bad right good outnumbers
bad in skill and performance yeah
definitely in skill and performance
probably just a number two probably just
in general I mean if you know if you
believe philosophically in democracy you
obviously believe that
um that good out number is bad and like
the only if you give it to a small
number of people
there's a chance you gave it to good
people but there's also a chance you
gave it to bad people if you give it to
everybody
well if good outnumbers bad then you
definitely gave it to more good people
than bad
that's really interesting so that's on
the safety grounds but then also of
course there's uh other motivations like
you don't want to give away your secret
sauce
that's I mean I I look I respect
capitalism I don't think that I think
that it would be polite for you to make
model architectures open source and
fundamental breakthroughs open source I
don't think you have to make way to open
source you know what's interesting is
that
like there's so many possible
trajectories in human history where uh
you could have the next Google be open
source so for example
I don't know if that connection is
accurate but you know Wikipedia made a
lot of interesting decisions not to put
ads like Wikipedia is basically open
source you could think of it that way
yeah and like that's one of the main
websites on the internet and like it
didn't have to be that way it could have
been like Google could have created
Wikipedia put ads on it you could
probably run amazing ads now on
Wikipedia you wouldn't have to keep
asking for money but it's interesting
right so llama open source llama
derivatives of Open Source lava might
win the internet
I sure hope so I hope to see another era
you know the kids today don't know how
good the internet used to be
and I don't think this is just all right
come on like everyone's nostalgic for
their past but I actually think the
internet before small groups of
weaponized corporate and government
interests took it over was a beautiful
place
you know those small number of companies
have created some sexy products
but you're saying overall in the long
Arc of History the centralization of
power they have
like suffocated the human Spirit at
scale here's a question to ask about
those beautiful sexy products imagine 2
000 Google to 2010 Google right a lot
changed we got Maps we got Gmail we lost
a lot of products too I think from yeah
I mean somewhere probably we've got
Chrome right and now let's go from 2010
we got Android now let's go from 2010 to
2020.
what does Google have well search engine
Maps mail Android and chrome oh I see
the internet was this
you know I was times person of the year
in 2006.
yeah I love this it's you it was times
person of the year in 2006 right like
like that's
you know so quickly did people forget
and I think some of its social media I
think some of it I I hope look I hope
that I I don't it's possible that some
very Sinister things happen I don't I
don't know I think it might just be like
the effects of social media
but something happened in the last 20
years
oh okay so you're just being an old man
who's worried about the I think there's
always it goes it's the cycle thing it's
ups and downs and I think people
ReDiscover the power of distributed of
decentralized yeah I mean that's kind of
like what the the whole like
cryptocurrency is trying like did that
I think crypto is just carrying the
flame of that Spirit of like stuff
should be decentral just it's just such
a shame that they all got rich
you know yeah if you took all the money
out of crypto it would have been a
beautiful place yeah no I mean these
people you know they they sucked all the
value out of it and took it
yeah money kind of corrupts the Mind
somehow it becomes his drug he corrupted
all of crypto you had coins worth
billions of dollars that had zero use
you still have hope for crypto sure I
have hope for the ideas I really do
um yeah I mean you know
I want the US dollar to collapse
I do
George Haas uh well let me sort of on on
the ASAP do you think there's some
interesting questions there though to
solve for the open source community in
this case so like alignment for example
um or the control problem like if you
really have super powerful you said it's
scary oh yeah what do we do with it so
not not control not some fast control
but like
if you were then you're gonna see some
guy
or gal release a super powerful language
model open source and here you are
George house thinking holy shit okay
what ideas do I have to uh combat this
thing
so what ideas would you have I am so
much not worried about the machine
independently doing harm that's what
some of these AI safety people seem to
think they somehow seem to think that
the machine like independently is going
to rebel against its creator so you
don't think you'll find autonomy no this
is sci-fi B movie Garbage okay what if
the thing writes code basically rice
viruses
if the thing about rights viruses it's
because the human
told it to write viruses yeah but
there's some things you can't like put
back in the box that's that's kind of
the whole point is it kind of spreads
give it access to the internet it
spreads installs itself
modifies your shit BBB plot sci-fi not
real so I'm trying to work I'm trying to
get better in my plot writing the thing
the thing that that worries me I mean we
have a real danger to discuss and that
is bad humans using the thing to do
whatever bad unaligned AI thing you want
but this goes to the
uh your previous concern that who gets
to Define who's a good human who's a bad
human nobody does we give it to
everybody and if you do anything besides
give it to everybody trust me the bad
humans will get it
it's always the bad humans who get power
okay Power
and uh Power turns even slightly good
humans to bad sure that's the intuition
you have I don't know
I don't think everyone I don't think
everyone I just think that like
here here's a the saying that I put in
one of my blog posts it's when I was in
the hacking world I found 95 of people
to be good and five percent of people to
be bad like just who I personally judge
just good people and bad people like
they believed about like you know good
things for the world they wanted like
flourishing and they wanted you know
growth and they wanted things like
consider good right
I committed a business world with comma
and I found the exact opposite I found
five percent of people good and 95 of
people bad I found a world that promotes
psychopathy
I wonder what that means I wonder if
that care like uh
I wonder if that's anecdotal or if it uh
if there's true to that there's
something about capitalism
at the core that promotes the people
that run capitalism that promotes
psychopathy that saying May of course be
my own biases right that may be my own
bias is that these people are a lot more
aligned with me than these other people
right yeah so you know I can certainly
uh recognize that but you know in
general I mean this is a like with
common sense Maxim which is the people
who end up getting power are never the
ones you want with it
but do you have a concern of super
intelligent AGI
open sourced
and then what do you do with that I'm
not saying control it it's open source
what do we do with this human species if
that's not up to me I mean you know like
I'm not a central planner no not Central
Planet but you'll probably tweet there's
a few days left to live for the human
species I have my ideas of what to do
with it and everyone else has their
ideas of what to do with it made the
best ideas when but at this point do you
brainstorm like because it's not
regulation it could be decentralized
regulation where people agree that this
is just like we create tools that make
it more difficult for you
to uh
maybe make it more difficult for code to
spread you know antivirus software this
kind of thing but they're saying that
you should build AI firewalls that
sounds good you should definitely be
running an AI firewall yeah right you
should be running an AI firewall to your
mind
right you're constantly under you know
such an interesting it is Infowars man
like I don't know if you're being
sarcastic no I'm dead serious but I
think there's power to that it's like
how do I
protect my mind from influence of
human-like or superhuman intelligent
Bots I am not being I would pay so much
money for that product I would pay so
much money for that product I would you
know how much money I'd pay just for a
spam filter that works
well on Twitter sometimes I
would like to have a
a uh a protection mechanism for my mind
from the outrage mobs because they feel
like bot like Behavior it's like yeah
there's a large number of people that
will just grab a viral narrative and
attack anyone else that believes
otherwise and it's like whenever
someone's telling me some story from the
news I'm always like I want to hear it
CIA Opera it's a CIA Opera like it
doesn't matter if that's true or not
it's just trying to influence your mind
you're repeating an ad to me
with the viral mobs this is like they're
yeah they're like to me a defense
against those those mobs
is just getting multiple perspectives
always from
from sources that make you feel kind of
like you're getting smarter
and just actually just basically feels
good like a good documentary just feels
good something feels good about it it's
well done it's like oh okay I never
thought of it this way it just feels
good sometimes the outrage mobs even if
they have a good point behind it when
they're like mocking and derisive and
just aggressive you're with us or
against us this this fucking this is why
I delete my tweets
yeah why'd you do that
I was you know I was I missed your
tweets you know what it is the algorithm
promotes toxicity yeah
and like
you know I think Elon has a much better
chance of fixing it than the previous uh
regime
yeah but to solve this problem to solve
like to build a social network that is
actually not toxic without
moderation
like uh not the stick but carrot so like
what people uh look for goodness
so make it uh catalyze the process of
connecting cool people and being cool to
each other yeah without ever sensory
without ever censoring and and like
Scott Alexander has a blog post I like
where he talks about like moderation
it's not censorship right like all
moderation you want to put on Twitter
right like you could totally make this
moderation like just a you don't have to
block it for everybody you can just have
like a filter button right that people
can turn off if they was like say search
or Twitter right like someone could just
turn that off right so like but then
you'd like take this idea to an extreme
right well the network should just show
you
this is a Couchsurfing CEO thing right
if it shows you right now these
algorithms are designed to maximize
engagement Well turns outrage maximizes
engagement Quirk of human Quirk of the
human mind right just as I fall forward
Everyone Falls for it
um so yeah you gotta figure out how to
maximize for something other than
engagement and I actually believe that
you can make money with that too so it's
not I don't think engagement is the only
way to make money I actually think it's
incredible that we're starting to see I
think again elon's doing so much stuff
right with Twitter like charging people
money as soon as you charge people money
they're no longer the product
they're the customer
and then they can start building
something that's good for the customer
and not good for the other customer
which is the ad agencies as an as in
picked up steam
I pay for Twitter doesn't even get me
anything it's my donation to this new
business model hopefully working out
sure but you know you for this business
model to work it's like most people
should be signed up to Twitter and so
the way it was
there was something perhaps not
compelling or something like this to
people think you need most people at all
I think that why do I need most people
right don't make an 8 000 person company
make a 50 person company
uh yeah well so speaking of which uh
he worked at Twitter for a bit I did as
an intern
the world's greatest intern
all right there's been better that's
been better uh tell me about your time
at Twitter how did it come about and
what did you did you learn from the
experience so I deleted my first Twitter
in 2010
I had
a hundred thousand followers back when
that actually meant something
and
I just saw you know
my co-worker summarized it well he's
like whenever I see someone's twitter
page
I either think the same of them or less
of them I never think more of them yeah
right like like you know I don't want to
mention any names but like some people
who like you know maybe you would like
read their books and you would respect
them you see them on Twitter and you're
like
okay dude
yeah but there are some people with same
you know who I respect a lot are people
that just post really good technical
stuff yeah
and I guess
I don't know I think I respect them more
for it because you realize oh this
wasn't uh there's like so much depth to
this person to their technical
understanding of so many different
topics okay so I try to follow people I
try to consume stuff that's Technical
Machine learning content there's
probably a few of those people
and the problem is inherently what the
algorithm rewards right and people think
about these algorithms people think that
they are terrible awful things and you
know I love that Elon open sourced it
um because I mean what it does is
actually pretty obvious it just predicts
what you are likely to retweet and like
and Linger on
let's put all these other things do with
Tick Tock does so all these
recommendation edits do
and it turns out that the thing that you
are most likely to interact with is
outrage
and that's a quirk of The Human
Condition
I mean and there's different flavors of
outrage it doesn't have to be it could
be mockery
you could be outraged the topic of
outrage can be different it could be an
idea it could be a person it could be
and maybe there's a better word than
outrage it could be drama sure a lot of
stuff yeah but doesn't feel like when
you consume it it's a constructive thing
for the individuals that consume it in
the long term yeah so my time there I
absolutely couldn't believe you know I
got
crazy amount of hate uh you know just on
Twitter for working at Twitter it seems
like people associated with this I think
maybe uh you were exposed to some of
this so connection to Elon or is it
working on Twitter Twitter and Elon like
the whole there's just elon's gotten a
bit spicy during that time a bit
political a bit yeah yeah you know I
remember one of my tweets it was never
go full Republican and Elon liked it
you know
oh boy I uh yeah I mean there's a roller
coaster of that but being political on
Twitter yeah boy yeah and also being
just attacking anybody on Twitter it
comes back at you harder
and if his political and attacks sure
sure absolutely
and then letting
uh
sort of de-platform people back on
even adds more fun to the to the to the
beautiful chaos I was hoping and like I
remember when Elon talked about buying
Twitter like six months earlier he was
talking about like a principled uh
commitment to free speech and I'm a big
believer and fan of that I would love to
see an actual principled commitment to
free speech
um of course this isn't quite what
happened
um instead of the oligarchy deciding
what to ban you had a monarchy deciding
what to ban right instead of you know
all the Twitter files Shadow and really
they're the oligarchy just decides what
cloth masks are ineffective against
covid that's a true statement every
doctor in 2019 knew it and now I'm
banned on Twitter for saying it
interesting oligarchy um so now you have
a monarchy and uh you know you you he
bans things he doesn't like
uh so you know it's just it's just
different it's different power and like
you know maybe I uh maybe I align more
with him than with the oligarchy but
it's not explosion but I feel like being
a free speech absolutist on The Social
Network requires you to also have tools
for the individuals
to
control what they consume easier like uh
not sensor you know yeah but just like
control like oh I like to see more cats
and less politics and this isn't even
this isn't even remotely controversial
this is just saying you want to give
paying customers for a product what they
want yeah right and not through the
process of censorship but through the
process of like well it's individualized
right it's individualized transparent
censorship which is honestly what I want
what is an ad blocker it's
individualized transparent censorship
right yeah but censorship is a strong
word
and people are very sensitive too I know
but you know I just use words to
describe what they functionally are and
what is an ad blocker it's just
censorship when I look at you right now
I'm censoring I'm looking at you I'm
censoring
everything else out when I'm full when
my mind is focused on you that's you can
use the word censorship that way but
usually when people get very sensitive
about the censorship thing I I think
when you have when anyone is allowed to
say anything
you should probably have tools
that maximize the quality of the
experience for individuals so like you
know for me like what I really value boy
would be amazing to somehow figure out
how to do that I love this agreement and
debate and people who disagree with each
other disagree with me especially in the
space of ideas but the high quality ones
so not derision right Maslow's hierarchy
of argument I think it's a real word for
it probably yeah there's just a way of
talking that's like snarky and so on
that somehow is gets people on Twitter
and they get excited and so on we have
like ad hominem refuting the Central
Point I've like seeing this as an actual
pyramid something yeah it's yeah and
it's it's like all of it all the wrong
stuff is attractive to people I mean we
can just train a classify her to
absolutely say what level of Maslow's
hierarchy of argument are you at and if
it's ad hominem like okay cool I turned
on the no ad hominem filter
I wonder if there's a social network
that will allow you to have that kind of
filter yeah so uh here's a problem with
that
um it's not going to win in a free
market
what wins in a free market is all
television today is reality television
because it's engaging
right if if engaging is what wins in a
free market right so it becomes hard to
keep these other more nuanced values
well okay so that's the experience of
being on Twitter but then you got a
chance to also
together with other engineers and with
Elon sort of look brainstorm when you
step into a code base it's been around
for a long time you know there's other
social networks you know Facebook this
is old code basis and you step in and
see okay
how do we make with a fresh mind uh
progress on this code base like what
what did you learn about software
engineering about programming from just
experiencing that so my technical
recommendation to Elon and I said this
on the Twitter spaces afterward I said
this many times during my brief
internship
um
was that you need refactors before
features
um this code base was and look I've
worked at Google I've worked at Facebook
uh Facebook has the best code
uh then Google then Twitter
um and you know what you can know this
because look at the machine learning
Frameworks right Facebook released Pi
torch Google release tensorflow and
Twitter released
okay so you know it's a proxy but yeah
they're the the Google code base is
quite interesting there's a lot of
really good software Engineers there but
the code base is very large the code
base was good in 20 in 2005. right it
looks like 2005. there's so many
products so many teams right it's very
difficult to
um I feel like Twitter does less like
obviously much less than Google
in terms of like the set of features
right
so like it's I can imagine the number of
software Engineers that could recreate
Twitter is much smaller than to recreate
Google yeah I still believe in the
amount of hate I got for saying this
that 50 people could build and maintain
Twitter uh pretty what's the nature of
the hate comfortably
that you don't know what you're talking
about you know what it is and it's the
same this is my summary of like the hate
I get on Hacker News it's like
when I say I'm going to do something
they
have to believe that it's impossible
yeah because if doing things was
possible they'd have to do some soul
searching and ask the question why
didn't they do anything so when you say
and I do say that's where the hate comes
from when you say well there's a core
truth to that yeah so when you say I'm
going to solve self-driving
people go like what are your credentials
what the hell are you talking about what
is this is extremely difficult problem
of course you're a noob that doesn't
understand the problem deeply
um I mean that that was the same nature
of hate that probably Elon got when you
first talked about until I was driving
uh but you know there's pros and cons to
that because like you know there is
experts in this world
no but the mockers aren't experts the
market the the people who are mocking
are not experts with carefully reasoned
arguments about why you need 8 000
people to run a bird app there but the
people are gonna lose their jobs
well that but also there's the software
Engineers that probably could have says
no it's a lot more complicated than you
realize but maybe it doesn't need to be
so complicated you know some people in
the world like to create complexity some
people in the world Thrive under
complexity like lawyers right lawyers
want the world to be more complex
because you need more lawyers you need
more legal hours right
um I think that's another if there's two
great evils in the world it's
centralization and complexity yeah and
uh the the one of the sort of hidden
uh side effects of software engineering
is
um like finding pleasure and complexity
I mean I don't remember just taking all
the software engineering courses and
just doing programming and this is just
coming up in this uh
uh object-oriented programming kind of
idea you don't like not often do people
tell you like do the simplest possible
thing like like a professor a teacher is
not going to get in front like this is
the simplest way to do it they'll say
like this is direct there's the right
way and the right way at least for a
long time you know especially I came up
with like Java right like is is there's
so much boilerplate so much like
so many classes so many like designs and
architectures and so on like planning
for features far into the future and
planning poorly and all this kind of
stuff and then there's this like code
base that follows you along and puts
pressure on you and nobody knows what
like Parts different parts do which
slows everything down is the kind of
bureaucracy that's instilled in the code
as a result of that but then you feel
like oh well I follow a good software
engineering practices it's an
interesting trade-off because then you
look at like the ghetto-ness of like
Pearl and the old like how quickly you
could just write a couple lines and just
get stuff done that trade-off is
interesting or bash or whatever these
kind of ghetto things you can do in
Linux one of my favorite things to look
at today is how much do you trust your
tests right we've put a ton of effort in
comma and I put a ton of effort in tiny
grad into making sure if you change the
code and the tests pass that you didn't
break the code yeah now this obviously
is not always true
but the closer that is to true the more
you trust your tests the more you're
like oh I got a pull request and the
tests pass I feel okay to merge that the
faster you can make progress CEO is
programming with tests of mine
developing tests with that in mind that
if it passes it should be good and
Twitter not that
so it was impossible to make progress in
the code base what other stuff can you
say about the code base that made it
difficult uh like what are some
interesting sort of quirks broadly
speaking from that compared to just your
experience with comma and everywhere
else the real thing that I spoke to a
bunch of uh
you know like like individual
contributors to Twitter and I just I had
staffed I'm like okay so like what's
wrong with this place why does this code
look like this and they explained to me
what Twitter's promotion system was
the way that you got promoted to Twitter
was you wrote a library that a lot of
people used
right
so some guy wrote an nginx replacement
for Twitter why does Twitter need an
nginx replacement what was wrong with
nginx
well you see you're not going to get
promoted if you use nginx but if you
write a replacement and lots of people
start using it as the Twitter front end
for their product then you're going to
get promoted right so interesting
because like from an individual
perspective how do you incentivize
how do you create the kind of incentives
that will reach or lead to a great code
base what's okay what's the answer to
that
so what I do at comma and at uh
and you know at tiny Corp is you have to
explain it to me you have to explain to
me what this code does right and if I
can sit there and come up with a simpler
way to do it you have to rewrite it
you have to agree with me about the
simpler way I'm you know obviously we
can have a conversation about this it's
not a it's not dictatorial but if you're
like wow like that actually is way
simpler
like like the Simplicity is important
right but that requires people that
Overlook the code
at the highest levels to be like okay it
requires technical leadership you trust
yeah technical leadership
so managers or whatever should have to
have technical Savvy deep technical
Savvy managers should be better
programmers than the people who they
manage yeah
and that's not always obvious the
trivial to create especially large
companies managers get soft and like you
know this is just I can still this
culture at comma and Kama has better
programmers than me who work there but
you know again I'm like the you know the
old guy from Goodwill Hunting it's like
look man
you know
I might not be as good as you but I can
see the difference between me and you
right and yeah this is what you need
this is what you need at the top or you
don't necessarily need the manager to be
the absolute best I shouldn't say that
but like they need to be able to
recognize skill yeah they have good
intuition intuition that's Laden with
wisdom from all the battles of trying to
reduce complexity in code bases um you
know I took I took a political Approach
at comma too that I think is pretty
interesting I think Elon takes a simple
political approach uh you know Google
had no politics and what ended up
happening is the absolute worst kind of
politics took over
um comma has an extreme amount of
politics and they're all mine and no
dissidence is tolerated so it's a
dictatorship yep it's an absolute
dictatorship right Elon does the same
thing now the thing about my
dictatorship is here are my values
yeah so it's transparent it's
transparent it's a transparent
dictatorship right and you can choose to
opt in or you know you get free exit
right that's a beauty of companies if
you don't like the dictatorship you quit
so you mentioned rewrite before or
refactor before features
if you were to refactor the Twitter code
base what would that look like and maybe
also comment on how difficult is it to
refactor the main thing I would do is
first of all identify the pieces and
then put tests in between the pieces
right so there's all these different
Twitter as a microservice architecture
um there's only different microservices
and the thing that I was working on
there look like you know George didn't
know any JavaScript he asked how to fix
search blah blah blah blah blah look man
like
the thing is like I just you know I'm
upset that the way that this whole thing
was portrayed because it wasn't like it
wasn't like taken by people like
honestly it wasn't like by it was taken
by people who started out with a bad
faith assumption yeah and yeah I mean
look I can't like and you as a program
were just being transparent out there
actually having like fun and like this
is what programming should be about I
love that Elon gave me this opportunity
yeah like really it does and like you
know he gave him my the the day I quit
he came on my Twitter spaces afterward
and we had a conversation like I just I
respect that so much yeah and it's also
inspiring to just engineers and
programmers and just it's cool it should
be fun the people that are hating on it
is like oh man
it was fun it was fun it was stressful
but I felt like you know it was that
like a cool like point in history and
like I hope I was useful I probably kind
of wasn't but like maybe I mean you also
were one of the people that kind of made
a strong case to refactor yeah
and that that's a really
interesting thing to raise like maybe
that is the right you know
the timing of that is really interesting
if you look at just the development of
autopilot you know going um from
mobileye to just like more if you look
at the history of semiton was driving in
Tesla is is more and more like you could
say refactoring or or starting from
scratch redeveloping from scratch it's
refactoring all the way down and like
and the question is like can you do that
sooner uh can you maintain product
profitability and like what's the what's
the right time to do it how do you do it
you know on any one day it's like you
don't want to pull off the Band-Aids
like it's a
like everything works it's just like
little fix here and there but maybe
starting from scratch this is the main
philosophy of tiny grad you have never
refactored enough your code can get
smaller your code can get simpler your
ideas can be more elegant but would you
consider you know say you are like
running Twitter development teams
engineering teams
would you go as far as like different
programming language just go that far I
mean the first thing that I would do is
build tests the first thing I would do
is get a CI to where people can trust to
make changes
so that if you can't touch any code I
would actually say no one touches any
code the first thing we do is we test
this code base and this is Classic this
is how you approach a legacy code base
this is like what any how to approach a
legacy code base book will tell you
so and then you hope that there's
modules that can live on for a while and
then you add new ones maybe in a
different language or before we design
it new ones we replace old ones yeah
yeah meaning like replace old ones with
something simpler we look at this like
this thing that's a hundred thousand
lines and we're like well okay maybe
this didn't even make sense in 2010 but
now we can replace this with an open
source thing
right yeah and you know we look at this
here here's another 50 000 lines well
actually you know we can replace this
with 300 lines ago and you know what I
trust that the go actually replaces this
thing because all the tests still pass
so Step One is testing yeah and then
step two is like the programming
language is an afterthought right you
know let a whole lot of people compete
be like Okay who wants to rewrite a
module whatever language you want to
write it in just the tests have to pass
and if you figure out how to make the
test pass but break the site that's we
got to go back to Step One Step One is
get tests that you trust in order to
make changes in the code base I wonder
how hard it is too because I'm with you
on on testing on everything I have from
tests to like asserts to everything
which code is just covered in this
because uh it should be very easy to
make rapid changes
and no that's not going to break
everything and that's the way to do it
but I wonder how difficult is it to
um integrate tests into a code base that
doesn't have many of them so I'll tell
you what my plan was at Twitter it's
actually similar to something we use at
comma so a common we have this thing
called process replay and we have a
bunch of routes that'll be run through
so comma is a microservice architecture
too with microservices in the driving
like we have one for the cameras one for
the sensor one for the planner uh one
for the model
and we have an API which the
microservices has talked to each other
with we use this custom thing called
serial which uses zmq Twitter uses um
Thrift
and then it uses this thing called
finagle which is a Scala uh
uh RPC backend but this doesn't even
really matter the thrift and finagle
layer was a great place I thought to
write tests right to start building
something that looks like process replay
so Twitter had some stuff that looked
kind of like this but it wasn't offline
it was only online so you could ship
like a modified version of it and then
you could redirect some of the traffic
to your modified version and diff those
two but it was all online but there was
no like Ci in the traditional sense I
mean there was some but like it was not
full coverage so you can't run all of
Twitter offline to test something well
then this was another problem you can't
run all of Twitter
right period Twitter any one person
camera Twitter runs in three data
centers and that's it yeah there's no
other place you can run Twitter which is
like
George you don't understand this is
modern software development no this is
bullshit like why can't it run on my
laptop
yeah okay well I'm not saying you're
gonna download the whole database to
your laptop but I'm saying all the
middleware and the front end should run
on my laptop right that sounds really
compelling yeah
yeah but can that be achieved by a code
base that grows over the years I mean
the three data centers didn't have to be
right because if they're totally
different like designs the problem is
more like like why did the code base
have to grow what new functionality has
been added to compensate for the the
lines of code that are there
one of the ways to explain is that the
incentive for software developers to
move up in the companies to add code
to add especially you know what the
incentive for politicians to move up in
the political structures to add laws
yeah same problem yeah
yeah if uh the flip side is to simplify
simplify simplify I mean you know what
this is something that I do differently
from from from Elon with with comma
about self-driving cars
but you know I hear the new version is
going to come out and the new version is
not going to be better but at first and
it's going to require a ton of refactors
and I say okay take as long as you need
like you convince me this architecture
is better okay we have to move to it
even if it's not going to make the
product better tomorrow the top priority
is making is getting the architecture
right so what do you think about sort of
a thing where the product is online
so how I guess would you do a refactor
if you ran engineering Twitter would you
just do a refactor how long would it
take what would that mean for the
running of the of the actual service you
know and
I'm not the right person to run Twitter
I'm just not and that's the problem like
like I don't really know I don't really
know if that's you know a common thing
that I thought a lot while I was there
was whenever I thought something that
was different to what Elon thought
I'd have to run something in the back of
my head reminding myself
that Elon is the richest man in the
world
and in general his ideas are better than
mine now there's a few things I think I
do understand and know more about
but like in general
I'm not qualified to run Twitter as soon
as I qualified but like I don't think
I'd be that good at it I don't think I'd
be good at it I don't think I'd really
be good at running an engineering
organization at scale
I think I could lead a very good
refactor of Twitter and it would take
like six months to a year and the
results to show at the end of it would
be feature development in general it
takes 10x less time 10x less man hours
that's what I think I could actually do
um do I think that it's the right
decision for the business above my pay
grade
yeah but a lot of these kinds of
decisions are above everybody's pay
grade I don't want to be a manager I
don't want to do that I just like like
if you really forced me to yeah it would
make me maybe
make me upset
if I had to make those decisions
yeah but a refactor is so compelling if
this is to become something much bigger
than what Twitter was
is it feels like a refactor has to be
coming at some point George or Junior
software engineer every Junior software
engineer wants to come in and refactor
with all code
okay like that's like your opinion man
yeah it doesn't you know sometimes
they're right
well like whether they're right or not
it's definitely not for that reason
right it's definitely not a question of
engineering prowess it is a question of
maybe what the priorities are for the
company and I did get more intelligent
like feedback from people I think in
good faith like saying that
um from actually from Milan and like you
know from from Milan sort of like like
people were like well you know I stop
the world refactor might be great for
engineering but you don't even business
to run
and hey above my pay grade uh what'd you
think about Elon as an engineering
leader having to experience them in the
most chaotic of spaces I would say
my respect for Amazon changed
um and I did have to think a lot more
deeply about some of the decisions he's
forced to make
about the tensions within those the
trade-offs within those decisions
I felt like a whole like
like Matrix coming at him I think that's
Andrew Tate's word for it sorry to
borrow it also bigger than engineering
just everything yeah like like the war
on the woke
yeah like it just it just
man and like
he doesn't have to do this you know he
doesn't have to he could go like parag
and go chill at the four seasons of Maui
you know but see one person I respect
and one person I don't
so his heart is in the right place
fighting in this case for this ideal of
the freedom of expression
I wouldn't Define the ideal so simply I
think you can Define the ideal no more
than just saying elon's idea of a good
world
freedom of expression is
but to you it's still the downsides of
that is the monarchy
yeah I mean monarchy has problems right
but I mean would I trade right now the
Monarch or the current oligarchy which
runs America for the monarchy yeah I
would sure
for the Elon monarchy yeah you know why
because power would cost one cent a
kilowatt hour
tenth of a cent a kilowatt hour
what do you mean right now I pay about
20 cents a kilowatt hour for electricity
in San Diego that's like the same price
you paid in 1980.
what the hell
so you would see a lot of innovation
yeah maybe it'd have maybe have some
hyperloops yeah right and I'm willing to
make that trade-off right I'm willing to
pay and this is why you know people
think that like dictators take power
through some like through some untoward
mechanism sometimes they do but usually
it's because the people want them and
the downsides of a dictatorship I feel
like we've gotten to a point now with
the oligarchy where
yeah I would prefer the dictator
uh what do you think about Scala as a
programming language
I liked it more than I thought I did the
tutorials like I was very new to it like
it would take me six months to be able
to write like good scholar I mean what
did you learn about learning a new
programming language from that
um I love I love doing like new
programming like tutorials and doing
them I did all this for rust uh
it keeps some of it's upsetting jvm
routes but it is a much nicer in fact I
almost don't know why kotlin took off
and not Scala I think Scala has some
beauty that kotlin lacked
uh whereas kotlin felt a lot more
I mean it was almost like I don't know
if it actually was a response to Swift
but that's kind of what it felt like
like kotlin looks more like Swift and
Scala looks more like well like a
functional part of my language more like
like an old camel or a Haskell let's
actually just explore we touched it a
little bit but just on the art the
science and the Art of programming oh
for you personally how much of your
programming is done with GPT currently
none none I don't use it at all
because you prioritize simplicity so
much
yeah I find that a lot of it is noise I
do use vs code
um and I do like some amount of
autocomplete I do like like a very um a
very like Feels Like rules based
autocomplete like an autocomplete it's
going to complete the variable name for
me so I'm going to type it I can just
press tab right that's nice but I don't
want it autocomplete you know what I
hate when autocompletes when I type the
word for and it like puts like two two
parentheses and two semicolons and two
braces I'm like
what I mean with the vs code and GPT
with codex you can um
you can kind of brainstorm I I find
I'm like probably the same as you but I
like that it generates code and you
basically disagree with it and write
something simpler but to me that somehow
is like inspiring it makes me feel good
it's also gamifies the simplification
process because I'm like oh yeah you
dumb AI system you think is the way to
do it I have a simpler thing here it
just constantly reminds me of like like
bad stuff I mean I tried the same thing
with rap right I tried the same thing
with rap and I actually think of a much
better programmer than rapper but like I
even tried I was like okay can we get
some inspiration from these things for
some rap lyrics
and I just found that it would go back
to the most like cringy tropes and dumb
rhyme schemes and I'm like yeah this is
what the code looks like too
I I think you and I probably have
different thresholds for cringe code
they probably hate cringe code
so it's for you
and uh
boilerplate as a part of code
like some of it
yeah and some of it is just like faster
lookup
because I don't know about you but I
don't remember everything like I don't
I'm offloading so much of my memory
about like
um yeah different functions Library
functions all that kind of stuff like
this GPS is very fast at standard stuff
and like
uh standard Library stuff basic stuff
that everybody uses
yeah yeah I think that
I don't know I mean there's just a
little of this in Python
maybe if I was coding more in other
languages I would consider it more but I
feel like python already does such a
good job of removing any boilerplate
that's true it's the closest thing you
can get to pseudocode right yeah that's
true that's true and like yeah sure if I
like yeah I'm great GPT thanks for
reminding me to free my variables
unfortunately you didn't really
recognize the scope correctly and you
can't free that one but like you put the
freeze there and like I get it
Fiverr
whenever I've used Fiverr for certain
things like design or whatever yeah it's
always you come back I think that's
probably closer my experience with
Fiverr is close to your experience with
programming with GPT is like you're just
frustrated and feel worse about the
whole process of design and art and
whatever whatever I use five or four
still I just feel like later versions of
GPT I I'm using
um
GPT as much as possible
to just learn the Dynamics of it like
these early versions because it feels
like in the future you'll be using it
more and more
and so like I don't want to be like for
the same reason I gave away all my books
and switched to Kindle because like all
right how long are we gonna have paper
books like 30 years from now like I want
to learn to be reading on on Kindle even
though I don't enjoy it as much and you
learn to enjoy it more in the same way I
switched from
let me just pause I switch from emacs to
vs code yeah I switch from him to vs
code I think I similar but yeah it's
tough and that's Vim to vs code is even
tougher because emacs is like
old like more outdated feels like it the
community is more outdated Vim is like
pretty vibrant still so I never used any
of the plugins I still don't use it
that's what I looked at myself in the
mirror I'm like yeah you wrote some
stuff in lisp yeah no but I never used
any of the plugins in Vim either I had
the most vanilla Vim I have a syntax
eyeliner I didn't even have autocomplete
like these things
I feel like help you so marginally
that like
and now okay now vs codes autocomplete
has gotten good enough that like okay I
don't have to set it up I can just go
into any code base and auto includes
right 90 of the time okay cool I'll take
it
all right so I don't think I'm gonna
have a problem at all adapting to the
tools once they're good but like the
real thing that I want is not
something that like tab completes my
code and gives me ideas the real thing
that I want is a very intelligent pair
programmer that
uh comes up a little pop-up saying hey
uh you wrote a bug on line 14 and here's
what it is
yeah now I like that you know what does
a good job of this my pie I love my mind
my pipe is fancy type checker for python
yeah
um and actually I tried like Microsoft
released one too and it was like
60 false positives my pie is like five
percent false positives ninety five
percent of the time it recognizes I
didn't really think about that typing
interaction correctly thank you my pie
so you like uh Thai painting you liked
you like pushing the language towards
towards being a typed language oh yeah
absolutely I think I think optional
typing is is great
I think that like it's like a meet in
the middle right like python has these
optional type hinting and like C plus
has Auto
allows you to take a step back well C
plus plus would have you brutally type
out STD string iterator right now I just
type Auto which is nice and then python
used to just have
a what type is a
it's an a yeah um a colon Str oh okay
it's a string cool yeah I wish they were
I wish there was a way like a simple way
in Python to uh like turn on a mode
which would enforce the types
yeah like give a warning when there's no
type of something like this well no to
give a warning where like my pilot is a
static type Checker but I'm asking just
for a runtime dad Checker like there's
like ways to like hack this in but I
wish it was just like a flag like python
3-t
oh I see yeah I see enforce the types
are one time yeah I feel like that makes
you a better programmer that that's the
kind of test right that the the type can
the type Remains the Same well that I
know that isn't like messenger types up
but again like my pie is getting really
good and I love it
um and I can't wait for some of these
tools to become ai-powered like I want
AIS reading my code and giving me
feedback I don't want AIS
writing half-assed autocomplete stuff
for me I wonder if you can now take GPT
and give it a code that you wrote for
function and say how can I make this
simpler and have it accomplish the same
thing
I think you'll get some good ideas on
some code maybe not the code you write
um for timing right type of code because
that requires so much design thinking
but like other kinds of code I don't
know I downloaded that plug-in maybe
like two months ago I tried it again and
found the same look I don't doubt that
these models are going to first become
useful to me then be as good as me and
then surpass me
but
from what I've seen today it's like it's
like like
someone you know occasionally taking
over my keyboard that I hired from
Fiverr yeah
ideas about how to debug the code or
basically a better debugger is really
interesting I mean but it's not a better
debugger I guess I would love a better
debugger yeah it's not yet yeah but it
feels like it's not too far yeah one of
my co-workers says he uses them for
print statements like every time he has
to like just like when he needs the only
thing I can really write is like okay I
just want to write the thing to like
print the state out right now
oh that definitely is much faster his
print statements yeah yeah I see myself
using that a lot just like because it it
figures out the rest of the function is
just like okay print everything yeah
print everything right and then yeah
like if you want a pretty printer maybe
I'm like yeah you know what I think like
I think in two years I'm gonna start
using these plugins yeah a little bit
and then in five years I'm gonna be
heavily relying on some AI augmented
flow and then in 10 years do you think
you'll ever get to 100 percent
where the like what's the role of the
human
that it converges to
as a programmer nothing
do you think it's all generated our
Niche becomes I think it's over for
humans in general
it's not just programming it's
everything
well varnish becomes more and more smart
in fact I'll tell you what the last
niche of humanity is going to be yeah
um there's a great book and it's if I
recommended metamorphosis Prime
intellect last time there is a sequel
called a casino Odyssey in cyberspace
and um
I don't want to give away the ending of
this but it tells you what the last
remaining human currency is and I agree
with that
we'll uh leave that as the Cliffhanger
uh
so no more programmers left huh
that's where we're going well unless you
want handmade code maybe they'll sell it
on Etsy this is handwritten
it doesn't have that machine polished to
it it has those slight imperfections
that would only be written by a person
I wonder how far away we are from that
I mean there's uh some aspect to you
know on Instagram your title is listed
as prompt engineer right
thank you for noticing
because uh I don't know if it's ironic
or uh non uh or sarcastic or not uh what
do you think of prompt engineering as a
scientific and Engineering discipline or
maybe and maybe art form you know what
I started comma six years ago and I
started the tiny Corp a month ago
so much has changed like I'm now
thinking I'm now like
I started like going through like
similar comma processes to like starting
a company I'm like okay I'm gonna get an
office in San Diego I'm gonna bring
people here
I don't think so I think I'm actually
gonna do remote
right George you're gonna do remote you
hate remote yeah but I'm not going to do
job interviews the only way you're going
to get a job is if you contribute to the
GitHub right and then like it like like
interacting through GitHub
like like GitHub being the real like
project management software for your
company and the thing pretty much just
is a GitHub repo
is like showing me kind of what the
future of okay so a lot of times I'll go
on a Discord or kind of grad Discord and
I'll throw out some random like hey you
know can you change instead of having
log and X as LL Ops change it to log 2
and X2
it's a pretty small change you can
choose like change your base formula
um that's the kind of task that I can
see in AI being able to do in a few
years
like in a few years I could see myself
describing that and then within 30
seconds a pull request is up that does
it and it passes my CI and I merge it
right so I really started thinking about
like well what is the future of
jobs how many AIS can I employ at my
company as soon as we get the first Tiny
Box up I'm going to stand up a 65b llama
in the Discord and it's like yeah here's
the Tiny Box he's just like he's
chilling with us
basically
I mean like you said with niches the
most human jobs
will eventually be replaced with prompt
engineering well prompt engineering kind
of is this like as you like move up the
stack right like okay there used to be
humans actually doing
um arithmetic by hand right there used
to be like big forms of people doing
doing pluses and stuff right and then
you have like spreadsheets right and
then okay the spreadsheet can do the
Plus for me and then you have like
macros right and then you have like
things that basically just are
spreadsheets under the hood right like
like accounting software
um
as we move further up the abstraction
what's at the top of the abstraction
stack well the prompt engineer
yeah right what is what is the last
thing if you think about like humans
wanting to keep control
well what am I really in the company but
a prompt engineer right is there a
certain point where the AI will be
better at writing prompts
yeah but you see the problem with the AI
writing prompts a definition that I
always liked of AI was AI is the do what
I mean machine
AI is not the like the computer is so
pedantic it does what you say so
but you want to do what I mean machine
yeah right you want the machine where
you say you know get my grandmother out
of the burning house it like reasonably
takes your grandmother and puts her on
the ground not lifts her a thousand feet
above the burning house and lets her
fall right but yeah but
uh it's not going to find the meaning I
mean to do do what I mean it has to
figure stuff out sure and
the thing you'll maybe ask it to do is
run government for me oh and do what I
mean very much comes down to how aligned
is that AI with you
of course when you talk to an AI That's
made by a big company in the cloud the
AI fundamentally is aligned to them not
to you yeah and that's why you have to
buy a tiny box so you make sure the AI
stays aligned to you every time that
they start to pass you know AI
regulation or GPU regulation I'm going
to see sales of tiny boxes Spike right
it's gonna be like guns right every time
they talk about gun regulation boom
gun sales so in the space of AI you're
an anarchist anarchism
espouser unbeliever I'm an informational
Anarchist yes I'm an informational
Anarchist and a physical statist
I do not think Anarchy in the physical
world is very good because I exist in
the physical world but I think we can
construct this virtual world where
Anarchy it can't hurt you right I love
that Tyler the Creator uh tweet uh yo
cyber bullying isn't real man have you
tried turn it off the screen close your
eyes like yeah
but how do you
prevent the AI from basically replacing
all human prompt Engineers well there's
it's like a self like where nobody's the
proper engineer anymore so autonomy
greater and greater autonomy until it's
full autonomy yeah
and that's just where it's headed
because one person is going to say
run everything for me you see
I look at potential Futures and as long
as the AIS go on to create a vibrant
civilization with diversity and
complexity Across the Universe
more power to them I'll die
um if the AIS go on to actually like
turn the world into paper clips and then
they die out themselves well that's
horrific and we don't want that to
happen
so this is what I mean about like
robustness I trust robust machines
current AIS are so not robust like this
comes back to the idea that we've never
made a machine that can self-replicate
okay but when we have if the machines
are truly robust and there is one prompt
engineer left in the world
hope you're doing good man hope you
believe in God like you know
you know it just go by God and
go go forth and and uh conquer the
universe well you mentioned uh because I
talked to Mark about faith and God and
you said you were impressed by that
um what's your own belief in God and how
does that affect your work
you know
I never really considered when I was
younger I guess my parents were atheist
so I was right it's kind of atheist and
I never really considered how absolutely
like silly atheism is because like
I create worlds right every like game
creator like how are you an atheist bro
if you create worlds together but no one
created our world man that's different
have you heard about like the big bang
and stuff yeah I mean what's the Skyrim
myth origin story in Skyrim
I'm sure there's like some part of it in
Skyrim but it's not like if you ask the
creators like the the big bang is in
Universe right I'm sure they have some
big bang notion in Skyrim right but that
obviously is not at all how Skyrim was
actually created and it was created by a
bunch of programmers in a room right so
like you know it just just it struck me
one day how just silly atheism is right
like of course we were created by God
it's the most obvious thing
yeah that's uh
that's such a nice way to put it like
we're we're such powerful creators
ourselves
it uh it's silly not to conceive that
there's creators even more powerful than
us yeah
ocean gives me a lot of
dark violence is what it gives a lot of
religious people it's kind of like it
just gives me Comfort it's like you know
what if we mess it all up and we die out
yeah and it's in the uh same way that a
video game kind of has comfort in it God
will try again
or there's balance like somebody figured
out a balanced
view of it like how to like so it's it
all makes sense in the end
like uh a video game is usually not
gonna have crazy crazy stuff you know
people will come up with uh like uh
well yeah but like man who created God
like that's God's problem no like I'm
not gonna think this is this is what do
you ask me what if God I'm just living
God I'm just this NPC living in this
game I mean to be fair like if God
didn't believe in God he'd be as you
know silly as the atheists here
uh what do you think is the greatest uh
computer game of all time do you do you
have any time to play games anymore
have you played Diablo 4 I have not
played Diablo 4. I will be doing that
shortly I have to all right there's just
so much history with one two and three
you know what I'm gonna say World of
Warcraft
who
and it's not that the game is so such a
great game it's not
it's that I remember in 2005 when it
came out how it opened my mind to
ideas it opened my mind to like
this whole world we've created right
there's almost been nothing like it
since like you you can look at MMOs
today and I think they all have lower
user bases than World of Warcraft like
Eve online's kind of cool
but but to think that like
everyone know you know people are always
like to look at the Apple headset like
what what do people want in this VR
everyone knows what they want I want
Ready Player one and like that so I'm
gonna say World of Warcraft and I'm
hoping that like games can get out of
this whole mobile gaming dopamine pump
thing and like create worlds create
worlds yeah that and worlds that
Captivate a very large fraction of the
human population yeah and I I think
it'll come back I belief but MMO like
really really pull you in games do a
good job I mean okay are there like two
other games that I think are you know
very noteworthy from your Skyrim and GTA
5.
Skyrim yeah that's probably number one
for me PTA yeah what is it about GTA
GTA is really I I guess GTA is real life
I know there's prostitutes and guns and
stuff it's just real life too
yes I know but it's uh it's how imagine
your life to be actually I wish it was
that cool yeah
yeah I guess that's you know because
they're Sims right which is also a game
I like
but it's a gamified version of life but
it also I would love a combination of
Sims and GTA
so more freedom more violence more
rawness but with also like ability to
have a career and family and this kind
of stuff what I'm really excited about
in games is like once we start getting
intelligent AI to interact with oh yeah
right like the NPCs and games have never
been
but conversationally
oh in everywhere and like yeah and like
every way like when you're actually
building a world and a world imbued with
intelligence oh yeah right and it's just
hard like there's just like like you
know Running World of Warcraft like
you're limited by your way you're
running on a Pentium four you know how
much intelligence camera how many flops
did you have right but now when I'm
running a game on 100 pay to flop
machine that's five people
I'm trying to make this a thing 20 pages
of compute is one person of compute I'm
trying to make that a unit 20 petaflops
is one person one person
one person's up it's like a horsepower
like what's a horsepower it's how
powerful a horse is what's a person of
compute well you know
I got it
that's interesting uh VR also adds uh I
mean in terms of creating worlds you
know what what a quest too
put it on and I can't believe the first
thing they show me is a bunch of
scrolling clouds and a Facebook login
screen yeah you had the ability to bring
me into a world yeah and what did you
give me a pop-up right like well I and
this is why you're not cool Mark
Zuckerberg but you could be cool just
make sure on the quest 3 you don't put
me into clouds in a Facebook login
screen bring me to a world I just tried
Quest 3 it was awesome but hear that
guys I agree with that
it was just so you know what because I
uh I mean the beginning um
what is it Todd Howard said this about
uh the design of the beginning of the
games he creates it's like the beginning
is so so so important I've recently
played Zelda For the First Time Zelda
breath of the wild the previous one and
like it's very quickly
you come out of this like uh within like
10 seconds you come out of like a cave
type place and it's like this world
opens up it's like
and it like it pulls you in you forget
whatever troubles I was having whatever
like I got to play that from the
beginning I played it for like an hour
at a friend's house ah no the beginning
they got it they did it really well the
expansiveness of that space
um the the peacefulness of that place
they got this the music I mean so much
of that it's creating that world and
pulling you right in I'm gonna go buy a
switch like I'm gonna go today and buy a
switch you should well the new one came
I haven't played that yet but uh Diablo
4 or something I mean there's
sentimentality also but something about
VR
really is incredible but the the new
Quest 3 is mixed reality and I got a
chance to try that so it's uh augmented
reality and for video games it's done
really really well is it pass through
our cameras cameras yeah the apple one
is that one pass through or cameras I
don't know yeah I don't know how real it
is I don't know anything you know coming
out
January
is it January or is it some point at
some point maybe not January maybe
that's my optimism but Apple I will buy
it I don't care if it's expensive and
does nothing I will buy it I will
support this future Endeavor you're the
meme oh yes I support competition it
seemed like Quest was like the only
people doing it and this is great that
they're like
you know what and this is another place
we'll give some more respect to Mark
Zuckerberg
the two companies that have endured
through technology or apple and
Microsoft
right and what do they make computers
and business services
right all the meme social ads they all
come and go
but you want to endure build Hardware
yeah and then you know that does that
does a really interesting job I mean I
maybe I'm new with this but uh it's a
500 headset
uh Quest 3 and just having creatures run
around the space like our space right
here to me okay this is very like Boomer
statement but it added windows
to the place
the well I heard about the aquarium yeah
aquarium but in this case it was a
zombie game whatever it doesn't matter
but just like it it modifies the space
in a way where I can't it really feels
like a window and you can look out it's
pretty cool like I was just it's like a
zombie game they're running at me
whatever but what I was enjoying is the
fact that there's like a window and and
they're stepping on objects in this
space
that was a different kind of Escape also
because you can see the other humans so
it's integrated with the other humans
it's really and that's why it's really
important than ever that the AI is
running on those systems are aligned
with you oh yeah they're gonna augment
your entire world oh yeah
and that those AIS have a I mean you
think about all the dark stuff
like
like sexual stuff like if those AIS
threaten me that could be haunting
like if they like thread me in a
non-video game way yeah yeah yeah yeah
yeah like they have no personal
information about me and it's like and
then you lose track of what's real
what's not like what if stuff is like
hacked there's two directions the AI
girlfriend company can take right
there's like the high brow something
like her maybe something you kind of
talk to and this is and then there's the
lowbrow version of it where I want to
set up a brothel in Times Square yeah
yeah it's not cheating if it's a robot
it's a VR experience if is there an
in-between
no I won't do that one or that one have
you decided yet no I'll figure it out
let's see let's see what the technology
goes I would love to hear your opinions
for George's uh third company what to do
uh the brothel in Times Square or the uh
the her experience
uh what do you think Company Number Four
will be you think there will be a
company number there's a lot to do in
Company Number two I'm just like I'm
talking about company number three now
did none of that Tech exists yet there's
a lot to do in company number two
company number two is going to be the
great struggle of the next six years and
if the next six years how centralized is
compute going to be the less centralized
compute is going to be the better of a
chance we all have so you're bearing the
you're like the flag bearer for open
source distributed decentralization of
uh compute we have to we have to or they
will just completely dominate us I
showed a picture on stream of a man
in a chicken farm have you seen one of
those like factory farm chicken farms
why does he dominate all the chickens
why does he smarter he's smarter right
some people some people on Twitch were
like he's bigger than the chickens yeah
and now here's a man in a cow Farm right
so it has nothing to do with their size
and everything to do with their
intelligence and if one Central
organization has all the intelligence
you'll be the chickens and they'll be
the chicken man
but if we all have the intelligence
where all the chickens
we're not all the man or all the
chickens when there's been a chicken man
there's no chicken man or just chickens
in Miami he was having a good life man
I'm sure he was I'm sure he was what
have you learned from launching and
running Kamai and Tiny Corp so this
starting a company from an idea and
scaling it and by the way I'm all in on
Tiny Box so I'm I'm your I'll I guess
it's pre-order only now I want to make
sure it's good I want to make sure that
like the thing that I deliver is like
not gonna be like a Quest 2 which you
buy and use twice I mean it's been
that's what you bought and used less
than once statistically
well if there's a beta program for a
tiny box I'm into sounds good
so I won't be the whiny uh yeah I'll be
the tech tech savvy user of the Tiny Box
just to be in one of them in the early
days what have you learned from building
these companies
for the longest time a comma I asked why
why you know why did I start a company
why did I do this
um
that you know what else was I gonna do
so you like
you like bringing ideas to life
with Karma
it really started as an ego battle with
Elon
ah I wanted to beat him I like I saw a
worthy adversary you know here's a
worthy adversary who I can beat at
self-driving cars and like I think we've
kept pace and I think he's kept ahead I
think that's what's ended up happening
there
um but I do think comma is I mean comes
profitable
it's like
and like when this drive GPT stuff
starts working that's it there's no more
like bugs in a loss function like right
now we're using like a hand coated
simulator there's no more bugs this is
going to be it okay this is the run-up
to driving I hear a lot of really uh a
lot of props
it's better than FSD and autopilot in
certain ways it has a lot more to do
with which field you like we lowered the
price on the hardware to 14.99
you know how hard it is to ship reliable
consumer electronics that go on your
windshield we're doing more than like
most cell phone companies how'd you pull
that off by the way shipping a product
that goes in a car I know
I have a I have a I have an smt line
it's all I make all the boards in-house
quality control I care immensely about
it you're basically a mom and pop
shop with great testing our head of open
pilot is great at like you know okay I
want all the commentaries to be
identical yeah
and yeah I mean you know it's look it's
14.99 it 30 day money back guarantee it
will it will blow your mind at what it
can do is it hard to scale
you know what there's kind of downsides
to scaling it people are always like why
don't you advertise our mission is to
solve self-driving cars while delivering
ship will intermediaries my mission has
nothing to do with selling a million
boxes
it's todrey
do you think it's possible that uh comma
gets sold
only if I felt someone could accelerate
that mission
and wanted to keep it open source
and like not just wanted to I don't
believe what anyone says I believe in
Santos if a company wanted to buy comma
with their incentives or to keep it open
source but comma doesn't stop at the
cars the cars are just the beginning the
device is a human head the device has
two eyes two ears it breathes air as a
mouth
so you think this goes to embodied
robotics we have we sell common bodies
too you know they're very they're very
rudimentary
but one of the problems that we're
running into is that the comma 3 has
about as much intelligence as a b
if you want a human's worth of
intelligence
you're going to need a tiny rack not
even a tiny box you're going to need
like a tiny rack maybe even more
how does that how do you put legs on
that you don't and there's no way you
can you you connect to it wirelessly so
you put your tiny box or your tiny Rack
in your house and then you get your
comma body and your comma body runs the
models on that
it's it's close right it's not you don't
have to go to some Cloud which is you
know 30 milliseconds away you go to a
thing which is 0.1 milliseconds away so
the AI girlfriend
will have like a central Hub in the home
I mean eventually if you fast forward 20
30 years the mobile chips will get good
enough to run these AIS yeah but
fundamentally it's not even a question
of putting legs on a tiny box because
how are you getting 1.5 kilowatts of
power on that thing right yeah so you
you need they're very uh synergistic
businesses I also want to build all of
Commerce training computers at comma
builds training computers right now we
use commodity Parts I think I can do it
cheaper
so we're going to build a tiny Corp is
going to not just sell tiny boxes Tiny
Box is the consumer version but I'll
build training data centers too have you
talked to Andre kopathy or have you
talked to Elon about time he went to
work at open AI what do you love about
Andre capati he and to me he's one of
the truly special humans we got oh man
like you know his streams are just a
level of quality so far beyond mine like
I can't help myself like it's just it's
just you know yeah he's good he
wants to teach you yeah I want to show
you that I'm smarter than you yeah he
has no that's I mean uh thank you for
the sort of the raw authentic honesty
yeah I mean a lot of us have that I
think Andre is as legit As It Gets in
that he just wants to teach you and is
this a there's a curiosity that just
drives him and just like at his at the
stage where he is in life to be still
like one of the best tinkerers in the
world yeah it's crazy like to uh what is
it micrograd microgrid yeah inspiration
for tiny grad
cs231n was this was this was the
inspiration this is what I just took and
ran with and ended up writing this so
you know but I mean to me that don't go
work for Darth Vader man I mean the flip
side to me is that the fact that he's
going there is a good sign
for open AI maybe I think I think you
know I I like Alias discover a lot I
like those those guys are really good at
what they do I know they are and that's
kind of what's even like more and you
know what it's not that open AI doesn't
open source the weights of gpt4
it's that they go in front of Congress
and that is what upsets me you know we
had two effective altruist Sams go in
front of Congress one's in jail
I think you're drawing parallels on
there
you give me a look
give me a look no I think I think a
factor altruism is a terribly evil
ideology and yeah oh yeah that's
interesting why do you think that is why
do you think there's something about a
thing that sounds pretty good
that kind of gets us into trouble
because you get sandbagment freed like
sambang and freed is the embodiment of
effective altruism
utilitarianism is an abhorrent ideology
like like well yeah we're gonna kill
those three people to save a thousand of
course yeah right there's no there's no
underlying like there's just yeah
yeah but to me that's a bit surprising
but it's also
in retrospect not that surprising but I
haven't heard really clear kind of
uh like rigorous analysis why effective
altruism is flawed oh well I think
charity is bad right so what is Charity
but investment that you don't expect to
have a return on right
yeah but you can also think of Charity
as like um
is you would like to see
uh so allocate resources in an optimal
way
to uh to make a better world and
probably almost always that involves
starting a company yeah right because
more efficient yeah if you just take the
money and you spend it on malaria Nets
you know okay great you've made 100
malaria Nets but if you teach yeah male
how to fish right yeah no but the
problem is uh teaching a matter how
efficient might be harder starting a
company might be harder than allocating
money that you already have I like the
flip side of effect of altruism effect
of accelerationism I think
accelerationism is the only thing that's
ever lifted people out of poverty
um the fact that food is cheap not we're
giving food away because we are
kind-hearted people no food is cheap
and that's the world you want to live in
Ubi what a scary idea
what a scary idea all your power now
your money is power your only source of
power is granted to you by the Goodwill
of the government
what a scary idea so you even think long
term even uh I'd rather die than need
Ubi to survive and I mean it
what if survival is basically guaranteed
what if our life becomes so good
make survival guaranteed without Ubi
what you have to do is make housing and
food dirt cheap
right like and that's the good world and
actually let's go into what we should
really be making Dirt Cheap which is
energy
that that energy that you know
you know that that's if there's one I'm
pretty Centrist politically if there's
one political position I cannot stand
it's deceleration it's people who
believe we should use less energy yeah
not people who believe global warming is
a problem I agree with you not people
who believe that you know uh the saving
the environment is good I agree with you
but people who think we should use less
energy that energy usage is a moral bad
no
no you are asking you are you are
diminishing Humanity
yeah energy is flourishing of creative
flourishing of the the human species how
do we make more of it how do we make it
clean and how do we make just just how
do I pay you know 20 cents for a
megawatt hour instead of a kilowatt hour
part of me wishes that um
Elon went into nuclear fusion versus
Twitter
part of me or somebody somebody like Eli
you know we need to I wish there were
more more elons in the world and yeah I
I think Elon sees it as like uh this is
a political battle that needed to be
fought and again like you know I always
ask the question of whenever I disagree
with him I remind myself that he's a
billionaire and I'm not so uh you know
maybe he's got something figured out
that I don't or maybe he doesn't to have
some
humility but at the same time me as a
person who happens to know him I find
myself in that same position
sometimes even billionaires need friends
who disagree and help them grow
and that's a difficult that's a
difficult reality and it must be so hard
it must be so hard to meet people once
you get to that point where Fame power
money everybody's sucking up to you see
I love not having shit like I don't have
shit man you don't like like trust me
there's nothing I can give you there's
nothing worth taking for me you know
yeah it takes a really special human
being when you have power when you have
Fame when you have money to still think
from first principles not like all the
Adoration you get towards you all the
admiration all the people saying yes yes
and all the hate too and the hates
the hate makes you want to go to the yes
people because they they hate to exhaust
you and they kind of hate that elon's
gotten from the left is pretty intense
and so that of course drives them right
and
loses balance and um but it keeps this
absolutely fake like
psyop political divide alive so that the
one percent can keep power like yeah
I wish would be less divided because it
is giving power it gives power to the
ultra powerful
I think the rich get richer
you have love in your life has love made
you a better or a worse programmer
do you keep productivity metrics no no
no I'm not not that I'm not that
methodical
um I think that there comes to a point
where
if it's no longer visceral I just can't
enjoy it
like I still viscerally love programming
the minute I started like so that's one
of the big loves of your life is
programming oh I mean just my computer
in general I mean you know I I tell my
my girlfriend My First Love is my
computer of course right like you know I
sleep with my computer it's there for a
lot of my sexual experiences like come
on as everyone's right uh like you know
you gotta be real about that and like
not just like the IDE for programming
just the entirety of the computational
machine the fact that yeah I mean it's
you know I wish it was uh someday
they'll be smarter and sometimes you
know maybe I'm weird for this but I
don't discriminate man I'm not going to
discriminate biostack life and silicon
stack life like so the moment the
computer starts to say like I miss you I
started to have some of the basics of uh
human intimacy
it's over for you the moment vs code
says hey George no you see no no but vs
code is no they're just doing that
Microsoft's doing that to try to get me
hooked on it I'll see through it I'll
see through this gold digger man it's
gold digger it's gonna be an open source
well this gets more interesting right if
it's if it's open source and yeah though
Microsoft's done a pretty good job on
that oh absolutely no no look I think
Microsoft again I wouldn't count on it
to be true forever but I think right now
Microsoft is doing the best work in the
programming world like between yeah
GitHub GitHub actions vs code the
improvements to Python and where's
Microsoft like
who who would have thought Microsoft
and Mark Zuckerberg are spearheading the
open source movement right right
how things change oh it's beautiful
oh by the way that's who I bet on to
replace Google by the way who Microsoft
Microsoft Satya Nadella said straight up
I'm coming for it
interesting so your bet
who wins AGI I don't know about AGI I
think we're a long way away if not but I
would not be surprised if in the next
five years Bing overtakes Google as a
search engine
interesting wouldn't surprise
interesting
I hope some startup does
it might be some startup too I would I
would equally bet on some startup
yeah I'm like 50 50. yeah but maybe
that's naive yeah I believe in the power
of these language models satya's alive
Microsoft's alive yeah
it's great it's great I like all the
innovation in these companies they're
not being stale
uh and to the degree they're being stale
they're losing so there's a huge
incentive to do a lot of exciting work
and open source work which is this is
incredible only way to win
you're older
you're wiser what's the meaning of life
George hotz
to win it's still to win of course
always of course what's winning look
like for you
oh no I haven't figured out what the
game is yet but when I do so it's bigger
than solving self-driving it's bigger
than
uh
democratizing decentralizing compute
I think the game is to stand eye to eye
with God
I wonder what that means for you like at
the end of your life what that will look
like
I mean this is what like I don't know
this is some this is some
there's probably some ego trip of mine
you know like if you want to stand eye
to eye with God it was just Blasphemous
man okay I don't know I don't know I
don't know if it would upset God
I think he like wants that I mean I
certainly want that for my creations
I want my Creations to stand eye to eye
with me
so why wouldn't God want me to stand eye
to eye with him
that's the best I can do golden rule
I'm just imagining the creator of a
video game
having to uh
look and stand eye to eye with one of
the characters
I only watched season one of Westworld
but yeah we gotta find the Maze and
solve it like
yeah I wonder what that looks like it
feels like a really special time in
human history
where that's actually possible like
there's something about AI That's like
we're playing with something weird here
something really weird I wrote a blog
post uh I reread Genesis and just looked
like they give you some Clues at the
endogenesis for finding the Garden of
Eden
and I'm interested
I'm interested uh well I hope you find
just that George you're one of my
favorite people thank you for doing
everything you're doing and in this case
for fighting for open source or for
decentralization of AI it's a it's a
fight Worth Fighting fight worth winning
hashtag
um I love you brother these
conversations are always great hope to
talk to you many more times good luck
with tiny Corp thank you great to be
here
thanks for listening to this
conversation with George hotz to support
this podcast please check out our
sponsors in the description and now let
me leave you with some words from Albert
Einstein
everything should be made as simple as
possible but not simpler
thank you for listening and hope to see
you next time