Kind: captions
Language: en
what do you think it takes to let's talk
about AGI a little bit what do you think
it takes to build a system of human
level intelligence we talked about
reasoning we talked about long term
memory but in general what does it take
you think well I can't be sure but I
think the deep learning plus may be
another small idea do you think self
play will be involved sort of like
you've spoken about the powerful
mechanism of self play where systems
learn by sort of exploring the world in
a competitive setting against other
entities that are similarly skilled as
them and so incrementally improving this
way do you think self play will be a
component of building an AGI system yeah
so what I would say to build AGI I think
is going to be deep learning plus some
ideas and I think self play will be one
of those ideas I think that that is a
very self play has this amazing property
that it can surprise us in truly novel
ways for example like we I mean pretty
much every self plays system both our
daughter bought I don't know if opening
I had a release about multi-agent where
you had the two little agents were
playing hide and seek
and of course also alpha zero they were
all produced surprising behaviors they
all produce behaviors that we didn't
expect they are creative solutions to
problems and that seems like an
important part of AGI that our systems
don't exhibit routinely right now and so
that's why I like this area
I like this direction because of its
ability to surprises surprises and an
age age a system would surprise is fun
yes but and to be precise not just not
just a random surprise but to find a
surprising solution to a problem it's
also useful right now a lot of the self
play mechanisms have been used in the
game context or at least in simulation
context how much
how much do you have far along the path
to eg I do you think we'll be done in
simulation how much faith promise do you
have in simulation versus having to have
a system that operates in the real world
with whether it's the real world of
digital real world data or real world
like actual physical world of the
robotics I don't think it's an either/or
I think simulation is a tool and it
helps you has certain its strengths and
certain weaknesses and we should use it
yeah but okay
I understand that that's uh that's true
but one of the criticisms of self play
one of the criticisms are reinforcement
learning is one of the the its current
power its current results while amazing
have been demonstrated in a simulated
environments or very constrained
physical environments do you think it's
possible to escape them escape the
simulated environments and be able to
learn in non simulated environments or
do you think it's possible to also just
similarly in the photorealistic and
physics realistic way the real world in
a way that we can solve real problems
with self play in simulation so I think
that transfer from simulation to the
real world is definitely possible and as
what has been exhibited many times in by
many different groups it's been
especially successful in vision also
open AI in the summer has demonstrated a
robot hand which was trained entirely in
simulation in a certain way that allowed
for seem to real transfer to occur
this is Safin for the Rubik's Cube yes
right and I don't know where that was
trained in simulation was trained in
simulation entirely really so what it
wasn't in the physical that the hand
wasn't trained no 100% of the training
was done in simulation and the policy
that was learned in simulation was
trained to be very adaptive so adaptive
that when you transfer it it could very
quickly adapt to the physical to the
physical world so the kind of
perturbations with the
giraffe or whatever the heck it was
those weren't were those part of the
simulation
well the simulation was generally so the
simulation was trained to be robust to
many different things but not the kind
of perturbations we've had in the video
so it's never been trained with the
glove it's never been trained really
there's a stuffed giraffe so in theory
these are novel perturbation correct
it's not a theory in practice and pray
that those are novel perturbation well
that's okay doesn't matter that's a
clean small-scale but clean example the
transfer from the simulated world to the
physical world yeah and I will also say
that I expect the transfer capabilities
of deep learning to increase in general
and the better the transfer capabilities
are the more useful simulation will
become because today and you could take
you could experience something in
simulation and then learn a moral of the
story which you could then carry with
you to the real world right as humans do
all the time and the play computer games
so let me ask sort of an embodied
question staying an AGI for a sec and do
you think a GI system we need to have a
body we need to have some of those human
elements of self awareness consciousness
sort of fear of mortality sort of
self-preservation in the physical space
which comes with having a body I think
having a body will be useful I don't
think it's necessary but I think it's
very useful to have a body for sure
because you can learn a whole new you
can learn things which cannot be learned
without a body but at the same time I
think that you can call if you don't
have a body you could compensate for it
and still succeed you think so yes well
there is evidence for this for example
there are many people who were born deaf
and blind and they were able to
compensate for the lack of modalities
I'm thinking about Helen Keller
specifically so even if you're not able
to physically interact with the world
and if you're not able to I mean I
actually was getting it maybe let me ask
on the more particularly I'm not sure if
it's connected to having a body or not
but the idea of consciousness and a more
constrained version of that is
self-awareness do you think an EGR
system should have consciousness it's
what we can't define Kyle whatever the
heck you think consciousness is yeah
hard question to answer given how hard
is to find do you think it's useful to
think about I mean it's it's definitely
interesting
it's fascinating I think it's definitely
possible that our systems will be
conscious do you think that's an
emergent thing that just comes from do
you think consciousness could emerge
from the representation that's stored
within your networks so like that it
naturally just emerges when you become
more and more you able to represent more
and more of the world well let's say I'd
make the following argument which is
humans are conscious and if you believe
that artificial neural Nets are
sufficiently similar to the brain then
there should at least exist
artificial neural Nets you should be
conscious to you're leaning on that
existence proof pretty heavily okay but
this is that that's that's the best
answer I can give no I I know I know
I know there's still an open question if
there's not some magic in the brain that
were not I mean I don't mean a non
materialistic magic but that that the
brain might be a lot more complicated
and interesting that would give it
credit for if that's the case then it
should show up and at some point at some
point if you'll find out if you can't
continue to make progress it I think it
I think it's unlikely so we talk about
consciousness but let me talk about
another poorly defined concept of
intelligence again we've talked about
reasoning I've talked about memory what
do you think is a good test of
intelligence for you are you impressed
by the test that Alan Turing formulated
with the imitation game of that with
natural language is there something in
your mind that you will be deeply
impressed by if a system was able to do
I mean lots of things there's certain
through certain frontier there is a
certain frontier of capabilities today
yeah and there exist things outside of
the frontier and I would be impressed by
any such thing
for example I would be impressed by deep
learning system which solves a very
pedestrian you know pedestrian tasks
like machine translation or computer
vision tasks or something which never
makes mistake a human wouldn't make
under any circumstances I think that is
something which have not yet been
demonstrated and I would find it very
impressive yes so right now they make
mistakes in different they might be more
accurate than human beings but they
still they make a different set of
mistakes so my my I would guess a lot of
the skepticism that some people have
about deep learning is when they look at
their mistakes and they say well those
mistakes they make no sense like if you
understood the concept you wouldn't make
that mistake us and I think that
changing that would be we would would do
that would that would inspire me that
would be yes this is this this is this
is progress yeah that's a really nice
way to put it but I also just don't like
that human instinct to criticize a model
is not intelligent that's the same
instinct as we do when we criticize any
group of creatures as the other because
it's very possible that GPT two is much
smarter than human beings and many
things definitely true it has a little
more breadth of knowledge yes
breadth of knowledge and even and even
perhaps depth on certain topics it's
kind of hard to judge what depth means
but there's definitely a sense in which
humans don't make mistakes that these
models do yes the same is applied to
autonomous vehicles the same is probably
going to continue being applied to a lot
of artificial intelligence systems we
find this is the annoying this is the
process of in the 21st century the
process of analyzing the progress of AI
is the search for one case where the
system fails in a big way where humans
would not and then many people writing
articles about it and then broadly as a
the public generally gets convinced that
the system is not intelligent and we
like pacify ourselves but I think it's
not intelligent because of this one
anecdotal case and
seems to continue happening yeah I mean
there is truth to that though there is
people although I'm sure that plenty of
people are also extremely impressed by
the system that exists today but I think
this connects to the earlier point we
discussed that it's just confusing to
judge progress in AI yeah and you know
you have a new robot demonstrating
something how impressed should you be
and I think that people will start to be
impressed once AI starts to really move
the needle on the GDP so you're one of
the people that might be able to create
an AGR system here not you but you and
open the AI if if you do create an AI
system and you get the sponge sort of
the evening with it
him/her what would you talk about do you
think the very first time first time
well the first time I'll just oh just
ask all kinds of questions and try to
make it to get it to make a mistake and
ever be amazed that it doesn't make
mistakes and just keep keep asking broad
okay what kind of questions do you think
would they be factual or would they be
personal emotional psychological what do
you think all of the above would you ask
for advice definitely I mean why would I
limit myself talking to a system of this
now again let me emphasize the fact that
you truly are one of the people that
might be in the room where this happens
so let me ask a sort of a profound
question about I just talked to Stalin
his story I've been talking to a lot of
people who are studying power Abraham
Lincoln said nearly all men can stand
adversity but if you want to test a
man's character give him power I would
say the power of the 21st century maybe
a 22nd but hopefully the 21st would be
the creation of an AGI system and the
people who have control direct
possession and control the AGI system so
what do you think
after spending that evening
having a discussion with the AGI system
what do you think you would do well the
ideal world would like to imagine is one
where humanity
I like the board the board members of a
company where they GI is the CEO so it
would be I would like the picture which
I would imagine is you have some kind of
different entities different countries
of cities and the people that live there
vote for what the AGI that represents
them should do and then AJ that
represents them goes and does it I think
a picture like that I find very
appealing and you could have multiple ad
you have an AJ for a city for a country
and it would be it would be trying to in
effect take the democratic process to
the next level and the board can always
fire the CEO essentially press the reset
button say we randomized the parameters
in well let me sort of that's actually
okay that that's a beautiful vision I
think as long as it's possible to press
the reset button do you think will
always be possible to press the reset
button so things that it's definite
definitely really possible to build so
you're talking so the question that I I
really understand from you is will the
real humans or we humans people have
control over the AI systems at the build
yes and my answer is it's definitely
possible to build AI systems which will
want to be controlled by their humans
Wow that's part of their so it's not
that just they can't help but be
controlled but that's that's the they
exist the one of the objectives of their
existence is to be controlled in the
same way that human parents generally
want to help their children they want
their children to succeed it's not a
burden for them they are excited to help
the children to feed them and to dress
them and to take care of them and I
believe
with hyster conviction that the same
will be possible for an AGI it will be
possible to program an AGI to design it
in such a way that it will have a
similar deep drive that it will be
delighted to fulfill and the drive will
be to help humans flourish but let me
take a step back to that moment where
you create the AGI system I think this
is a really crucial moment and between
that moment and the the Democratic board
members with the AGI at the head there
has to be a relinquish enough power so
it's George Washington despite all the
bad things he did one of the big things
he did is you relinquish power he first
of all didn't want to be President and
even when he became president he gave he
didn't keep just serving as most
dictators do for indefinitely do you see
yourself being able to relinquish
control over an AGI system given how
much power you can have over the world
at First Financial just make a lot of
money right and then control by having
possession and say GI system
I find it trivial to do that I'd finally
trivial to little really really this
kind of pot I mean you know there's a
kind of scenario you are describing
sounds terrifying to me that's all I
would absolutely not want to be in that
position do you think you represent the
majority or the minority of people in
the I community
well I mean it's an open question an
important one our most real good is
another way to ask it so I don't know if
most people are good but I think that
when it really counts people can be
better than we think
that's beautifully put yeah are there
specific mechanism you can think of
aligning AIG values to human values is
that do you think about these problems
of continued alignment as we develop the
AI systems yeah definitely in some sense
the kind of question which you are
asking is so
if I were to translate the question to
today's terms yes it would be a question
about how to get an RL agent that's
optimizing a value function which itself
is learned and if you look at humans
humans are like that because the reward
function the value function of humans is
not external it is internal right and
there are definite ideas of how to train
a value function basically an objective
you know an as objective as possible
perception system that will be trained
separately to recognize to internalize
human judgments on different situations
and then that component wouldn't be
integrated as the value as the base
value function for some more more
capable RL system you could imagine a
process like this I'm not saying this is
the process I'm saying this is an
example of the kind of thing you could
do
you