Kind: captions
Language: en
the following is a conversation with
russ tedrick a roboticist
and professor at mit and vice president
of robotics research
at toyota research institute or tri
he works on control of robots in
interesting
complicated underactuated stochastic
difficult to model situations
he's a great teacher and a great person
one of my favorites at mit
we'll get into a lot of topics in this
conversation from his time leading
mit's delta robotics challenge team
to the awesome fact that he often runs
close to a marathon a day
to and from work barefoot
for a world-class roboticist interested
in elegant efficient control
of underactually dynamical systems like
the human body this fact makes russ
one of the most fascinating people i
know
quick summary of the ads three sponsors
magic spoon cereal
better help and expressvpn please
consider supporting this podcast by
going to magicspoon.com
lex and using code lex at checkout going
to betterhelp.com
lex and signing up at expressvpn.com
lexpod click the links in the
description
buy the stuff get the discount it really
is the best way to support this podcast
if you enjoy this thing subscribe on
youtube review it with five stars nappa
podcast
support it on patreon or connect with me
on twitter
at lex friedman as usual i'll do a few
minutes of ads now
and never any ads in the middle that can
break the flow of the conversation
this episode is supported by magic spoon
low carb keto friendly cereal i've been
on a mix of keto or carnivore diet for a
very long time now
that means eating very little carbs
i used to love cereal obviously most
have crazy amounts
of sugar which is terrible for you so i
quit years ago
but magic spoon is a totally new thing
zero sugar
11 grams of protein and only three net
grams of carbs
it tastes delicious it has a bunch of
flavors they're all good
but if you know what's good for you
you'll go with cocoa my favorite flavor
and the flavor of champions click the
magicspoon.com
lex link in the description use code lex
at checkout to get the discount
and to let them know i sent you so buy
all
of their cereal it's delicious and good
for you
you won't regret it the show is also
sponsored by
better help spelled h-e-l-p-help
check it out at betterhelp.com lex
they figure out what you need and match
you with a licensed professional
therapist
in under 48 hours it's not a crisis line
it's not self-help it is professional
counseling done securely
online as you may know i'm a bit from
the david goggins line of creatures
and still have some demons to contend
with usually on long runs or
all-nighters full of self-doubt i think
suffering is essential for creation
but you can suffer beautifully in a way
that doesn't destroy you
for most people i think a good therapist
can help in this
so it's at least worth a try check out
the reviews
they're all good it's easy private
affordable available worldwide
you can communicate by text anytime and
schedule
weekly audio and video sessions check it
out at betterhelp.com
lex this show is also sponsored by
expressvpn get it at expressvpn.com
lexpod to get a discount and to support
this podcast
have you ever watched the office if you
have
you probably know it's based on the uk
series also called
the office not to steer up trouble but i
personally think the british version is
actually more brilliant
than the american one but both are
amazing anyway
there are actually nine other countries
with their own version of the office
you can get access to them with no geo
restriction when you use
expressvpn it lets you control where you
want sites to think you're located
you can choose from nearly 100 different
countries
giving you access to content that isn't
available in your region so again
get it on any device at expressvpn.com
lexbod to get an extra three months free
and to support this podcast and now
here's my conversation with russ tedjerk
what is the most beautiful motion of a
animal or robot
that you've ever seen i think the most
beautiful
motion of a robot has to be the passive
dynamic walkers
i think there's just something
fundamentally beautiful the ones in
particular that steve collins built with
andy rowena
at cornell a 3d walking machine
so it was not confined to a boom or a
plane
that you put it on top of a small ramp
give it a little push
it's powered only by gravity no
controllers no batteries whatsoever
it just falls down the ramp and at the
time
it looked more natural more graceful
more human-like
than any robot we'd seen to date powered
only by gravity
how does it work well okay the simplest
model is kind of like a slinky it's like
an elaborate slinky
one of the simplest models we use to
think about it is actually a rimless
wheel
so imagine taking a bike's bicycle wheel
but take the rim
off so it's now just got a bunch of
spokes if you give that a push
it still wants to roll down the ramp but
every time its
foot its spoke comes around and hits the
ground it loses a little energy
every time it takes a step forward it
gains a little energy
those things can come into perfect
balance and actually they
they want to it's a stable phenomenon if
it's going too slow it'll speed up
if it's going too fast it'll slow down
and it comes into a stable periodic
motion
now you can take that rimless wheel
which doesn't look very much like a
human walking
take all the extra spokes away put a
hinge in the middle
now it's two legs that's called our
compass gate walker
that can still you give it a little push
starts falling down a ramp
looks a little bit more like walking at
least it's a biped
but what steve and andy and ted mcgear
started the whole exercise but what
steve and andy did was they took it to
this
beautiful conclusion where they built
something that had knees
arms a torso the arms swung naturally
uh give it a little push and that looked
like a stroll through the park
how do you design something like that i
mean is that art or science
it's on the boundary i think there's a
science to getting
close to the solution i think there's
certainly art in the way that they
they made a beautiful robot but
but then the finesse because because
this was work
they were working with a system that
wasn't perfectly modeled wasn't
perfectly controlled
there's all these little tricks that you
have to tune the suction cups at the
knees for instance so
they stick but then they release at just
the right time or there's all these
little tricks of the trade
which really are art but it was a point
i mean it made the point
and we were at that time the walking
robot the best walking robot in the
world was honda's asimo
absolutely marvel of modern engineering
it's 90s
this was in 97 when they first released
it sort of announced p2 and then it went
through it was asimo by then
in 2004 um
and it looks like this very cautious
walking
like you're walking on on hot coals or
something like that
i think it gets a bad rap asimo is a
beautiful machine it does walk with its
knees bent
our atlas walking had its knees bent but
actually ezimo was pretty fantastic
but it wasn't energy efficient neither
was atlas
when we worked on atlas none of our
robots have
that have been that complicated have
been very energy efficient
but there was a there's a thing that
happens when you do control
when you try to control a system of that
complexity you try to use your motors
to basically counteract gravity
take whatever the world's doing to you
and push back
erase the dynamics of the world and
impose the dynamics you want because you
can make them simple
and analyzable mathematically simple
and this was a very sort of beautiful
example that you don't have to do that
you can just let go let physics do most
of the work right and you just have to
give it a little bit of energy this one
only walked down a ramp it would never
walk on the flat to walk on the flat you
have to give a little energy at some
point
but maybe instead of trying to take the
forces imparted to you by the world
and replacing them what we should be
doing is letting the world push us
around
and we go with the flow very zen very
zen robot
yeah but okay so that sounds very zen
but you can i can also imagine
how many like failed
versions they had to go through like how
many like
i would say it's probably would you say
it's in the thousands that they've had
to have the system
fall down before they figured out how
they could i don't know if it's
thousands but uh it's a lot it takes
some patience there's no question
so in that sense control might help a
little bit
oh the abs i think everybody even at the
time
said that the answer is to do with that
with control but it was just pointing
out that maybe the way we're doing
control right now
isn't the way we should got it so what
what about on the animal side
the ones that figured out how to move
efficiently is there anything you find
inspiring or beautiful in the movement
of anybody i do have a favorite example
okay so it sort of
goes with the passive walking idea so is
there
you know how energy efficient are
animals okay there's a great series of
experiments
by george lotter at harvard and mike
tranifilo at mit
they were studying fish swimming in a
water tunnel
okay and one of these the type of fish
they were studying
were these rainbow trout because they
there was a
phenomenon well understood that rainbow
trout when they're swimming upstream at
mating season
they kind of hang out behind the rocks
and it looks like i mean that's tiring
work swimming upstream
they're hanging out behind the rocks
maybe there's something energetically
interesting there so they tried to
recreate that
they put in this water tunnel a rock
basically a cylinder
that had the same sort of vortex street
the eddies coming off the back of the
rock that you would see in a stream
and they put a real fish behind this and
watched how it swims
and the amazing thing is that if you
watch from above
what the fish swims when it's not behind
a rock it has a particular gate
you can identify the fish the same way
you look at a human looking walking down
the street you sort of have a sense of
how human walks
the fish has a characteristic gate you
put that fish behind the rock its gate
changes
and what they saw was that it was
actually resonating
and kind of surfing between the vortices
yeah
now here was the experiment that really
was the clincher because there was still
it wasn't clear how much of that was
mechanics of the fish how much of that
is control
the brain so the clincher experiment and
maybe one of my favorites to date
although there are
many good experiments they took
this was now a dead fish um they took a
dead fish
they put a string that went that tied
the mouse of the fish to the rock so it
couldn't go back and get
caught in the grates uh and then they
asked what would that dead fish do
when it was hanging out behind the rock
and so what you'd expect it sort of
flopped around like a dead fish
in the in the vortex wake until
something sort of amazing happens and
this video
is worth putting in
right what happens uh the dead fish
basically starts swimming upstream
right it's completely dead no brain no
motors
no control but it somehow the mechanics
of the fish resonate with the vortex
street
and it starts swimming upstream it's one
of the best examples ever
who do you give credit for that too
is that just evolution constantly just
figuring out by killing a lot of
generations of animals
uh like the most efficient motion is
that uh or maybe the
physics of our world completely like
it's like evolution applied not only to
animals but just the entirety of it
somehow
drives to efficiency like nature likes
efficiency
i don't know if that question even makes
any sense i understand the question
that's reason i mean
do they co-evolve yeah somehow yeah like
i don't know if an environment can
evolve but um
i mean there are experiments that people
do careful experiments that show that
um animals can adapt to unusual
situations and recover efficiency
so there seems like at least in one
direction i think
there is reason to believe that the
animal's motor system
and probably its mechanics adapt
in order to be more efficient but
efficiency isn't the only goal of course
sometimes it's too easy to think about
only efficiency but we have to do a lot
of other things
first not get eaten and then
all other things being equal try to save
energy by the way let's uh
draw a distinction between control and
mechanics like how
how can how would you define each yeah i
mean i think part of the point is that
we shouldn't draw a line as as clearly
as we tend to
but the you know on a robot we have
motors
and we have the links of the robot let's
say
if the motors are turned off the robot
has some passive dynamics
okay gravity does the work you can put
springs i would call that mechanics
right if we have springs and dampers
which our muscles are springs and
dampers and tendons
but then you have something that's doing
active work putting energy in
your motors on the robot the
controller's job is to
send commands to the motor that add new
energy into the system
right so the mechanics and control
interplay somewhere the divide is around
you know did you decide to send some
commands to your motor or did you just
leave the motors off and let them do
their work
would you say is most of nature
on the dynamic side or the control side
so like if you look at biological
systems if
you know we're living in a pandemic now
like do you think a virus is a
do you think it's a dynamic system or um
or is there a lot of control
intelligence i think it's both but i
think we
maybe have underestimated how important
the dynamics are
right um i mean even our bodies
the mechanics of our bodies certainly
with exercise they evolved but
so i actually i lost a finger in early
2000s
and it's my fifth metacarpal
it turns out you use that a lot in ways
you don't expect when you're opening
jars
even when i'm just walking around if i
bump it on something
there's a bone there that was used to
taking
contact my fourth metacarpal wasn't used
to taking contact it used to hurt
it still does a little bit but actually
my bone has remodeled
right over the lat over a couple years
the geometry the mechanics of that bone
change to address the new circumstances
so the idea that somehow
it's only our brain that's adapting or
evolving is not right
maybe sticking on evolution for a bit
because
it's tended to create some interesting
things uh
by peter walking
do you uh why the heck did evolution
give us
i think we're are we the only mammals
that walk on two feet
no i mean there's a bunch of animals
that do it
a bit there's a i think we are the most
successful bypass
i think some uh i think i read somewhere
that
um the reason the
you know evolution made us walk on two
feet is because uh there's an advantage
to being able to carry food back to the
tribe or something like that
so like you can carry it's kind of this
communal
cooperative thing so like to carry stuff
back
to um to a place of shelter and so on to
share with others um do you understand
at all the value
of uh walking on two feet from both a
robotics and a human perspective
yeah there are some great books written
about evolution of
walking evolution of the human body i
think it's
easy though to make bad evolutionary
arguments
sure most of them are probably bad but
what else can we do i mean i think um
a lot of what dominated our evolution
probably was not
the things that worked well sort of in
the steady state um
you know when things are when things are
good but but uh
for instance people talk about what we
should eat now because
our ancestors were meat eaters or or
whatever oh yeah i love that
yeah but probably you know the reason
that
one pre uh pre-homo sapien species
versus another
survived was not because of whether they
ate well
uh when there was lots of food but when
the ice age came
you know probably one of them happened
to be in the wrong place
one of them happened to forage a food
that was okay
even even when the glaciers came or
something like that i mean
there's a million variables that
contributed and we can't
and our actually the amount of
information we're working with and
telling these stories
these evolutionary stories is uh is very
little
so yeah just like you said it seems like
if we
if we study history it seems like
history turns on like these little
events
that uh that otherwise would seem
meaningless
but in the grant like when you in
retrospect
were turning points absolutely and that
that's probably how like somebody got
hit in the head with a rock
because somebody slept with the wrong
person back in the
cave days and somebody get angry and
that turned uh
you know warring tribes combined with
the environment
all those millions of things and the
meat eating
which i get a lot of criticism because i
i don't know um i don't know what your
dietary processes are like but
these days i been eating only meat
which is um there's a large community
people who say yeah probably make
evolutionary arguments and say you do a
great job there's
probably an even larger community of
people including my mom
who says it's deeply unhealthy it's
wrong but i just feel good doing it
but you're right these evolutionary
arguments can be flawed but is there
anything interesting to pull out for um
there's a great book by the way um look
a series of books by nicholas taleb
about fooled by randomness and black
swan um
highly recommend them but yeah they make
the point nicely that
probably it was a few
random events that yes maybe it was
someone getting hit by a rock as you say
uh that said do you think i don't know
how to ask this question or how to talk
about this but there's something elegant
and beautiful about moving on two feet
obviously biased because i'm human but
from a robotics perspective too you work
with robots on two feet
is it um is it all useful to build
robots that are on two feet as opposed
to four
is there something useful about it the
most um i mean the reason i
spent a long time working on bipedal
walking was because it was hard
and it was um it challenged control
theory in ways that i thought were
important
um i wouldn't have
ever tried to convince you that you
should
start a company around bipeds or
something like this
there are people that make pretty
compelling arguments right i think the
most compelling one
is that the world is built for the human
form
and if you want a robot to work in the
world we have today
then you know having a human form is a
pretty good way to go
there there are places that a biped can
go that would be hard for
other form factors to go even natural
places
but um you know at some point in the
long run
we'll be building our environments for
our robots probably and so maybe that
argument falls aside
so you famously run barefoot
do you still run barefoot i still run
barefoot that's so awesome
much to my wife's chagrin
do you want to make an evolutionary
argument for why running barefoot is
advantageous
um what have you learned about um
human and robot movement in general from
running barefoot
human or robot and or well you know it
happened the other way
right so i was studying walking robots
and
i was there's a great conference called
the dynamic walking
conference where it brings together both
the biomechanics community
and the walking robots community and so
i've been going to this for years and
hearing
talks by people who study barefoot
running and other the mechanics of
running
so i i did eventually read born to run
most people read born to run in the
first thing
right the other thing i had going for me
is actually that i
i wouldn't i wasn't a runner before and
i learned to run
after i had learned about barefoot
running i mean started running longer
distances
so i didn't have to unlearn and i'm
definitely
um i'm a big fan of it for me but i'm
not gonna
i tend to not try to convince other
people there's people who run
beautifully
with shoes on and that's good um
but here's why it makes sense for me um
it's all about the long-term game right
so i think it's just too easy to run 10
miles
feel pretty good and then you get home
at night and you realize
uh my knees hurt i did something wrong
right
um if you take your shoes off
then if you hit hard with your foot at
all
um then it hurts you don't like run 10
miles
and then and then realize you've done
something some damage you have immediate
feedback
telling you that you've done something
that's that's maybe sub-optimal and you
change your gait
i mean it's even subconscious if i right
now having run many
miles barefoot if i put a shoe on my
gate changes
in a way that i think is not as good um
so
so it makes me land softer and
i think my my goals for running are to
do it for as long as i can
into old age um not to win any
races and so for me this is a you know
a way to protect myself yeah i think um
first of all i've tried running barefoot
many years ago
uh probably the other way just just just
uh
reading born to run but
just to understand because i felt like i
couldn't
put in the miles that i wanted to and it
feels like
running for me and i think for a lot of
people
was one of those activities that we do
often and never really
try to learn to do correctly like it's
funny there's so many activities
we do every day like brushing our teeth
right i think a lot of us at least me
probably have never deeply studied how
to properly brush my teeth
right or wash as now with a pandemic or
how to properly wash our hands or do it
every day
but we haven't really studied like am i
doing this correctly but running felt
like one of those things
it was absurd not to study how to do
correctly because it's the source of so
much pain
and suffering like i hate running but i
do it
i do it because i hate it but it i feel
good afterwards
but i think it feels like you need to
learn how to do it properly so that's
where barefoot running came in and then
i quickly realized that
my gait was completely wrong i was
taking huge like steps
and landing hard on the heel all those
elements and so yeah from that i
actually learned to take
really small steps look i
already forgot the number but i feel
like it was 180 a minute or something
like that
and i remember i was uh i actually just
took songs that are 180 beats per minute
and then like tried to run at that beat
uh
just to teach myself it took took a long
time and i feel like uh
after a while you learn to run you
adjust it properly
without going all the way to barefoot
but i feel like barefoot is the legit
way to do it
i mean i think a lot of people would be
really curious about it
can you if they're interested in trying
what would you
how would you recommend a start or try
or
explore slowly that's the biggest thing
people do is they
are excellent runners and they're used
to running long distances or running
fast and they take their shoes off and
they
hurt themselves instantly trying to do
something that they were used to doing
i i think i lucked out in the sense that
i i couldn't
run very far when i first started trying
and i run with minimal shoes too i mean
i will
you know bring along a pair of actually
like aqua socks or something like this i
can just
slip on or running sandals i've tried
all of them
what's the difference between a minimal
shoe and nothing at all
what's like feeling wise what does it
feel like
there is i mean i noticed my gate
changing right
so um i mean your your foot
has as many muscles and sensors as your
hand does right
sensors ooh okay and we do amazing
things with our hands
and we stick our foot in a big solid
shoe right so there's
i think you know when you're barefoot
you're
you're just giving yourself more
proprioception and that's why you're
more aware of some of the gait
flaws and stuff like this now you have
less protection too
so um rocks and stuff i mean
yeah so so i think people are who are
afraid of barefoot running
they're worried about getting cuts or
getting stepping on rocks
first of all even if that was a concern
i think those are all like
uh very short-term you know if i get a
scratch or something it'll heal in a
week
if i blow out my knees i'm done running
forever so i will trade the short term
for the long term
anytime but even then you know this
again to my wife's chagrin um your feet
get tough
right and uh uh cows okay yeah i can run
over almost anything now
i mean what uh maybe can you talk about
is there tin like is there tips or
tricks that you have
uh suggestions about like if i wanted to
try it
you know there is a good book actually
uh there's probably more
good books since i read them but uh
ken bob barefoot ken bob saxton um
he's an interesting guy but i think his
book captured
uh the right way to describe running
barefoot running to somebody
better than any other i've seen
so you run pretty good distances
and you bike and is is there um
you know if we talk about bucket list
items is there something crazy on your
bucket list athletically that
you hope to do one day
i mean my commute is already a little
crazy um what are we talking about here
what what uh what distance are we
talking about well i live about 12 miles
from mit but you can find lots of
different ways to get there so i mean
i've run
there for a long many years a bike there
um blaze
yeah but normally i would try to run in
and then bike home
bike in run home but you have run there
and back before sure
barefoot yeah uh yeah or with minimal
shoes or whatever that
12 12 times two yeah okay
it became kind of a game of how can i
get to work i've rollerbladed i've
done all kinds of weird stuff but uh my
favorite one these days is i've been
taking the charles river to work
so i can put in a little row boat
not so far from my house but the charles
river takes a long way
to get to mit so i can spend a long time
getting there
and it's you know it's not about i don't
know it's just about
uh i've had people ask me how can you
justify taking that time
uh but for me it's just a magical time
to think to compress decompress
um you know especially i'll wake up do a
lot of work in the morning and then
i kind of have to just let that settle
before i i'm ready for all my meetings
and then
on the way home it's a great time to
load it sort of let that settle
so you you lead a like a a large
group of people i mean you're
is there days where you're like oh shit
i gotta get to work in an hour
like i i mean uh
is is there is there a tension there
where and like if we look at the grand
scheme of things just like you said long
term
that meeting probably doesn't matter
like you can always say
i'll just i'll run and let the meeting
happen how it happens
like what uh how do you
that zen how do you uh what do you do
with that tension
between the real world saying urgently
you need to be there
this is important everything is melting
down
how we're going to fix this robot
there's this uh
critical meeting and then there's this
the zen beauty of just
running the simplicity of it you along
with nature
what do you do with that i would say i'm
not a fast runner particularly
probably my fastest splits ever was when
i had to get to daycare on time because
they were going to charge me
you know some some dollar per minute
that i was late uh
i've run some fast splits to daycare
but that those times are passed now
i think work you can find a work-life
balance in that way i think you just
have to
i think i am better at work because i
take time to
think on the way in so i plan my day
around it
and i i rarely feel that those are
really in
at odds so what the bucket list item
if we're talking 12 times 2
or approaching a marathon uh
what uh have you run an ultra marathon
before do you do races is there what's
uh to win
i'm not gonna like take a dinghy across
the atlantic or something if that's what
you want but uh uh
but if someone does and wants to write a
book i would totally read it because i'm
a sucker for that kind of thing
no i do have some fun things that i will
try you know i like to
when i travel i almost always bike to
logan airport and fold up a little
folding bike on and then take it with me
and bike to wherever i'm going and
i've it's taken me or i'll take a
stand-up paddleboard these days on the
airplane and then i'll try to paddle
around where i'm going or whatever
and i've done some crazy things but um
but not for the
you know i've i now talk i don't know if
you know who david goggins is by any
chance
not well but yeah but i i talk to him
now every day so he's the person
who made me uh do this stupid challenge
so he he's insane and he does things for
the purpose
in in the best kind of way he does
things
like for the explicit purpose of
suffering
like he picks the thing that like
whatever he thinks he can do he does
more uh so is that do you have
that thing in you or you uh i think it's
become the
opposite it's uh so you're like that
dynamical system that
the walker the efficient uh yeah it's uh
leave no pain right you should end
feeling better than you started okay but
um it's mostly i think and kovit has
tested this because i've lost my commute
i think i'm perfectly happy walking
around uh
around town with my wife and uh kids if
they could get them to go
and it's more about just getting outside
and getting away from the keyboard for
some time just to let things compress
let's go into robotics a little bit what
to use the most beautiful idea in
robotics
whether we're talking about control or
whether we're talking about optimization
the math side of things
or the engineering side of things or the
philosophical side of things
i think i've been lucky to experience
something that
not so many roboticists have experienced
which is to hang out with
some really amazing control theorists
and uh
the clarity of thought that some of the
more mathematical control theory can
bring
to even very complex messy looking
problems
is really it really had a big impact on
me
and and uh i had a day even like
just a couple weeks ago where i had
spent the day on a zoom
robotics conference having great
conversations with lots of people
i felt really good about the ideas that
were flowing and
and the like and then i had a you know
late afternoon meeting with uh one of my
favorite control theorists and
um and we went from these
from these abstract discussions about
maybes and what-ifs and
and what a great idea to these super
precise
statements about systems that aren't
that much
more simple or or abstract than the ones
i care about
deeply and the contrast of that is
um i don't know it really gets me i
think
people underestimate um
maybe the power of clear thinking
and so for instance deep learning
is amazing um
i use it heavily in our work i think
it's changed the world
unquestionable it makes it easy to get
things
to work without thinking as critically
about it so i think one of the
challenges as an educator
is to think about how do we make sure
people get a taste
of the more rigorous thinking that i
think
goes along uh with with some different
approaches
yeah so that's really interesting so
understanding like the fundamentals the
first principles of the of the
the the problem where in this case is
mechanics
like how a thing moves
how thing behaves like all the forces
involved
like really getting a deep understanding
of that i mean from physics the first
principle thing
come from physics and here it's
literally physics
yeah and this applies in deep learning
this applies to um
not just i mean it applies so cleanly in
in robotics but
it also applies to just in any data set
i find this true i mean driving as well
there's a lot of folks in it that work
on autonomous vehicles
that don't study driving
like deeply i i might be coming a little
bit from the psychology side
but i remember i spent
a ridiculous number of hours at lunch
at this like lawn chair and i would sit
somewhere
somewhere on mit's campus there's a few
interesting intersections and we just
watched people cross
so we were studying um pedestrian
behavior
and i felt like as you record a lot of
video to try
and just the computer vision extracts
their movements how they move their head
and so on
but like every time i felt like i didn't
understand
enough i i just i felt like i wasn't
understanding what how are people
signaling to each other
what are they thinking how cognizant
are they of their fear of death
like what we like what's the game what's
the underlying game theory here what are
what are the
the the incentives and then i finally
found a live stream
uh of an intersection that's like high
def that i just i would
watch so i wouldn't have to sit out
there but that's interesting so like
that's tough that's a tough example
because i mean the learning humans are
involved
not just because human but i think um
the learning mantra is the basically the
statistics of the data will tell me
things i need to know right and
you know for the example you gave of all
the nuances of
um you know eye contact or hand gestures
or whatever that are happening
for these subtle interactions between
pedestrians and traffic
right maybe the data will tell us
they'll tell that story
i may be even i uh one level more
meta than than what you're saying
for a particular problem i think it
might be the case that data
should tell us the story but i think
there's a rigorous thinking
that is just an essential skill for a
mathematician or an engineer
that um i just don't want to lose it
yes there are there are certainly super
rigorous um
rigorous control oh sorry machine
learning people
i just think deep learning makes it so
easy
to do some things that um our next
generation
are um not immediately rewarded
for going through some of the more
rigorous approaches and i wonder where
that takes us
i just well i'm actually optimistic
about it i just want to
do my part to try to steer that rigorous
thinking
so there's like two questions i want to
ask
do you have sort of a good example
of rigorous thinking where it's easy to
get lazy
and not do the rigorous thinking and the
other question i have is like do you
have advice
of um how to practice rigorous
thinking and um you know in all the
computer science disciplines that we've
mentioned
yeah i mean there are times where
problems that can be solved with
well-known mature methods
could also be solved with with a deep
learning
approach and
there's an argument that you must use
learning even for the parts we already
think we know because if the human has
touched it
then you've if you've biased the system
and you've
suddenly put a bottleneck in there that
is your own mental model but
something like inverting a matrix you
know i i think we know how to do that
pretty well even if it's a pretty big
matrix and we understand that pretty
well and
you could train a deep network to do it
but you shouldn't probably
so so in that sense rigorous thinking is
uh
understanding the the scope and the
limitations of the mess
of the methods that we have like how to
use the tools
of mathematics properly yeah i think
you know taking a class on analysis
is all i'm sort of arguing is to take
take a chance to stop and
and force yourself to think rigorously
about even
you know the rational numbers or
something you know it doesn't have to be
the end-all problem but that exercise of
clear thinking i think uh
goes a long way and i just want to make
sure we we keep preaching don't lose it
yeah
but do you think uh when you're doing
like rigorous thinking or like maybe
uh trying to write down equations or
sort of explicitly like formally
describe a system
do you think we naturally simplify
things too much
is that a danger you run into like uh
in order to be able to understand
something about the system
mathematically
we uh make it too much of a toy example
but i think that's the good stuff right
um that's how you understand the
fundamentals
i think so i think maybe even that's a
key to intelligence or something but
i mean okay what if newton and galileo
had deep learning
and and they had done a bunch of
experiments
and they told the world here's your
weights of your neural network i've
we've solved the problem
yeah you know where would we be today i
don't i don't think we'd be as far
as we as we are there's something to be
said about having a the simplest
explanation
for a phenomenon so i don't doubt that
we can train neural networks to predict
even
physical you know
f equals m a type equations
but um i maybe
i want another newton to come along
because i think there's more to do in
terms of
coming up with the simple models for
more complicated tasks yeah uh
let's not offend the ai systems from 50
years from
now that are listening to this that are
probably better at
might be better coming up with f equals
m a equations themselves
so sorry i actually think um learning is
probably a route
to achieving this but the representation
matters
right and i think having
a function that takes my inputs to
outputs
that is arbitrarily complex may not be
the end
goal i think there's still you know the
most
simple or parsimonious explanation for
the data
simple doesn't mean low dimensional
that's one thing i think that we've
a lesson that we've learned so you know
a standard way to do
model reduction or system identification
and controls is to
the typical formulation is that you try
to find the minimal state
dimension realization of a system that
hits some error bounds or something like
that and that's maybe not
i think we're we're learning that that
was that the
state dimension is not the right metric
of complexity of complexity but for me i
think a lot about contact
the mechanics of contact the robot hand
is picking up an object or something
and when i write down the equations of
motion for that they're they look
incredibly complex not because
actually not so much because of the
dynamics of the hand when it's moving
but it's just the interactions and when
they turn on and off
right so having a high dimensional you
know but
simple description of what's happening
out here is fine but if when i actually
start touching
i write down a different dynamical
system for every
polygon on my robot hand and every
polygon on the
object whether it's in contact or not
with all the combinatorics that explodes
there
then that's too complex so i need to
somehow summarize that with a
more intuitive physics
way of thinking and yeah i'm very
optimistic that machine learning will
get us there
first of all i mean i'll probably do it
in the introduction but you're
one of the great robotics people at mit
you're a professor at mit
you've teach them a lot of amazing
courses you
run a large group and you have a
important history for mit i think as
being a part of the darpa robotics
challenge
can you maybe first say what is the dark
robotics challenge and then
tell your story around it your journey
with it yeah sure um
so the darpa robotics challenge it came
on the tales of the darpa
grand challenge and darpa urban
challenge which were the
challenges that brought us put a
spotlight on self-driving
cars
guild pratt was at darpa and
pitched a new challenge that involved
disaster response
it didn't explicitly require humanoids
although humanoids came into the picture
this happened shortly after the
fukushima disaster
in japan and our challenge was motivated
roughly by that
because that was a case where if we had
had robots that were ready to be sent in
there's a chance that we could have
averted disaster
and certainly after the um in the
disaster response
there were times we would love we would
have loved to have sent robots in
so in practice what we ended up with was
a
grand challenge a darpa robotics
challenge
where boston dynamics was
was to make humanoid robots people like
me
and the the amazing team at mit
were competing first in a simulation
challenge
to try to be one of the ones that wins
the right to work on
one of the uh the boston dynamics
humanoids in order to compete in
the the final challenge which was a
physical challenge
and at that point it was already so it
was decided as humanoid robots
there were there were two tracks there
you could enter as a hardware team where
you brought your own robot
or you could enter through the virtual
robotics challenge as a software team
that would try to win the right to use
one of the boston dynamics robots which
are called
atlas atlas humanoid robots yeah it was
a 400-pound
marvel but a you know pretty big scary
looking
robot expensive too expensive at the
time yeah
okay so uh i mean how did you feel
at the prospect of this kind of
challenge i mean it seems
you know autonomous vehicles yeah i
guess that sounds hard
but uh not really from a robotics
perspective it's like
didn't they do in the 80s is the kind of
feeling i would have
uh like when you first look at the
problem it's on wheels
but like humanoid robots
that sounds really hard
so what like what are your the
psychologically speaking what were you
feeling excited
scared why the heck did you get yourself
involved in this kind of
messy challenge we didn't really know
for sure
what we were signing up for in the sense
that you could have something that
as it was described in the call for
participation
that could have put a huge emphasis on
the dynamics of walking
and not falling down and walking over
rough terrain or the same description
because the robot had to go into this
disaster area and
turn valves and and pick up a drill
cut the hole through a wall it had to do
some interesting things
the challenge could have really
highlighted perception and
autonomous planning or it ended up
that you know locomoting over a complex
terrain played a pretty big role in the
competition
so and the degree of autonomy wasn't
clear
the decree of autonomy was always a
central part of the discussion
so um what wasn't clear was how we would
be able how far we'd be able to get with
it
so the idea was always that you want
semi-autonomy
that you want the robot to have enough
compute that you can have a degraded
network link to a human and so the same
way you we had degraded networks
at many natural disasters you'd send
your robot in
you'd be able to get a few bits back and
forth but you don't get to have enough
potentially to fully
uh operate the robot in every joint of
the robot
so and then the question was and the
gamesmanship of the
organizers was to figure out what we're
capable of push us as far as we could
so that um it would differentiate the
teams that
put more autonomy on the robot and had a
few clicks and just said go there do
this go there do this versus someone
who's picking
every footstep or something like that so
what were some memories
painful triumphant from the experience
like what was that journey maybe
if you can dig in a little deeper maybe
even on the technical side and the team
side that that whole process of um
from the early idea stages to actually
competing
i mean this was a defining experience
for me i i
it was it came at the right time for me
in my career i had gotten tenure before
i was
do a sabbatical and most people do
something you know
relaxing and restorative for a
sabbatical so you got tenure before the
the before this yeah yeah yeah it was a
good time for me
i had i had we had a bunch of algorithms
that we were very happy with we wanted
to see how far we could push them and
this was a chance to really test our
metal
to do more proper software engineering
the team we all just worked our butts
off
we you know we're in that lab almost all
the time
okay so i mean there were some of course
high highs and low lows
throughout that anytime you're you know
not sleeping and
devoting your life to a 400 pound
humanoid um
i remember actually one funny moment
where we're all super tired and
so atlas had to walk across cinder
blocks that was one of the obstacles
and i remember atlas was powered down
and hanging limp you know on the on its
harness
and the the humans were there like
laying you know picking up and laying
the brick down so that the robot could
walk over it and i thought what is wrong
with this you know we've got a robot
just watching us do all the manual labor
so that it can take its little
um stroll across the train but
i mean even the even the virtual
robotics challenge was was
super nerve-wracking and dramatic i
remember
um so so we were using gazebo as a
simulator
uh on the cloud there was all these
interesting challenges i think
um the investment that that osrs
fc whatever they were called at that
time brian gerkey's team at open source
robotics
um they were pushing on the capabilities
of gazebo in order to scale it to the
complexity
of these challenges so um
you know up to the virtual competition
so the virtual competition
was you will sign on at a certain time
and we'll have a network connection to
another
machine on the cloud that is running the
simulator of your robot
and your controller will run on this
this controller this computer and and
the physics will run on the other and
you have to connect
now um the physics they wanted it to run
at real-time rates
because there was an element of human
interaction um and humans
could if you do want to tell the op it
works way better if it's at
frame rate oh cool but it was very hard
to simulate these
comple these complex scenes at real-time
rate
so right up to like days before the
competition
the the simulator wasn't quite
at real time rate and that was great for
me because my controller was solving a
big pretty big optimization problem
and it wasn't quite at real-time rate so
i was fine i was keeping up with the
simulator we were both running at about
0.7
and i remember getting this email and by
the way the perception
folks on our team hated that that they
knew that if my controller was too slow
the robot was going to fall down and and
you know no matter how good their
perception system was if i can't make my
controller fast
anyways we get this email like three
days before the virtual competition well
you know it's for all the marbles we're
going to either get a humanoid robot
or we're not and we get an email saying
good news
we made the robot does the simulator
faster it's now one point
and uh yeah we're i was just like oh man
what are we going to do here so
yeah that came in late at night for me
um a few days ahead a few days ahead
i went over there was it happened that
frank permentor who's a
a very very sharp he's a he was a
student at the time
working on optimization was he was still
in lab
uh frank we need to make this quadratic
programming solver faster
not like a little faster it's actually
you know um
and we wrote a new solver for that qp
together that night
and you start terrifying so there's a
really hard optimization problem
that you're constantly solving you
didn't make the optimization problem
simpler
you you wrote any solver so um i mean
your observation is almost spot on well
what we
did was what everybody i mean people
know how to do this but we had not yet
done
this idea of warm starting so we are
solving a big optimization problem
at every time step but if you're running
fast enough the optimization problem
you're solving on the last
time step is pretty similar to the
optimization you're going to solve with
the next
we had course had told our commercial
solver to use warm starting
but even the interface to that
commercial solver
was causing us these delays so what we
did
was we basically wrote we called it
fastqp at the time
we wrote a very lightweight very fast
layer
which would basically check if nearby
solutions to the quadratic program
were which were very easily checked uh
could stabilize the robot
and if they couldn't we would fall back
to the solver you couldn't really test
this well right
um or like i mean so we always knew that
if we
fell back if we it got to the point
where if for some reason
things slowed down and we fell back to
the original solver the robot would
actually literally fall down
um so it was it was a harrowing
sort of edge we're ledge we were sort of
on but
i mean actually like the the 400 pound
humor could come crashing to the ground
if you if you
if your solver is not fast enough but
you know that we have lots of good
experiences
so can i ask you a weird question i
i get um about idea of hard work
so um actually people
like students of yours that i've
interacted with
and just and robotics people in general
but they uh they have
moments at moments have worked harder
than
uh most people i know in terms of if you
look at different disciplines of how
hard people work
but they're also like the happiest like
just like
i don't know um it's the same thing with
like running people that push themselves
to like the limit
they all also seem to be like the most
like full of life somehow
uh and i get often criticized like
you're not getting enough sleep what are
you doing to your body
blah blah blah like this kind of stuff
and
i usually just kind of respond like i'm
i'm doing what i
love i'm passionate about i love it i
feel like it's
it's invigorating i actually think i
don't think the lack of sleep
is what hurts you i think what hurts you
is uh stress and lack of doing things
that you're passionate about
but in this world yeah i mean can you
comment about
uh why the heck robotics people
are uh
willing to push themselves to that
degree is there value in that
and why are they so happy i think
i think you got it right i mean i think
the causality is not
that we work hard and i think other
disciplines work very hard too but it's
i don't think it's
that we work hard and therefore we are
happy
i think we found something that we're
truly passionate about
it makes us very happy and then we
get a little involved with it and spend
a lot of time on it um
what a luxury to have something that you
want to spend all your time on
right we could talk about this for many
hours but
maybe if we could pick is there
something on the technical side on the
approach
you took that's interesting that turned
out to be a terrible failure or a
success that you carry into your work
today
about all the different ideas that were
involved
in um making whether in the in the
simulation or in the in the real world
making this semi-autonomous system work
i mean it really did teach me
something fundamental about what it's
going to take to get robustness out of a
system of this complexity
i would say the darpa challenge really
was foundational in my thinking i think
the autonomous driving community thinks
about this i think lots of
people thinking about safety critical
systems that might have machine learning
in the loop
are thinking about these questions for
me the darpa challenge was
the moment where i realized you know
we've spent
every waking minute running this robot
and
again the in for the physical
competition days before the competition
we saw the robot fall down in a way
it had never fallen down before i
thought
you know how could we have found that
you know we only have one robot it's
running almost all the time
we just didn't have enough hours in the
day to test that robot
something has to change right and then i
think that
i mean i would say that the team that
won
was was from kaist was the team that had
two robots and was able to do not only
incredible engineering just absolutely
top-rate engineering but also they were
able to test
at a rate and um discipline that we
didn't keep up with
what does testing look like what are we
talking about here like what's
what's a a loop of test like a
from start to finish what is a loop of
testing yeah i mean i think
there's a whole philosophy to testing
there's the unit tests and you can do
that on a hardware
you can do that in a small piece of code
you write one function you should write
a test that
that checks that function's input
outputs you should also write an
integration test at the other extreme of
of running the whole system together you
know where that
that try to turn on all the different
functions that you've
you think are correct it's much harder
to write the specifications for a system
level test
especially if that system is as
complicated as a humanoid robot
but the philosophy is sort of the same
i'm the real robot
it's it's no different but on a real
robot it's impossible to run the same
experiment twice
so if you if you see a failure
you hope you caught something in the
logs that tell you what happened
but you'd probably never be able to run
exactly that experiment again
and right now
i think our philosophy is just
basically monte carlo estimation is just
run as many experiments as we can maybe
try to set up the environment
to to make the things we are worried
about
happen as often as possible but really
we're relying on
somewhat random search in order to test
maybe that's all we'll ever be able to
but i think uh
you know because there's an argument
that the things that will get you are
the
the things that are really nuanced in
the world and it'd be very hard to for
instance put back in a simulation
yeah the i guess the edge cases
what was the the hardest thing like so
you said walking over
rough terrain like the just taking
footsteps
i mean people there's it's so dramatic
and painful in a certain kind of way to
watch these videos from
the drc of robots falling
yep it's just so heartbreaking i don't
know maybe it's because
for me at least we anthropomorphize the
robot
um of course there's everything funny
for some reason like humans falling is
funny
uh for i don't it's some dark reason i'm
not sure why
it is so but it's also like tragic and
painful
and uh so speaking of which i mean what
what made the robots fall and fail
uh in your view so i can tell you
exactly what happened on our we i
contributed one of those our team
contributed one of those spectacular
falls
every one of those falls the has a
complicated story i mean
one time the power effectively went out
on the robot
because it had been sitting at the door
waiting for a green light to be able to
proceed
and its batteries you know and therefore
it just fell
backwards and smashed its head across
ground and it was hilarious but it
wasn't because of bad software
right um but for ours so the hardest
part of the challenge the hardest task
in my view was getting out of the
polaris
it was actually relatively easy to drive
the polaris we have
can you tell the stars no the story of
the car
[Laughter]
people should watch this video i mean
the the the thing you've come up with is
just brilliant but uh anyway sorry
what's uh
yeah we we kind of joke we call it the
big robot little car problem because
um somehow the race organizers decided
to give us a 400 pound humanoid and they
also provided the vehicle
which was a little polaris and the robot
didn't really fit in the car so
you couldn't drive the car with your
feet under the steering column
we actually had to straddle the the main
column of the uh and have basically one
foot in the passenger seat one foot in
the driver's seat and then drive
with our left hand but the hard part was
we had to then park the car
get out of the car uh it didn't have a
door that was okay but
it's just uh getting up from crouched
from sitting
when you're in this very constrained
environment uh first of all
i remember after watching those videos i
was much more cognizant of
how hard is it it is for me to get in
and out of the car
and out of the car especially like it's
actually a really difficult control
problem
yeah and i i'm very cognizant of it when
i'm like
injured for whatever reason it's really
hard yeah
so so how did you how did you approach
so so we had a
you know you think of um nasa's
operations and they have these
checklists
you know pre-launch checklists and
they're like we weren't far off from
that we had this big checklist and
on the first day of the competition we
were running down our checklist and one
of the things we had to do
we had to turn off the controller the
piece of software that was running
that would drive the left foot of the
robot in order to accelerate on the gas
and then we turned on our balancing
controller and
the nerves jitters of the first day of
the competition someone forgot to check
that box and turn that controller off
so um we used a lot of motion planning
to figure out a a sort of configuration
of the robot that we get up and
and over we relied heavily on our
balancing controller
and and basically there was when the
robot was in one of its most
precarious you know sort of
configurations trying to
sneak its big leg out of the out of the
side
the other controller that thought it was
still driving
told its left foot to go like this and
uh
and that wasn't good um but but it
turned disastrous for us
because um what happened was a little
bit of push here actually
if you we have videos of us you know
running into the robot with a
10-foot pole and it kind of will recover
but this is a case where
there's no space to recover so a lot of
our secondary balancing mechanisms about
like take a step to recover they were
all disabled because we were in the car
and there's no place to step
so we're relying on our just lowest
level reflexes
and even then i think just hitting the
foot on the seat
on the on the floor we probably could
have recovered from it but the thing
that was bad that happened is when we
did that and we
jostled a little bit the tailbone of our
robot
hat was only a little off the seat it
hit the seat
and the other foot came off the ground
just a little bit and nothing in our
plans
had ever told us what to do if your
butt's on the seat
and your feet are in the air feeding air
and then
the thing is once you get off the script
things can go very wrong because even
our state
estimation our system that was trying to
collect all the data from the sensors
and understand
what's happening with the robot it
didn't know about this situation so it
was predicting things that were just
wrong
and then we did a violent shake and
fell off in our uh face first on out of
the robot
but like into the destination
that's true we fell in we got our point
for egress
but so uh is there any hope for that's
interesting is there any hope
for uh atlas to be able to do something
when it's just on its butt
and feet in the air absolutely so you
can
no so that's um that is one of the big
challenges and i think it's still true
um you know boston dynamics and and
um animal and there's this incredible
work on
on legged robots happening around the
world
most of them still are are very good at
the case where you're
making contact with the world at your
feet and they have typically point feet
relatively they're
balls on their feet for instance if that
if those robots get in a situation where
the elbow hits the wall or something
like this that's a pretty different
situation now they have
layers of mechanisms that will make i
think the the more mature solutions have
have ways in which the controller won't
do stupid things
but a human for instance is able to
leverage
incidental contact in order to
accomplish a goal in fact i might if you
push me i might actually
put my hand out and make a new brand new
contact
the feet of the robot are doing this on
quadrupeds but
we mostly in robotics are afraid of
contact on the rest of our body
which is crazy there's this whole field
of
motion planning collision-free motion
planning
and we write very complex algorithms so
that the robot can dance around and make
sure it doesn't touch
the world um so people are just afraid
of contact
because contact is seen as a difficult
it's still a difficult control problem
and sensing problem now you're a serious
person
uh i'm a little bit of an idiot and i'm
going to ask you some dumb questions
uh so i do uh i do martial arts uh
so like jiu jitsu there's wrestled my
whole life
so let me let me ask the question um
you know like whenever people learn that
i do any kind of ai or
like i mention robots and things like
that they say when am i gonna have
robots that um you know that can
win in a wrestling match or in a fight
against a human so we just mentioned
sitting on your butt
if you in the air that's a common
position jiu jitsu when you're on the
ground you're when
you're down opponent um like what
how difficult do you think is the
problem
and when will we have a robot that can
defeat a human in a wrestling match
and we're talking about a lot like if i
don't know if you're familiar with
wrestling but
essentially um not very
it's basically the art of contact
it's like it's because you're you're
you're picking contact points
and then using like leverage like to uh
off balance to to trick people
like you uh make them feel like you're
doing one thing
and then they they change their balance
and then you uh switch what you're doing
and then
results in a throw or whatever so like
it's basically the art of
multiple contacts so awesome that's a
nice description of it
so there's also an opponent in there
right so so if
very dynamic right if you are wrestling
a human and uh are
in a game theoretic situation with a
human that's
still hard
but just to speak to the you know
quickly reasoning about contact
part of it for instance yeah maybe even
throwing the game theory out of it
almost like uh yeah almost like a
non-dynamic opponent
right there's reasons to be optimistic
but i think our
best understanding of those problems are
still pretty hard
um i have been increasingly focused on
manipulation partly where that's a case
where the contact has to be much more
rich
and there are some really impressive
examples of of deep learning
policies controllers that
that can appear to do good things
through contact
we've even got new examples of of
you know deep learning models of
predicting what's going to happen to
objects as they go through contact
but i think the challenge you just
offered there
still eludes us right the ability to
make a decision based on those models
quickly
you know i have to think though it's
hard for humans too when you get that
complicated i think probably
you had maybe a slow-motion version of
where you learn the basic skills
and you've probably gotten better at it
and and
um there's there's much more subtlety
but it might still be hard to actually
you know
really on the fly take a you know model
of your humanoid and figure out how to
how to plan the optimal sequence that
might be a problem we never solve well
the rapid
the i mean one of the most amazing
things to me about
the we could talk about martial arts uh
we could also talk about dancing
it doesn't really matter too human
i think it's the most interesting study
of contact it's not even the dynamic
element of it
it's the like when you get good at it
it's so effortless like i can just
i'm very cognizant of the entirety of
the learning process
being essentially like learning how to
move my body
in a way that i could throw very large
weights around effortlessly
like and and i can feel the learning
like i'm a huge believer in drilling of
techniques
and you can just like feel your i don't
you're not feeling
you're feeling um sorry you're learning
it
intellectually a little bit but a lot of
it is the body
learning it somehow uh like
instinctually and whatever that learning
is
that's really i'm not even sure if
that's
um equivalent to uh like a
deep learning learning a controller i
think it's something more
it feels like there's a lot of
distributed learning going on
yeah i think there's hierarchy and
composition
yeah um probably in the systems that we
don't
capture very well yet uh you have layers
of control systems you have reflexes at
the bottom layer and you have a
you know a system that's capable of
planning a vacation to
some distant country which is probably
you probably don't have a
controller a policy for every possible
destination you'll ever pick right um
but there's something magical in the in
between
and how do you go from these low-level
feedback loops to something that
feels like a pretty complex set of
outcomes
you know my guess is i think i think
there's evidence that you can plan at
some of these levels
right so uh josh tenenbaum just showed
it
in his talk the other day he's got a
game he likes to talk about
i think he calls it the pick 3 game or
something
where he puts a bunch of clutter down in
front of a person
and he says okay pick three objects and
it might be a
telephone or a shoe or a kleenex box or
whatever and apparently you pick three
items and then you pick he says okay
pick the
first one up with your right hand the
second one up with your left hand
now using those objects those now as
tools pick up the third object
right so that's down at the level of
of physics and mechanics and contact
mechanics that
that i think we do learning we do have
policies for we do control for
almost feedback but somehow we're able
to still i mean i've never picked up a
telephone with a shoe and a water bottle
before and somehow
and it takes me a little longer to do
that the first time
but most of the time we can sort of
figure that out so
yeah i think the amazing thing is this
ability to be flexible
with our models plan when we need to
use our well-oiled controllers when we
don't
when we're in familiar territory um
having models i think the the other
thing you just said was something about
i think your awareness of what's
happening is even changing as you as you
get
as you improve your expertise right so
maybe you have a very approximate model
of the mechanics to begin with and as
you gain
expertise you get a more refined version
of that model you're aware of
muscles or balance components that you
just weren't even aware of before so how
do you scaffold that
yeah plus the fear of injury the
ambition of goals of excelling
and uh fear of mortality
let's see what else is in there as the
motivations
uh overinflated ego in the beginning
uh like and then a crash of confidence
in the middle
all of those seem to be essential for
the learning process
and also and if all that's good then
you're probably optimizing energy
efficiency
yeah right so we have to get that right
uh so um
you know there was this idea that you
would have
uh robots play soccer better
than human players by 2050 that was the
goal
uh world basically was the goal to beat
world champion team to become a world
cup
be like a world cup right level team so
are we gonna see
that first or um a
robot if you're familiar there's an
organization called ufc
for mixed martial arts are we going to
see a world cup championship
soccer team out of robots or a ufc
champion
mixed martial artist uh that's a robot
i mean it's very hard to to say one
thing is a harder
one some problems harder than the other
what probably matters is
um who who who started the organization
that that i mean i think robocup has a
pretty serious following and there
is a history now of people playing that
game learning about that game building
robots to play that game building
increasingly more human robots
it's got momentum and so if you want to
uh
to have mixed martial arts compete you
better start your
start your organization now right um
i think almost independent of which
problem is technically harder because
they're
both hard and they're both different
that's a good point i mean
those videos are just hilarious like uh
especially the humanoid robots trying to
um trying to play soccer
i mean they're kind of terrible right
now i mean i guess there is
robo sumo wrestling there's like the
robo one competitions
um where they do have these robots that
go on the table and basically fight so
maybe i'm wrong
maybe first of all do you have a year in
mind for
uh robocup just from a robotics
perspective
it seems like a super exciting
possibility
that um like in the physical
space this is what's interesting i think
the world is captivated
i think it's really exciting it's um
it inspires a huge number of people when
a machine beats a human
at a game that humans are really damn
good at so you're talking about chess
and go
but that's in the in the world of uh
digital i don't think
machines have beat humans at a game in
the physical space yet but that would be
just
you have to make the rules very
carefully right i mean if
if atlas kicked me in the shins i'm down
and uh you know
and and game over so there's you know
it's
it's very subtle on yeah i think that's
fair i think the fighting one is a weird
one yeah because uh
you're talking about a machine that's
much stronger than you but
yeah in terms of soccer basketball all
those kinds of soccer right i mean as
soon as there's contact
or whatever and there's there are some
things that the robot will do better i
think
if you really set yourself up to try to
see
could robots win the game of soccer as
the rules were written
the right thing for the robot to do is
to play very differently than a human
would play it's you're not going to get
you know the perfect
soccer player robot you're going to get
something that
exploits the rules exploits its super
actuators it's super low bandwidth um
you know feedback loops or whatever and
it's going to play the game differently
than you want it to play yeah
um and it i bet there's ways there's i
bet there's loopholes
right we saw that in the in the darpa
challenge
that that it's very hard to write a set
of rules that someone can't find
uh a way to exploit let me ask another
ridiculous question
i promise i think this might be the last
ridiculous question but
i doubt it i i aspire to ask as many
uh ridiculous questions of uh of a
brilliant mit
professor okay uh i don't know if you've
seen the
black mirror it's funny i i
never watched the episode i know when it
happened though
because i gave a talk to some mit
faculty
one day on a unassuming you know monday
or whatever i was telling about the
state of robotics
and i showed some video of from boston
dynamics of the quadruped
spot at the time it was the early
version of spot
and there was a look of horror that went
across the room
and i said what you know i've shown
videos like this a lot of times what
happened and it turns out that
this video had gone yeah this black
mirror episode had changed the way
people watched
um yeah the videos i was putting out the
way they see these kinds of robots so
i talked to so many people who are just
terrified because of that episode
probably of these kinds of robots they
i almost want to say they almost kind of
like enjoy being terrified
i don't even know what it is about human
psychology that kind of
imagine doomsday the destruction of the
universe
or our society and kind of
like enjoy being afraid um i don't want
to simplify it but it feels like
they talk about it so often it almost
there does seem to be an addictive
quality to it um i talked to a guy
that says this a guy named joe rogan
who's kind of the
flag bearer for being terrified of these
robots
uh do you have a two questions one do
you have an understanding of why people
are afraid
of robots and the second question is
uh in black mirror just to tell you the
episode i don't even remember it that
much
anymore but these robots i think they
can shoot like a
pellet or something they basically have
it's basically a spot with a gun
and um how far are we away from
having robots that go rogue like that
you know basically spot
that goes rogue for some reason
and somehow finds a gun right
so i mean i'm i'm not a psychologist
um i think i don't know exactly why
people react the way they do
i think i think we have to be careful
about the way robots influence our
society and the like i think that's
something that's a responsibility that
roboticists
need to embrace i don't think
robots are going to come after me with a
kitchen knife or a pellet gun
right away and i mean if they were
programmed in such a way but
i used to joke with atlas that
all i had to do was run for five minutes
and it's battery would run out but uh
actually they've got a very big battery
in there by the end so it was over an
hour
um i think the fear is a bit cultural
though because i
i mean you notice that like i think
in my age in the u.s we grew up watching
terminator
right if i had grown up at the same time
in japan i probably would have been
watching astro boy
and there's a very different reaction to
robots
in different countries right so i don't
know if it's a human
innate fear of metal marvels or if it's
something that we've done to ourselves
with our sci-fi
yeah the stories we tell ourselves
through uh through movies through
just uh through popular media
but if if i were to tell you know if
if you were my therapist and i said i'm
really terrified
that we're going to have these robots
very soon that will hurt us
like how do you approach
making me feel better like
why shouldn't people be afraid there's a
i think there's a video that went viral
recently everything everything was spot
in boston today which goes viral in
general
but usually it's like really cool stuff
like they're doing flips and stuff or
like sad stuff would
be it's atlas being hit with a
broomstick or something like that
but uh there's a video where i think uh
one of the new production spot robots
which are awesome
it was like patrolling somewhere in like
in some
country and like people immediately were
like
saying like this is like the dystopian
future
like the surveillance state for some
reason like you can just have a camera
like
something about spot being able to walk
on four feet
with like really terrified people so
like what
what do you say to those people
i think there is a legitimate fear there
because
so much of our future is uncertain
but at the same time technically
speaking it seems like we're not there
yet
so what do you say i mean i think
technology
is um complicated it can be used in many
ways i think there are
purely software um
attacks that somebody could use to do
great damage
maybe they have already um you know i
think
uh wheeled robots
could be used in bad ways too
drones right um
i don't think that let's see
i don't want to be um building
technology just
because i'm compelled to build
technology and i don't think about it
but i would consider myself a
technological optimist
i guess um in the sense that
i think we should continue to create and
evolve and our
world will change um and if we will
introduce new challenges we'll screw
something up maybe
but i think also we'll invent ourselves
out of those challenges and life will go
on
so it's interesting because you didn't
mention like this is
technically too hard i don't think
robots are i think people
attribute a robot that looks like an
animal as maybe having a level of
self-awareness
or consciousness or something that they
don't have yet
right so it's not i think
our ability to anthropomorphize those
robots is probably
um we're assuming that they have a level
of intelligence that they don't yet have
and that might be part of the fear so in
that sense
it's too hard but um
you know there are many scary things in
the world right so
i think we're right to ask those
questions we're right to
um think about the implications of our
work
right in the in this in the short term
as we're working on it for sure
is there something long-term that
scares you about our future with ai and
robots
a lot of folks from elon musk to sam
harris
to a lot of folks talk about the
you know existential threats about
artificial intelligence
oftentimes robots kind of um inspire
that the most
because of the anthropomorphism do you
have any fears
it's an important question um
i actually i think i like rod brooks
answer maybe the best
on this i think and it's not the only
answer he's given over the years but
maybe one of my favorites is
he says it's not going to be he's got a
book flesh and machines i believe
it's not going to be the robots versus
the people we're all going to be robot
people
because you know we already have
smartphones some of us have
um serious technology implanted in our
bodies already whether we have a
hearing aid or a pacemaker or anything
like this
um people with amputations might have
prosthetics
that's a trend i think that
is likely to continue i mean this is now
uh
wild speculation but uh
i mean when do we get to cognitive
implants and the like
and yeah with neurolink brain computer
interfaces
that's interesting so there's a there's
a dance between humans and robots it's
going to be
it's going to be impossible to be scared
of
the other out there the robot because
the robot will be part of us
essentially it'd be so intricately sort
of part of our society
that and it might not even be implanted
part of us but just it's so much a part
of our
yeah our society so in that sense the
smartphone is already the robot we
should be afraid of yeah
uh i mean yeah and all the usual fears
arise
of the misinformation the
manipulation all those kinds of things
that um
that the problems are all the same
they're all they're human problems
essentially it feels like
yeah i mean i think the the way we
interact with each other online is
changing
the value we put on you know personal
interaction and that's a crazy big
change that's going to happen
and rip through our system has already
been ripping through our society right
and that has implications that are
massive i don't know if they should be
scared of it or go with the flow but
i don't see you know some battle lines
between humans and robots being
the first thing to worry about i mean i
do want to just
as a kind of comment maybe you can
comment about your just feelings about
boston dynamics in general
but you know i love science i love
engineering i think there's so many
beautiful ideas in
it and when i look at boston dynamics or
legged robots in general i think
they inspire people curiosity
and feelings in general excitement
about engineering more than almost
anything else in popular culture
and i think that's such an exciting plus
like responsibility and possibility for
robotics
and boston dynamics is riding that wave
pretty damn well like
they've found it they've discovered that
hunger and
curiosity in the people and they're
doing magic with it
i don't care if the i mean i guess their
company have to make money right
but uh they're already doing incredible
work and inspiring the world about
technology
i mean do you have do you have thoughts
about boston dynamics maybe
others your own work and robotics and
inspiring the world in that way i
completely agree i think
boston dynamics is absolutely awesome
i think i show my kids those videos
you know and the best thing that happens
is sometimes they've already seen them
you know uh
right i think i i just think
it's a pinnacle of success in robotics
that um
is just one of the best things that's
happened i absolutely completely agree
one of the heartbreaking things to me is
how many
robotics companies fail
how hard it is to make money with the
robotics company
like irobot like went through hell just
to
arrive at a roomba to figure out one
product
and then there's so many um home
robotics companies like
gebo and
anki anki
the cutest toy that's a great robot i
thought uh went down
i'm forgetting a bunch of them but a
bunch of robotics rods company
rethink robotics um
like do you um do you have any
anything hopeful to say about the
possibility of making money with robots
oh i think um you can't just look at the
failures you can all i mean boston
dynamics is a success there's lots of
companies that are still
doing amazingly good work in robotics
i mean this is the this is the
capitalist ecology or something right i
think you have many companies you have
many startups and they
push each other forward and many of them
fail and some of them get through and
that's sort of
the natural way of things the way of
those things i don't know that
is robotics really that much worse i i
feel the pain that you feel too every
time i
read one of these i um sometimes it's
friends and and i definitely
wish it went better or would differently
but i think it's healthy and good to
have um
bursts of ideas burst of activities
ideas
if they are really aggressive they
should fail sometimes
certainly that's the research mantra
right if you're
succeeding at every problem you attempt
then you're not
choosing aggressively enough is it
exciting to you uh the new spot
oh it's okay it's so good what are you
getting him as a pet
uh it yeah i mean i have to dig up
75k right now it's so cool that there's
a price tag you can go
and and then actually buy it and i have
a skydio r1
uh love it so um
no i would i would i would absolutely be
a customer uh
i wonder what your kids would think
about i i actually um
zach from boston dynamics would let my
kid drive in one of their demos one time
and uh that was just so good
so good and again forever be grateful
for that
and there's something magical about the
anthropomorphization of that arm
it adds another level of human
connection
i'm not sure we understand from a
control
aspect uh the value of
anthropomorphization
um i i think that's an understudied and
understood engineering problem there's
been a psycho
psychologists have been studying it i
think it's part
like manipulating our mind to believe
things uh
is a valuable engineering like this is
another degree of
freedom that can be controlled i like
that yeah i think that's right i think
you know there's something that humans
seem to do or maybe my
dangerous introspection is uh
i think we are able to make very simple
models that
assume a lot about the world very
quickly
and then uh it takes us a lot more time
like you're wrestling you know you
probably
thought you knew what you're doing with
wrestling and you were fairly functional
as a complete wrestler
and then you slowly got more expertise
so maybe
it's natural that our first
first level of defense against seeing a
new robot is to think of it
in our existing models of how humans and
animals behave
and it's just as you spend more time
with it then you'll develop more
sophisticated models that will
appreciate the differences exactly
can you say what does it take to control
a robot
like what is the control problem of a
robot and in general what is a robot
in your view like how do you think of
this system
what is a robot what is a robot i think
robotics ridiculous questions no no it's
good um
i mean there's standard definitions of
combining
computation with some ability to do
mechanical work
i think that gets us pretty close but i
think
robotics has this problem that once
things really work
we don't call them robots anymore like
your my dishwasher at home is
pretty sophisticated beautiful
mechanisms
there's actually a pretty good computer
probably a couple chips in there doing
amazing things we don't think of that as
a robot anymore
which isn't fair because then what
roughly it means that robots
robotics always has to solve the next
problem and
doesn't get to celebrate its past
successes i mean even factory room
floor robots are super successful
they're amazing but that's not the ones
i mean people think of them as robots
but they don't
if you ask what are the successes of
robotics somehow
it doesn't come to your mind immediately
so the definition of robot is
a system with some level automation that
fails frequently
something like it's it's the computation
plus mechanical
work and unsolved problems
solve problem yeah so so from a
perspective of control
and mechanics dynamics what
what is a robot so there are many
different types of robots
the control that you need for a
um a jibo robot you know some some robot
that's sitting on your
countertop and and interacting with you
but not touching you
for instance is very different than what
you need for an autonomous car or an
autonomous
drone it's very different than what you
need for a robot that's going to
walk or pick things up with its hands
right
my passion has always been for them
places where you're interacting more
you're doing more dynamic
interactions with the world so walking
now manipulation and the control
problems there
are are beautiful i think
contact is one thing that differentiates
them from many of the control problems
we've solved classically
right like modern control grew up
stabilizing fighter jets that were
passively unstable and
there's like amazing success stories
from control all over the place
um power grid i mean there's all kinds
of it's it's it's
everywhere uh that we don't even realize
just like ai
is now so you mentioned contact like
what's contact so an airplane
is of extremely complex system or a
spacecraft landing or whatever
but at least it has the luxury of
things change relatively continuously
that's an oversimplification but
if i make a small change in the command
i send to my actuator
then the path that the robot will take
tends to take a
change only by a small amount and
there's a feedback mechanism here
there's a feedback mechanism and
thinking about this as
locally like a linear system for
instance i can use
more linear algebra tools to study
systems like that
generalizations of linear algebra to to
these
smooth systems what is contact the
robot has something very discontinuous
that happens when it makes or breaks
when it starts touching the world and
even the way it touches or the order of
contacts can change the outcome in
potentially
unpredictable ways not unpredictable but
complex ways i do think there's a little
bit of
people a lot of people will say that
contact is hard in robotics
even to simulate um and i think there's
a little bit of a
there's truth to that but but maybe a
misunderstanding around that
so what is limiting is that when we
think about our robots when we write our
simulators
we often make an assumption that that
objects are rigid
and when it comes down you know that
they that their mass moves all
you know it stays in a constant position
relative to each other itself
um and and that leads to
some paradoxes when you go to try to
talk about rigid body mechanics and
contact
and so for instance if i have a
three-legged stool
with just a imagine it comes to a point
at the
at the leg so it's only touching the
world at a point
if i draw my physics my high school
physics
diagram of this system then there's a
couple of things that i'm given by
by elementary physics i know if the
system if the table is at rest if it's
not moving
it's zero velocities that means that the
normal force
all the forces are in balance so the
the force of gravity is being countered
by the forces that the ground is pushing
on my table legs i also know since it's
not rotating
that there that the moments have to
balance and since it can
in it's a three-dimensional table it
could fall in any direction
it actually tells me uniquely what those
three normal forces have to be
if i have four legs on my table
four-legged table
and they were perfectly machined to be
exactly the right same height and
they're set down and the table's not
moving
then the basic conservation laws don't
tell me
there are many solutions for the forces
that the ground could be putting on my
legs
that would still result result in the
table not moving
now the reason that seems fine i could
just pick one
but it gets funny now because if you
think about friction we
what we think about with friction is we
our standard model says
the amount of force that your that the
table will push back if i were to now
try to push my table sideways i guess i
have a table here
is proportional to the normal force
so if i have if i'm very barely touching
and i push i'll slide but if i'm pushing
more and i push i'll slide less
it's called coulomb friction is our
standard model
now if you don't know what the normal
force is on the four legs
and you push the table then you don't
know
what the friction forces are going to be
right and so you can't actually tell
the laws just don't aren't explicit yet
about which way the table is going to go
it could veer off to the left it could
veer off to the right it could go
straight
so the rigid body assumption of contact
leaves us with some paradoxes which are
annoying for
for writing simulators and for writing
controllers
we still do that sometimes because soft
contact is
potentially harder numerically or
whatever and the best simulators do both
or do some combination of the two
but but anyways because of these kind of
paradoxes there's all kinds of
paradoxes in contact uh mostly due to
these rigid body assumptions
it becomes very hard to like write the
same kind of control laws
that we've been able to be successful
with for like fighter jets
we haven't been as successful writing
those controllers for
manipulation and so you don't know
what's going to happen at the point of
contact
at the moment of contact there are
situations absolutely where you
where our laws don't tell us so the
standard approach that's okay
i mean instead of having a differential
equation
you end up with a differential inclusion
it's called it's a set valued
equation it says that i'm in this
configuration i have these forces
applied
on me um and there's there's a set of
things that could happen
right and um and those aren't
continuously i mean what
uh so when you're seeing like non-smooth
they're not only not smooth but this is
discontinuous
the non-smooth comes in when i make or
break a new contact first
or when i transition from stick to slip
so you typically have static friction
and then you'll start sliding and
that'll be a discontinuous change in
in velocity for instance especially if
you come to rest or
that's so fascinating okay so uh so what
do you
what do you uh do sorry i interrupted
you um
what's the hope under so much
uncertainty about what's going to happen
what are you supposed to do i mean
control has an answer for this robust
control is one approach
but but roughly you can write
controllers which try to
still perform the right task despite all
the things that could possibly happen
the world might want the table to go
this way in this way but if i write a
controller
that pushes a little bit more and pushes
a little bit i can
certainly make the table go in the
direction i want
it just puts a little bit more of a
burden on the control system
right and this discontinuities do change
the control system
because um the way we write it down
right now
every different control con
configuration
including sticking or sliding or parts
of my body that are in contact or not
looks like a different system and i
think of them i reason about them
separately or differently and the
combinatorics of that
blow up right so i just don't have
enough time to compute
all the possible contact configurations
of my humanoid
interestingly i i mean i'm a humanoid
i have lots of degrees of freedom lots
of joints
i only i've only been around for a
handful of years it's getting up there
but
i haven't had time in my life to visit
all of the states
in my system certainly all the contact
configurations
so if step one is to consider every
possible contact configuration that i've
i'll ever be in that's probably a
that's probably not a problem i need to
solve right
just as a small attention what's a
contact configuration
what like just so we can uh
yeah enumerate what are we talking about
yeah how many are there
the simplest example maybe would be
imagine a robot with a flat foot
and we think about the phases of gate
where the heel
strikes and then the four the front toe
strikes
and then you can heal up toe off
those are each different contact
configurations
i only had two different contacts but i
ended up with four different
contact configurations now of course
i might have my my robot might actually
have bumps on it or other things so it
could be much more subtle than that
right but it's just even with one sort
of box
interacting with the ground already in
in the plane has that many right and if
i was just even a 3d
foot then probably my left toe might
touch just before my right toe and
things get subtle now if i'm a dexterous
hand
and i go to talk about just grabbing a
water bottle if every
if i have to enumerate every possible
order
that my hand came into contact with the
with the bottle
then i'm dead in the water my any
approach that we were able to get away
with that in walking
because we mostly touch the ground
within a small number of points for
instance
and we haven't been able to get dextrous
hands that way
so i mean you've mentioned that
people think that contact is really hard
and that that's the reason that
robotic manipulation is problem is
really hard
is there any flaws in that
thinking so i think simulating contact
is one aspect i know people often say
that we don't
that one of the reasons that we have a
limit in robotics is because we do not
simulate contact accurately
in our simulators and i think that is
the extent to which that's true is
partly because our
simulators we haven't got mature enough
simulators
there are some things that are still
hard difficult that has changed
but but we actually we know what the
governing equations are they have some
foibles
like this indeterminacy but we should be
able to simulate them accurately
we have incredible open source community
in robotics but it actually just takes a
professional engineering team
a lot of work to write a very good
simulator like that
uh now where does um i believe you've
written drake
there's a team of people i certainly
spend a lot of hours on it myself
well what is drake and what um what does
it take to
to to create a simulation environment
uh for for the kind of difficult control
problems we're talking about
right so drake is the simulator that
that i've been working on
um there are other good simulators out
there i don't like to think of drake as
just a simulator
because because we write our controllers
in drake we write our perception systems
a little bit in drake
but we write all of our our you know low
level control and even planning
and uh optimization intelligence
optimization capabilities
absolutely yeah i mean drake is three
things roughly
it's an optimization library which is um
sits on it it provides a layer of
abstraction in c
plus and python for commercial solvers
you can write linear programs quadratic
programs
you know semi-definite programs sums of
squares programs
the ones we've used mixed integer
programs and it will do the work to
curate those and send them to whatever
the right solver is for instance and it
provides a level of abstraction
the second thing is is a system modeling
language
a bit like labview or simulink where you
can make block diagrams out of complex
systems or it's like ross in that sense
where
you might have lots of ross nodes that
are each doing some part of your system
but to contrast it with ross
we try to write if you write a drake
system then you have to
it asks you to describe a little bit
more about the system
if you have any state for instance in
the system there any variables that are
going to persist you have to declare
them
parameters can be declared and the like
but the
advantage of doing that is that you can
if you like
run things all on one process but you
can also
do control design against it you can do
i mean simple things like
rewinding and playing back your your
your simulations for instance you know
these things you get some rewards for
spending a little bit more upfront cost
in describing each system
and and i i was inspired to do that
because i think the complexity of atlas
for instance
um is just so great and i think although
i mean
ross has been incredible absolutely huge
fan of what it's done for the
robotics community but it um
the ability to rapidly put different
pieces together and have a functioning
thing is very good but i do think that
it's hard to think clearly about a bag
of disparate
parts mr potato head kind of software
stack
and if you can
you know ask a little bit more out of
each of those parts then you can
understand
the way they work better you can try to
verify them
and the like um you can do learning
against them
and then one of those systems the last
thing i i said the first two things that
drake is but
the last thing is that there is a set of
multi-body equations rigid body
equations
that is trying to provide a system that
simulates physics
and that um we also have renderers and
other things but i think the
physics component of drake is is special
in the sense that
um we have done excessive amount of
engineering
to to make sure that we've written the
equations correctly every possible
tumbling satellite or spinning top or
anything that we could possibly write as
a test is tested
um we are making some you know i think
fundamental improvements on the way you
simulate contact
yes what does it take to uh simulate
contact
i mean it just seems uh
i mean there's something just beautiful
the way you were like explaining
contact and you're like tapping your
fingers on the on the table while you're
while you're doing it just um easily
right
easily just like just not even like
it was like helping you think i guess um
what i um see you have this like awesome
demo of um
loading or unloading a dishwasher
just picking up a plate
uh grasping it like for the first time
um that's just seems like so difficult
what how do you simulate any of that
so it was really interesting that what
happened was that
um we started getting more professional
about our software development during
the darpa robotics challenge
i learned the value of software
engineering and how these
how to bridle complexity i guess that's
that's what i i want to somehow
fight against and bring some of the
clear thinking of controls into
these complex systems we're building for
robots
um shortly after the darpa robotics
challenge
toyota opened a research institute tri
toyota research institute
um they put one of their there's there's
three locations one of them is just down
the street from mit
and uh and i helped ramp that up
uh right out as a part of my uh
the end of my sabbatical i guess um
so so tri is uh
has given me the tri robotics effort has
made this investment in simulation
in drake and michael sherman leads a
team there of just
absolutely top notch dynamics experts
that
are trying to write those simulators
that can pick up the dishes
and there's also a team working on
manipulation there that is
taking problems like loading the
dishwasher
and we're using that to study these
really hard corner cases kind of
problems in manipulation
so for me this you know simulating
the dishes we could actually write a
controller if we just cared about
picking up dishes in the sink once we
could write a controller without any
simulation whatsoever
and we could call it done but we want to
understand like
what is the path you take to actually
get to
a robot that could perform that for any
dish
uh in anybody's kitchen with with enough
confidence that it could be
a commercial product right and and
it has deep learning perception in the
loop it has complex dynamics in the loop
it has controller it has a planner
and how do you take all of that
complexity
and put it through this engineering
discipline and verification
and validation process to actually get
enough confidence to deploy i mean the
darpa challenge
made me realize that that's not
something you throw over the fence and
hope that somebody will harden it
for you that there are really
fundamental challenges
in uh in closing that last gap they're
doing the validation
and the testing i think it might even
change the way we have to think about
the way we write systems
what happens if you if you have the
robot
running lots of tests it and it
screws up it breaks a dish right how do
you capture that i said you can't run
the same simulation or the same
experiment twice
in in a real on a real robot do we have
to be able to bring that
one off exp failure back into simulation
in order to change our controllers
study it make sure it won't happen again
do we
is it enough to just try to add that to
our distribution
and understand that on average we're
going to cover that situation again
there's like really subtle questions at
the corner cases that
i think we don't yet have satisfying
answers for like how do you find the
corner cases
that's one kind of is there do you think
this possible to create a
systematized way of discovering
corner cases efficiently yeah in
whatever the problem is yes i mean i
think we have to get better at that
i mean control theory has um for
for decades talked about active
experiment design
so people call it curiosity these days
it's roughly this idea of trying to
exploration or exploitation but but in
the active experiment design is even
is is more specific you could try to
understand the uncertainty in your
system design the experiment that will
provide the maximum information to
reduce that uncertainty
if there's a parameter you want to learn
about what is the optimal
trajectory i could execute to learn
about that parameter
for instance scaling that up to
something that has a deep network in the
loop and the planning in the loop is
tough
we've done some work on you know with
matt o'kelly and amancina we have
we've worked on um some falsification
algorithms that are trying to do rare
event simulation that try to just hammer
on your simulator
and if your simulator is good enough you
can um
you can spend a lot of time or you can
write good algorithms
that try to spend most of their time in
the corner cases
so you basically imagine you're you're
building an autonomous car and you want
to put it in
i don't know downtown new delhi all the
time right an accelerated testing
if you can write sampling strategies
which figure out where your controller
is performing badly in simulation
and start generating lots of examples
around that
you know it's just the space of possible
places where that can be
where things can go wrong is very big so
it's hard to write those algorithms yeah
rare event simulation is just like a
really compelling notion
uh if it's possible we joked and we
called we call it the black swan
generator
it's a black swan right because you
don't just want the rare events you want
the ones that are
highly impactful i mean that's the most
those are the most sort of profound
questions we ask of our world
like uh what's the
uh what's the worst that can happen uh
but what we're really asking
isn't some kind of like computer science
worst case analysis
we're asking like what are the millions
of ways
this can go wrong and that's like our
curiosity
we humans i think are pretty bad at uh
we just like run into it and i think
there's a distributed sense because
there's now like 7.5 billion of us
and so there's a lot of them and then a
lot of them write blog posts about the
stupid thing they've done so we learn
in a distributed way um there's there's
something that's going to be important
for robots too yeah i mean that's
that's another massive theme at toyota
research
for robotics is this fleet learning
concept is
um you know the idea that i as a
humanoid
don't have enough time to visit all of
my states right there's just a
it's very hard for one robot to
experience all the things
but that's not actually the problem we
have to solve right um
we're gonna have fleets of robots that
can have very similar appendages
and at some point maybe collectively
they have
enough data that their computational
processes should be set up differently
than ours right
it's a have this vision of just i mean
all these
uh dishwasher unloading robots
i mean um that robot dropping a plate
and a human looking at the robot
probably pissed off
yeah but uh that's a special moment to
record
i think one one thing in terms of fleet
learning
and i've seen that because i i've talked
to a lot of folks um
just like like tesla users or tesla
drivers
they're not another another company
that's using this kind of fleet learning
idea
and one hopeful thing i have about
humans
is they really enjoy when a system
improves learns so they enjoy fleet
learning and they're
the reason it's hopeful for me is
they're willing to put up with
something that's kind of dumb right now
and they're like
if it's improving they almost like enjoy
being part of the like teaching it
almost like we if you have kids like
you're teaching them something right
um i think that's a beautiful thing
because that that gives me hope that we
can put
dumb robots out there uh as long i mean
the problem with
on the tesla side with cars cars can
kill you
that's that makes the problem so much
harder dishwasher
unloading is a little safe that's why
home robotics is uh
it's really exciting and just to clarify
i mean for people who might not know
a tri toyota research institute so
they're
uh i mean they're they're pretty well
known for like autonomous vehicle
research
but they're also interested in in
home robotics yep there's a big there's
a big group working on
multiple groups working on home robotics
it's a major part of the
portfolio also there's also a couple
other projects an
advanced materials discovery i'm using
ai and machine learning to discover new
materials for
um for car batteries and then the like
for instance yeah
and that's been actually an incredibly
successful team uh there's new projects
starting up too so
do you see a future of uh where
like robots are in our home and and
like robots that have like
um actuators that look like arms
in our home or like you know more like
humanoid type robots
or is this are we gonna is we're gonna
do the same thing that you just
mentioned that
you know the dishwasher is no longer a
robot we're going to just
not even see them as robots but do i
mean what's your
vision of the home of the future 10 20
years from now
50 years if you get crazy yeah i think
we already have roombas cruising around
we have
uh uh you know alexa's or google homes
on their our
kitchen counter it's only a matter of
time till they spring arms and start
doing something
useful like that um so
i do think it's coming i think it's lots
of people have lots of motivations
for doing it it's been super interesting
actually learning about
toyota's vision for it which is about
helping people age in place
because i think that's not necessarily
the first entry the most lucrative
entry point but it's the problem maybe
that
we really need to solve no matter what
and
so i think i think there's a real
opportunity it's a delicate
problem how do you work with people help
people
keep them active engaged you know
but improve their quality of life and uh
and and help them age in place for
instance
it's interesting because older folks are
also i mean there's a contrast there
because um
they're not always the the folks who are
the most comfortable technology for
example
so there's a there's a there's a
division that's interesting there that
you can do so much good with a robot
for for older folks but there's a
there's a gap to feel of understanding i
mean
it's actually kind of beautiful um robot
is learning about the human and the
human is kind of learning about this
new robot thing and it's uh also with um
at least with uh like when i talk to my
parents about robots there's a little
bit of a
blank slate there too like
you can i mean they don't know anything
about robotics
so it's completely like wide open
they don't have that they haven't my
parents haven't seen black mirror
so like they they there's it's a blank
slate here's a cool thing like what can
you do for me
yeah so it's an exciting space i think
it's a really important space
i do feel like you know a few years ago
uh drones were successful enough in
academia they kind of
broke out and started in industry and
autonomous cars
have been happening it does feel like
manipulation
in logistics of course first but in the
home
shortly after seems like one of the next
big things that's going to really
pop so uh i don't think we talked about
it but
uh what's soft robotics so we talked
about
like rigid bodies like if we can just
linger on this whole touch
thing um yeah so what's soft robotics
so i told you that i
really dislike the fact that robots are
afraid of touching the world
all over their body so there's a couple
reasons for that
if you look carefully at all the places
that robots actually do touch the world
they're almost always soft they have
some sort of pad on their fingers or a
rubber
sole on their foot but if you look up
and down the arm
we're just pure aluminum or something
so uh so that makes it hard actually in
fact
hitting the table with your you know
your rigid arm or nearly rigid arm
is a is a has some of the problems that
we talked about
in terms of simulation i think it it
fundamentally changes the mechanics of
contact when you're soft
right you you turn point contacts into
patch contacts which can have torsional
friction
you can have um distributed load if i
want to pick up an egg
right if i pick it up with two points
then
in order to put enough force to sustain
the weight of the egg i might have to
put a lot of force to break the egg if i
envelop it with a
with contact all all around then i can
distribute my
force across the shell of the egg and
have a better chance of not breaking it
so soft robotics is for me a lot about
changing the mechanics
of contact does it make the problem a
lot harder
um
quite the opposite uh it it changes the
computational problem
i think because of the i think our
world and our mathematics has biased us
towards ridgid
but it really should make things better
in some ways right um
it's it's a i think the the
the future is unwritten there um but the
other thing is
ultimately sorry to interrupt they think
ultimately it will make things simpler
if we embrace the softness of the world
it makes
um it makes things smoother right
so the the result of
small actions is less discontinuous but
it also means
potentially less you know
instantaneously bad
for instance i won't necessarily contact
something and send it flying off
the other aspect of it that just happens
to dovetail really well is that
if soft robotics tends to be a place
where we can embed a lot of sensors to
so if you change your your hardware
and make it more soft then you can
potentially have a tactile sensor which
is
measuring the deformation
so there's a team at tri that's working
on
soft hands and and you get so much more
information
if you you can put a camera behind the
skin roughly
and and get fantastic tactile
information
which is um it's super important like in
manipulation one of the things that
really
is frustrating is if you work super hard
on your head mounted
on your perception system for your head
mounted cameras and then you've
identified an object you reach down to
touch it and the first the last thing
that happens right before
the most important time you stick your
hand and you're occluding
your head mounted sensors right so in
all the part that really matters
all of your off board sensors are you
know are occluded
and really if you don't have tactile
information then you're
you're blind in an important way so
it happens that soft robotics and
tactile sensing
tend to go hand in hand i think we've
kind of talked about it but
uh you taught a course on under actuator
robotics
i believe that was the name of it
actually that's right
can you talk about it in that context
what is
under actuated robotics right so
under-actuated robotics is my graduate
course it's it's online
mostly now so i mean in the sense that
the lecturer's versions of it i think
right the youtube really great i
recommend it highly look on youtube for
the
2020 versions until march and then you
have to go back to 2019 thanks to
covet um no i've poured my heart into
that
class and lecture one is basically
explaining what the word underactuated
means so people are very kind to show up
and then
maybe have to learn what the title of
the course means over the course of the
first lecture
that that first lecture is really good
you should watch it
it's it's a strange name but um
i thought it captured the essence of
what
control was good at doing and what
control was bad at doing
so what do i mean by under actuated so
a mechanical system
has many degrees of freedom for instance
i think of a joint as a degree of
freedom
and it has some number of actuators
motors
so if you have a robot that's bolted to
the table
that has five degrees of freedom
and five motors then you have a fully
actuated robot
if you have if you take away one of
those motors
then you have an under actuated robot
now why
on earth i i have a good friend who who
likes to tease me he said russ if you
had more research funding would you work
on fully actuated robots
yeah and uh the answer is no
the world gives us under-actuated robots
whether we like it or not i'm a human
i'm an under actuated robot even though
i have more muscles
than my big degrees of freedom because i
have in some places
multiple muscles attached to the same
joint
but still there's a really important
degree of freedom that i have which is
the location of my
center of mass in space for instance
all right i'm i can jump into the air
and there's no motor that connects my
center of mass to the ground
in that case so i have to think about
these the implications
of not having control over everything
the passive dynamic walkers are the
extreme view of that where you've taken
away all the motors and you have to let
physics do the work
but it shows up in all the walking
robots where you have to use some of the
actuators
to push and pull even the degrees of
freedom that you don't have an actuator
on
that's referring to walking if you're
like falling forward
like is there a way to walk that's fully
actuated
so it's a subtle point when you're when
you're
in contact and you have your feet
on the ground there are still limits to
what you can do right
unless i have suction cups on my feet i
cannot accelerate my center of mass
towards the ground
faster than gravity because i can't get
a force pushing me down
right but i can still do most of the
things that i want to so you can get
away
with basically thinking of the system as
fully actuated unless you suddenly need
it to accelerate down super fast
but as soon as i take a step i i get
into more
nuanced territory and to get to really
dynamic robots or airplanes or
other things i think you have to embrace
the
under actuated dynamics manipulation
people
think is manipulation under under
actuated my even if my arm
is fully actuated i have a motor if my
goal is to control
the position and orientation of this cup
then i don't have an actuator for that
directly so i have to use my actuators
over here to control this thing
now it gets even worse like what if i
have to button my shirt
okay what are the degrees of freedom of
my shirt right i suddenly
that's a hard question to think about it
kind of makes me queasy as a
thinking about my state-space control
ideas
but actually those are the problems that
make me so excited about manipulation
right now is that it
it breaks some of the it breaks a lot of
the foundational control stuff that i've
been thinking about
is there um what are some interesting
insights
you could say about trying to solve an
under actuated
like control in in an underactuated
system
so i think the philosophy there is let
physics do
more of the work the technical approach
has been optimization so you typically
formulate your decision making for
control
as an optimization problem and you use
the language of optimal control
and sometimes numero often numerical
optimal control
in order to make those decisions and
balance
you know these complicated equations of
and in order to control
you don't have to use optimal control to
do under-actuated
systems but that has been the technical
approach that has borne the most fruit
in our
at least in our line of work and there's
some
so in under actuator systems when you
say let physics do some of the work
so there's a kind of feedback feedback
loop
that observes the state that the physics
brought you to
so like you've there's a there's a
perception there this is there's a
feedback
somehow do you do um do you
ever loop in like complicated perception
systems into this whole picture
right right around the time of the darpa
challenge we had a complicated
perception system in the darpa challenge
we also started to embrace perception
for our
flying vehicles at the time we had a a
really good project
on trying to make airplanes fly at high
speeds through forests
um sirtex caramel was on that
project and it was a really fun team to
to work
on he's carried it farther much farther
forward since then
so that's using cameras for perception
so that was using cameras uh that was a
at the time we felt like lidar was too
too heavy and two power
heavy to to be carried on on a light uav
and we were using cameras
and that was a big part of it was just
how do you do even stereo
matching at a fast enough rate with a
small camera
a small onboard compute since then
we have now the so the deep learning
revolution unquestionably changed
what we can do with perception for
robotics and control so in manipulation
we can address
we can use perception in a i think a
much deeper way
and um we get into not only i think the
the first use of it naturally would be
to
ask your deep learning system to look at
the cameras and
produce the state which is like the pose
of my thing for instance
but i think we've quickly found out that
that's not always the right
thing to do um why is that because
what's the state of my shirt imagine
i've always
very noisy i mean or it's um
if the first step of me trying to button
my shirt
is estimate the full state of my shirt
including
like what's happening in the backyard or
whatever whatever that's just not the
right
specification there are aspects of the
state that are very important
to the task there are many that are
unobservable and not not important to
the task
so you really need it begs new questions
about
state representation another example
that we've been playing with in lab has
been
just the idea of chopping onions okay
or carrots turns out to be better so
the onions stink up the lab uh and
they're
hard to see in a camera but uh
so details matter yeah details matter
you know so um
moving around a particular object right
then i think about oh it's got a
position or an orientation in space
that's the description i want
now when i'm chopping an onion okay the
first chop comes down
i have now a hundred pieces of onion
does my control system really need to
understand the position and orientation
and even the shape of the hundred pieces
of onion in order to make a decision
probably not you know and like if i keep
going i'm just getting more and more is
my state space getting bigger as i cut
it's it um it it's not right
yes so there's a i think there's
a richer uh idea of state
it's not the state that is given to us
by lagrangian mechanics
there is a there is a proper lagrangian
state of the system but the relevant
state for this
is some latent state is what we call it
in machine learning
but you know there's some some different
state representatives
some compressed representation some and
that's what i
i worry about saying compressed because
it doesn't i don't
mind that it's low dimensional or not
but it has to be something that's easier
to think about
by us humans or my algorithms or
the algorithms being like control
optimal so for instance if the contact
mechanics
of all of those onion pieces and all the
permutations of possible touches between
those onion pieces
you know you can give me a high
dimensional state representation i'm
okay if it's
linear but if i have to think about all
the possible shattering combinatorics of
that
then my robot is going to sit there
thinking and uh
the soup's gonna get cold or something
so um since you taught the course
i've it kind of entered my mind um the
idea of under actuated as
really compelling to see the to see the
world in this kind of way
um do you ever you know if we talk about
onions or you talk about
the world with people in it in general
do you see the world
as a basically an underactuated system
do you like often
look at the world in this way or is this
uh overreach
um under actuated as a way of life man
exactly um i guess that's what i'm
asking
i do think it's everywhere i think some
in some places
um we already have natural tools to deal
with it
you know it rears its head i mean in
linear systems it's not a problem we
just we just
like an under actuated linear system is
really not sufficiently distinct from a
fully actuated
linear system it's it's a it's a subtle
point about when that becomes
a bottleneck and what we know how to do
with control it happens to be a
bottleneck
although we've gotten incredibly good
solutions now but for a long time that i
felt that that was the key bottleneck
in legged robots and roughly now the
under actuated course is
you know me trying to tell people
everything i can
about how to make atlas do a backflip
right
i have a second course now in that i
teach in the other semesters which is
on on manipulation and that's where we
get into now more of the that's a newer
class
i'm hoping to put it online this fall
completely and uh that's going to have
much more aspects about these perception
problems and the state representation
questions and then how do you do control
and the the thing that's a little bit
sad
is that uh for me at least is there's a
lot of manipulation tasks that people
want to do and should want to do they
could
start a company with it and make very
successful that don't actually
require you to think that much about
under or dynamics at all even
but certainly under actuated dynamics
once i have if i
if i reach out and grab something if it
if i can sort of assume it's rigidly
attached to my hand then i can do a lot
of interesting meaningful things with it
without really ever thinking about the
dynamics of that object
so they built we've built systems that
kind of
reduced the need for that enveloping
grasps and the like
um but i think the really good problems
in manipulation so
manipulation by the way is more than
just pick and
place that's like a lot of people think
of that just grasping
i don't mean that i mean buttoning my
shirt i mean tying shoe laces
how do you program a robot to tie
shoelaces and not just one shoe but
every shoe right that's a really good
problem it's tempting to write down like
the infinite dimensional state of the
of the laces that's probably not needed
to write a good controller
i know we could hand design a controller
that would do it
but i don't want that i want to
understand the principles
that would allow me to solve another
problem that's kind of like that
but i think if we can stay pure in our
approach
then the challenge of tying anybody's
shoes
is a great challenge that's a great
challenge i mean and the soft
touch comes into play there that's
really interesting
let me ask another ridiculous question
on this topic um
how important is touch we haven't talked
much about humans
but i have this argument with my dad
where like i think you can fall in love
with the robot
based on uh language alone and he
believes that
touch is essential i touch and smell he
says but
um so
in terms of robots you know connecting
with humans
and uh we can go philosophical in terms
of like a deep meaningful connection
like
love but even just like collaborating in
an interesting way
how important is touch like
uh from the engineering perspective and
the philosophical one
i think it's super important let's even
just in a practical sense if we forget
about the emotional
part of it but for robots to interact
safely while they're doing
meaningful mechanical work in the pro
in the you know close contact with or
vicinity
of people that need help i think we have
to have them
they have we have to build them
differently um they have to be afraid
not afraid of touching the world
so uh i think baymax is just awesome
that's just
like the the the movie of big hero 6 and
the
the concept of baymax that's just
awesome i think we should
and we have some folks at toyota that
are trying to toyota research that are
trying to build baymax roughly
and i think it's just a fantastically
good project
i think it will change the way people
physically interact
the same way i mean you gave a couple
examples earlier but but if i um
if the robot that was walking around my
home looks more like a teddy bear
and a little less like the terminator
that could change completely the way
people perceive it
and interact with it and maybe they'll
even want to teach it like you said
right you could um not quite gamify it
but somehow
instead of people judging it and looking
at it as if uh
it's not doing as well as a human
they're going to try to help out the
cute teddy bear
right who knows but i i think we're
building robots wrong
and being more soft and more contact
is important right yeah and
like all the magical moments i can
remember with robots
well first of all just visiting your lab
seeing atlas
but also spot menu when i first spot saw
spot many in person and hung out with
him
her uh it i don't have trouble
engendering robots i feel robotics
people really say always
it i kind of like the idea that it's a
her or
him uh there's a magical moment but
there's no touching uh i guess the
question i have have you ever been um
like have you had a human robot
experience where like
a robot touched you
and like it was like wait like was there
a moment that you've forgotten that a
robot
is a robot and like the
anthropomorphization
stepped in and for a second you forgot
that it's not human
i mean i think when you're in on the
details
then we we of course anthropomorphized
our work with atlas but in you know
in verbal communication and the like i
think we were pretty aware
of it as a machine that needed to be
respected
um i actually i worry more about the
smaller robots that
could still you know move quickly if
programmed wrong and uh
and we have to be careful actually about
safety and the like right now
and that if we build our robots
correctly i think then
those a lot of those concerns could go
away and we're seeing that trend we're
seeing the lower
cost lighter weight arms now that could
be
fundamentally safe um
i mean i do think touch is so
fundamental ted adelson
is uh is great he's a perceptual
scientist
at mit and he
studied vision most of his life and he
said when i had
kids i expected to be fascinated by
their perceptual development
but what really what he noticed was felt
more impressive more dominant was the
way that they would
touch everything and lick everything and
pick things up stick it on their tongue
and whatever
and he said watching his daughter
uh convinced him that actually he needed
to study tactile sensing more
so there's something very
important i think it's a little bit also
of the
passive versus active part of the world
right you can
passively perceive the world
but it's fundamentally different if you
can do an experiment right
and if you can change the world and you
can learn a lot more
than a passive observer so
you can in dialogue that was your
initial example you could have an
active experiment exchange but i think
if you're just a camera watching youtube
i think that's a very different problem
than if you're a
robot that can apply force and touch
i i i think it's important
yeah i think it's just an exciting area
of research i think you're probably
right that this hasn't been
under researched it's uh
to me as a person who's captivated by
the idea of human robot interaction
it feels like such a rich
opportunity to explore touch not even
from a safety perspective but like you
said the emotional
too i mean safety comes first um
but the next step is like
you know uh like a real human connection
even in the war like even in the
industrial setting
it just feels like uh it's nice for the
robot
i don't know i you know you might
disagree with this but um
because i think it's important to see
robots as tools
often but i don't know i think they're
just always going to be more effective
once you humanize them
uh like it's convenient now to think of
them as tools because we want to focus
on the safety
but i think ultimately to create
like a good experience for the worker
for the person
there has to be a human element
i don't know for me i i it feels like
like an industrial robotic arm
would be better if as a human element i
think like we think robotics had that
idea with
baxter and having eyes and so on having
i don't know i'm a big believer in that
i it's not my area
but i am also a big believer do you have
an emotional connection to atlas
like yeah do you miss him i mean
yes i i i don't know if i'd more so than
if i had a different science project
that i'd worked on
super hard right but uh um
yeah i mean the robot we basically had
to do heart surgery on the robot in the
final competition because we melted the
core
um and uh and
yeah there was something about watching
that robot hanging there we know we had
to compete with it in an hour and it was
getting its
guts ripped out those are all historic
moments
i think if we look back like 100 years
from now
um yeah i think those are important
moments in robotics
i mean these are the early day you look
at like the early days of a lot of
scientific disciplines they look
ridiculous they're full of failure
but it feels like robotics will be
important
in the coming uh 100 years and these are
the early days so
so i think a lot of people are look at
uh
a brilliant person such as yourself and
and are curious about the intellectual
journey they've took
um is there maybe three books
technical fiction philosophical that um
had a big impact on your life that you
would recommend
perhaps others reading
yeah so um i actually didn't read that
much as a kid but i
read fairly voraciously now um
there are some recent books that if
you're interested in this kind of topic
like ai superpowers by kaifuli is just a
fantastic read
you must read that um
yuval harari is just i think that
can open your mind um sapiens
sapiens as as the first one homo deuce
is the second
yeah i think we mentioned the black swan
by taleb i think that's a good sort of
mind opener
i actually um
so so there's maybe a more controversial
recommendation i could give um great
well i would love something
sure in some sense it's it's so
classical it might surprise you but
i actually recently read um mortimer
adler's
how to read a book not so long it was a
while ago but
some people hate that book i
loved it i think we're in this time
right now where
um boy we're just inundated with
research papers that you could read on
archive with
limited peer review and just this wealth
of information
um i don't know i think the
passion of um what you can get out of a
book
a really good book or a really good
paper if you find it
the attitude the realization that you're
only going to find a few that really
are worth all your time but then
once you find them you should just dig
in and and and
understand it very deeply and it's worth
you know marking it up and and uh you
know having the hard copy
writing in the the side notes side
margins
um i think that was really
it i read it at the right time where i
was just feeling just
overwhelmed with really low quality
stuff
i guess and similarly
uh i'm just giving more than three now
i'm sorry if i've exceeded my
my quota but on that topic just real
quick is uh
so basically finding a few companions
to keep for the rest of your life in
terms of papers and books and so on and
those are the ones like not doing um
what is it fomo fear missing out
constantly trying to update yourself
but really deeply making a life journey
of studying a particular paper
essentially
a set of papers yeah i think
when you really find something which a
book that resonates with you might not
be the same book that resonates with me
but um when you really find one that
resonates with you i think the dialogue
that happens
and that's what i love that adler was
saying you know i think
socrates and plato say um
the the written word is never going to
capture
the beauty of dialogue right but adler
says no no
um a a really good book
is a dialogue between you and the author
and it crosses time and space and uh
i don't know i think it's a very
romantic there's a bunch of like
specific advice which you can just gloss
over but the
romantic view of how to read and really
appreciate it is is is so good
and similarly teaching i uh um
i thought a lot about teaching and uh
and so isaac asimov
great science fiction writer has also
actually spent a lot of his
career writing nonfiction right his
memoir is fantastic
he was passionate about explaining
things right he wrote all kinds of books
on all kinds of topics in science
he was known as the great explainer and
some you know i
i do really resonate with his style
and uh and just his way of
talking about you know by communicating
and explaining to something is really
the way that you learn something i think
i
think about problems very differently
because of the way i've been
given the opportunity to teach them at
mit
and we have questions asked you know the
fear of the lecture
the experience of the lecture and the
questions i get and the interactions
just forces me to be rock solid on on
these ideas in a way that
i didn't have that i i don't know i
would be in a different intellectual
space
also video does that scare you that your
lectures are online
and people like me and sweatpants can
sit sipping coffee and watch
what you give lectures that i think it's
great
i do think that something's changed
right now which is
you know right now we're giving lectures
over zoom
i mean giving seminars over zoom and
everything
um i'm trying to figure out i think it's
a new medium
do you think it's trying to figure out
how to use it
yeah i've been um i've been
quite um cynical
about the human to human connection over
over that medium but i think that's
because it's
hasn't been explored fully and teaching
is a different thing
every lecture is a is a i'm sorry every
seminar even
i think every talk i give i i
you know there's an opportunity to give
that differently i can
i can deliver content directly into your
browser you have a webgl
engine right there i could i can throw
3d
uh content into your browser while
you're listening to me right yeah
and i can assume that you have a you
know at least a
powerful enough laptop or something to
watch zoom while i'm doing that while
i'm giving a lecture
that that's a that's a new communication
tool that i didn't have last year
right and uh i think robotics can
potentially
benefit a lot from teaching that way
we'll see it's going to be an experiment
this fall
i'm thinking a lot about it yeah and
also like
um the the length of lectures or the
length of
like um there's something so like i
guarantee you
you know it's like 80 percent of people
who started listening to our
conversation
are still listening to now which is
crazy to me
but so there's a there's a patience and
a interest in long-form content
but at the same time there's a magic to
forcing yourself to condense an idea
to as short as possible
uh as short as possible like clip it can
be part of a longer thing but like just
like really
beautifully condensed idea there's a lot
of opportunity there
that's easier to do and remote with
i don't know uh with editing too
editing is an interesting thing like
what uh
you know most professors don't get when
they give a lecture you don't get to go
back and edit out parts
like chris like crisp it up a little bit
that's also it can do magic like if you
remove like five to ten minutes from an
hour lecture
it can it can actually cr it can make
something special of a lecture i've uh
i've seen that in myself and and
in others too because i edit other
people's lectures to extract clips
it's like there's certain tangents
they're like that lose they're not
interesting they're
they're they're mumbling they're just
not they're not clarifying they're not
helpful at all
and once you remove them it's just i
don't know
editing can be magic uh take a lot of
time
yeah it takes it depends like what is
teaching you have to ask
um yeah because i find the editing
process is
also beneficial as
uh for teaching but also for your own
learning
i don't know if have you watched
yourself in the survey
have you watched those videos it's i
mean not all of them okay
it could be it could be painful yeah and
to see like how to
improve so do you find that i know you
segment your
um your podcast do you think that helps
people
with the the attention span aspect of it
or is it segment like
sections like yeah we're talking about
this topic whatever no
no that just helps me it's actually bad
so uh
and you've been incredible uh so i'm i'm
learning like i'm afraid of conversation
this is
even today i'm terrified of talking to
you i mean it's something
i'm um trying to remove from myself
i there's this a guy i mean i've learned
from a lot of people but
really um there's been a few people
who's been inspirational to me in terms
of conversation
whatever people think of him joe rogan
has been inspirational to me because
comedians have been too being able to
just
have fun and enjoy themselves and lose
themselves in conversation
that requires you to be a great
storyteller
to be able to uh pull a lot of different
pieces of information together
but mostly just to enjoy yourself in
conversations i'm
trying to learn that these notes are you
see me looking down
that's like a safety blanket that i'm
trying to let go of more and more
cool so that's that people love just
regular conversation
that's what they the structure is like
whatever
i would say i would say maybe
like 10 to like so there's a bunch of
you know
there's uh probably a couple thousand
phd students listening to this right now
right and they might know what we're
talking about
but there's somebody i guarantee you
right now
in russia some kid who's just like who's
just smoke some weed is sitting back and
just
enjoying the hell out of this
conversation not really understanding he
kind of watched some boston dynamics
videos he's just enjoying it
um and i salute you sir uh no but just
like there's a
so much variety of people that just have
curiosity about engineering about
sciences
about mathematics and um and also like i
should
that i mean uh enjoying it is one thing
but i also
often notice it inspires people to
there's a lot of people who are like in
their undergraduate studies trying to
figure out what
uh trying to figure out what to pursue
and those these conversations can really
spark the direction there
of their life and in terms of robotics i
hope it does because uh
i'm excited about the possibilities for
robotics brings on that
topic um do you have advice
like what advice would you give to a
young person
about life a young person about life or
a young person
about life and robotics uh it could be
in robotics
it could be in life in general it could
be career
it could be uh relationship advice it
could be
running advice just like they're um
that's one of the things i see you like
to talk to like 20 year olds
they're they're like how do i how do i
do this thing
what do i do um if they come up to you
what would you tell them i think
it's an interesting time to be a kid
these days
everything points to this being sort of
a winner take all economy and the like i
think
the people that will really excel
in my opinion are going to be the ones
that can think deeply
about problems
you have to be able to ask questions
agilely and use the internet for
everything it's good for and stuff like
this and i think a lot of people will
develop those skills
i think the the leaders
thought leaders you know robotics
leaders whatever
are going to be the ones that can do
more and they can think very deeply and
critically
um and that's a harder thing to learn
i think one one path to learning that is
through mathematics
through engineering
i would encourage people to start math
early i mean i
didn't really start i mean i was always
in the
the better math classes that i could
take but i wasn't pursuing
super advanced mathematics or anything
like that until i got to mit i think mit
lit me up and really started
the life that i'm living now but
yeah i really want kids to to dig deep
really understand things building things
too i mean pull things apart
put them back together like that's just
such a good way to really
understand things and expect it to be
a long journey right it's uh you don't
have to know everything
you're never gonna know everything so
think deeply and
stick with it enjoy the ride
but just make sure you're not um
yeah just just make sure you're you're
you're stopping to think about why
things work yeah it's true it's uh it's
easy to lose yourself in the
in the in the distractions of the world
we're overwhelmed with content right now
but
you have to stop and pick some of it and
really understand
yeah on the book point i've read
animal farm by george orwell a
ridiculous number of times so for me
like that book i don't know if it's a
good book in general but for me it
connects deeply somehow
uh it somehow connects
so i was born in the soviet union so it
connects to me to the entire
history of the soviet union and to world
war ii
and to the love and hatred and suffering
that went on there
and the uh the
corrupting nature of power and greed and
just somehow i just that that that book
has taught me more about life than like
anything else
even though it's just like a silly like
childlike book about
adam pigs it's like i don't know why it
just connects and inspires
uh the same there's a few um
yeah there's a few technical books too
and algorithms that just
yeah you return too often right i'm i'm
i'm with you
uh yeah there's uh i don't and i've been
losing that
because of the internet i've been like
uh going and
i've been going on archive and blog
posts and github
and and the new thing and of um
you lose your ability to really master
an idea right
wow exactly right what's a fond memory
from childhood
when baby russ tedrick
well i guess i just said that um
at least my current life begins began
when i got to mit
if i have to go farther than that yeah
what was was there life before mit
oh absolutely but but let me
actually tell you what happened when i
first got to mit because that
i think might be relevant here but i
yeah you know i i had taken a computer
engineering degree at michigan
i enjoyed it immensely learned a bunch
of stuff i was i liked computers i
liked how to like programming
um but when i did get to mit and started
working with sebastian sung
theoretical physicist computational
neuroscientist
um the culture here was just different
um it demanded more of me certainly
mathematically
and in the critical thinking and i
remember
the day that i uh borrowed one of the
books from
my advisor's office and walked down to
the charles river and was like
i'm getting my butt kicked you know um
and i think that's going to happen to
everybody who's doing this kind of stuff
right i think uh
i expected you to ask me the meaning of
life you know i think that the uh
um somehow i think that's that's gotta
be part of it
this doing hard things
yeah did you uh did you consider
quitting at any point did you consider
this isn't for me
no never that i mean i was
it was working hard but i was loving it
right there's
i think the there's this magical thing
where you
you know i'm lucky to surround myself
with people that basically
almost every day i'll i'll i'll
see something i'll be told something or
something that i realize wow i don't
understand that and if i could just
understand that there's there's
something else to learn that if i could
just learn that thing
i would connect another piece of the
puzzle and and uh
you know i think that is just such an
important
aspect and being willing to
understand what you can and can't do and
and
loving the journey of going and learning
those other things
i think that's the best part i don't
think there's a
better way to end it or us i've um
you've been an
inspiration to me since i showed up at
mit
uh your work has been an inspiration to
the world this conversation was amazing
i can't wait to see what you do next
with robotics home robots i
i hope to see you work in my home one
day thanks so much for talking today has
been awesome cheers
thanks for listening to this
conversation with rasteric and thank you
to our sponsors
magic spoon serial betterhelp and
expressvpn
please consider supporting this podcast
by going to magicspoon.com
lex and using codelex at checkout going
to betterhelp.com
lex and signing up at expressvpn.com
lexpod click the links buy the stuff
get the discount it really is the best
way to support this podcast
if you enjoy this thing subscribe on
youtube review it with five stars and
apple podcast
support on patreon or connect with me on
twitter at lex friedman spelled
somehow without the e just f-r-i-d-m-a-n
and now let me leave you with some words
from neil degrasse tyson
talking about robots in space and the
emphasis we humans put
on human-based space exploration
robots are important if i don my pure
scientist hat
i would say just send robots i'll stay
down here and get the data
but nobody's ever given a parade for a
robot nobody's ever named
a high school after a robot so when i
down my public educator hat
i have to recognize the elements of
exploration that excite people
it's not only the discoveries and the
beautiful photos that come down from the
heavens
it's the vicarious participation in
discovery itself
thank you for listening and hope to see
you next time
you