Kind: captions
Language: en
the following is a conversation with
Vijay Kumar he's one of the top
roboticist in the world a professor at
the University of Pennsylvania a Dean
Afeni engineering former director of
grasp lab or the general robotics
automation sensing in perception
laboratory a pen that was established
back in 1979 that's 40 years ago Vijay
is perhaps best known for his work in
multi robot systems robot swarms and
micro aerial vehicles robots that
elegantly cooperate in flight under all
the uncertainty and challenges that the
real-world conditions present this is
the artificial intelligence podcast if
you enjoy it subscribe on YouTube give
it five stars and iTunes supported on
patreon
simply connect with me on Twitter at lex
friedman spelled fri d
ma an and now here's my conversation
with Vijay Kumar
what is the first robot you've ever
built over a part of building way back
when I was in graduate school I was part
of a fairly big project that involved
building a very large hexapod suede
close to 7,000 pounds and it was powered
by hydraulic actuation or was actuated
by hydraulics with 18 motors hydraulic
motors
controlled by an Intel 80-85 processor
and an internal 8086 coprocessor and so
imagine this huge monster that had 18
joints each controlled by an independent
computer and there was a 19th computer
that actually did the coordination
between these 18 joints so as part of
this project and my thesis work was how
do you coordinate the 18 legs and in
particular the the pressures and the
hydraulic cylinders to get efficient
locomotion it sounds like a giant mess
so how difficult is it to make all the
motors communicate presumably you have
to send signals hundreds of times a
second or at least this was not my work
but the folks who worked on this wrote
what I believe to be the first
multiprocessor operating system this was
in the 80s and you have to make sure
that obviously messages got across from
one joint to another you have to
remember the the clock speeds on those
computers were about half a megahertz
right the eighties so not to romanticize
the notion but how did it make you feel
to make to see that robot move it was
amazing
in hindsight it looks like well we built
this thing which really should have been
much smaller and of course today's
robots are much smaller you look at you
know Boston Dynamics or ghost robotics
has been off from from Penn but back
then you're stuck with the substrate you
had the compute you had so things were
unnecessarily big but at the same time
and this is just human psychology
somehow bigger means grander you know
people never had the same appreciation
for nanotechnology or nano devices as
they do for the space shuttle or the
Boeing 747 yeah you've actually done
quite a good job at illustrating that
small is beautiful in terms of robotics
so what is on that topic is the most
beautiful or elegant robe
in motion that you've ever seen not to
pick favorites or whatever but something
that just inspires you that you remember
well I think thing that I'm I'm most
proud of that my students have done is
really think about small UAVs that can
maneuver and constrain spaces and in
particular their ability to coordinate
with each other and form
three-dimensional patterns so once you
can do that you can essentially create
3d objects in the sky and you can deform
these objects on the fly
so in some sense your toolbox of what
you can create has suddenly got enhanced
and before that we did the
two-dimensional version of this so we
had ground robots forming patterns and
and so on so that that was not as
impressive it was not as beautiful but
if you do it in 3d suspended in midair
and you've got to go back to 2011 when
we did this now it's actually pretty
standard to do these things eight years
later but back then it was a big
accomplishment so the distributed
cooperation is where is what Beauty
emerges in your eyes I think beauty to
an engineer is very different from from
Beauty to you know someone who's looking
at robots from the outside if you will
yeah but what I meant there so before we
said that grand is associated with size
and another way of thinking about this
is just the physical shape and the idea
that you can get physical shapes in
midair and have them deform that's
beautiful but the individual components
the agility is beautiful too right
so then how quickly can you actually
manipulate these three-dimensional
shapes and the individual components yes
right oh by the way said UAV unmanned
aerial vehicle what was a good term for
drones UAVs quad copters is there a term
that's then being standardized I don't
know if that is everybody wants to use
the word drones and I often said there's
drones to me is a pejorative word it
signifies something that's that's dumb
the pre program that does one little
thing and
to anything but drones so I actually
don't like that word but that's what
everybody uses you could call it
unpiloted and paladin ah but even
unpiloted could be radio control could
be remotely controls in many different
ways and I think the right word is
thinking about it is an aerial robot you
also say agile autonomous aerial robot
right yeah so agility is an attribute
but they don't have to be so what
biological system because you've also
drawn a lot of inspiration of those I've
seen bees and ants that you've talked
about what living creatures have you
found to be most inspiring as an
engineer instructive in your work in
robotics to me so ants are really quite
incredible creatures right so you I mean
the individuals arguably are very simple
and how they're they're built and yet
they're incredibly resilient as a
population and as individuals they're
incredibly robust so you know if you
take an ant at six legs you remove one
like it still works just fine and it
moves along and I don't know that even
realizes it's lost alike so that's the
robustness at the individual ant level
but then you you look about this
instinct for self-preservation of the
colonies and they adapt in so many
amazing ways you know transcending
transcending gaps and and by just
chaining themselves together when you
have a flood being able to recruit other
team mates to carry big morsels of food
and then going out in different
directions looking for food
and then being able to demonstrate
consensus even though they don't
communicate directly with each other the
way we communicate with each other in
some sense they also know how to do
democracy probably better than what we
do yeah somehow the even democracy is
emergent it seems like all the phenomena
that we see is all emergent it seems
like there's no centralized communicator
there is so that I think a lot is made
about that word
emergen and means lots of things of
different people but you're absolutely
right I think as an engineer you think
about what element elemental behaviors
were primitives you could synthesize so
that the whole looks incredibly powerful
incredibly synergistic the whole
definitely being greater than the sum of
the parts and ants are living proof of
that
so when you see these beautiful swarms
where there's biological systems of a
robots do you sometimes think of them as
a single individual living intelligent
organism so it's the same as thinking of
our human civilization as one organism
or do you still as an engineer think
about the individual components and all
the engineering that went into the
individual components oh that's very
interesting
so again philosophically as engineers
what we want to do is to go beyond the
individual components the individual
units and think about it as a unit as a
cohesive unit without worrying about the
individual components if you start
obsessing about the individual building
blocks and what they do you inevitably
will find it hard to scale up and just
mathematically just think about
individuals things you want a model and
if you want to have ten of those then
you essentially are taking cartesian
products of ten things that makes it
really complicated than to do any kind
of synthesis or design and that high
dimensional space is really hard so the
right way to do this is to think about
the individuals in a clever way so that
at the higher level when you look at
lots and lots of them abstractly you can
think of them in some low dimensional
space so what does that involve for the
individual you have to try to make the
way they see the world as local as
possible and the other thing do you just
have to make them robust to collisions
like you said with the ants if something
fails that the whole swarm doesn't fail
right I think as engineers we do this I
mean you know think about we build
planes will rebuild iPhones
and we know that by taking individual
components well engineered components
with well specified interfaces that
behave in a predictable way you can
build complex systems so that's
ingrained I would I would claim and most
engineers thinking and it's true for
computer scientists as well I think
what's different here is that you want
the individuals to be robust in some
sense as we do in these other settings
but you also want some degree of
resiliency for the population and so you
really want them to be able to
re-establish communication with their
neighbors you want them to rethink their
strategy for group behavior you want
them to reorganize and that's where I
think a lot of the challenges lie so
just at a high level what does it take
for a bunch of which we call them flying
robots to create a formation just for
people when I familiar with robotics in
general how much information is needed
how do you how do you even make it
happen without a centralized controller
so I mean there are a couple of
different ways of looking at this if you
are a purist you think of it as a as a
way of recreating what nature does
so nature forms groups for several
reasons but mostly it's because of this
instinct that organisms have of
preserving their colonies their
population which means what you need
shelter you need food you need to
procreate and that's basically it so the
kinds of interactions you see are all
organic they're all local and the only
information that they share and mostly
it's indirectly is to again preserve the
herd of the flock or the swarm in and
either by looking for new sources of
food are looking for new shelters right
as engineers when we build swarms we
have a mission and when you think of a
mission and it involves mobility most
often it's described in some kind of a
global coordinate system as a human as
an operator as a commander or as a
collaborator I have my coordinate system
and I want the robots to be consistent
with that so I might think of it
slightly differently I might want the
robots to recognize that coordinate
system which means not only do they have
to think locally in terms of who their
immediate neighbors are but they have to
be cognizant of of what the global
environment looks like so if I go if I
say surround this building and protect
this from intruders well they're
immediately in a building centered
coordinate system and I have to tell
them where the building is and they're
globally collaborating on the map of
that building there they're maintaining
some kind of global not just in the
frame of the building but there's
information that's ultimately being
built up explicitly as opposed to kind
of implicitly like nature might correct
correct so in some sense nature is very
very sophisticated but the tasks that
nature solves or needs to solve are very
different from the kind of engineered
tasks artificial paths that we are
Forrester address and again there's
nothing preventing us from solving these
other problems but ultimately through
our impact you want these forms to do
something useful and so you're kind of
driven into this very unnatural if you
will unnatural meaning not like how
nature does setting and it's a little
probably a little bit more expensive to
do it the way nature does because nature
is less sensitive to the loss of the
individual and cost wise in robotics I
think you're more sensitive to losing
individuals I I think that's true
although if you look at the price to
performance ratio of robotic components
it's it's coming down dramatically right
it continues to come down so I think
we're asymptotically approaching the
major where we would get yeah the cost
of individuals will really become
insignificant yeah so let's step back at
a high low of you the impossible
question of what kind of as an overview
what kind of autonomous flying vehicles
are there in general I think the ones
that receive a lot of notoriety are
obviously the military vehicles military
vehicles are controlled by a base
station but have a lot of human
supervision but I have limited autonomy
which is the ability to go from point A
to point B and even the more
sophisticated now sophisticated vehicles
can do autonomous takeoff and landing
and those usually have wings and they're
heavy usually their wings but then
there's nothing preventing us from doing
this for helicopters as well so I mean
there are many military organizations
that have autonomous helicopters in the
same vein and by the way you look at
auto pilots and airplanes and it's it's
actually very similar in fact I can one
interesting question we can ask is if
you look at all the air safety
violations all the crashes that occurred
yeah would there happen if the plane
were truly autonomous and I think you'll
find that
any other cases you know because of
pilot error we made silly decisions and
so in some sense even an air-traffic
commercial air traffic there's a lot of
applications although we only see
autonomy being enabled at very high
altitudes when when the pilot to the the
plane is on autopilot there's still a
role for the human and that kind of
autonomy is you're kind of implying I
don't know what the right word is but
it's a little dumb dumber and it could
be right so so in the lab right course
we could we can we can afford to be a
lot more aggressive and the question we
try to ask is can we make robots that
will be able to make decisions without
any kind of external infrastructure
right so what does that mean so the most
common piece of infrastructure that
airplanes use today is GPS GPS is also
the most brutal form of information if
you have driven in a city tried to use
GPS navigation you know in tall
buildings you immediately lose GPS and
so that's not a very sophisticated way
of building autonomy I think the second
piece of infrastructure they rely on is
communications again it's very easy to
jam communications in fact if you use
Wi-Fi you know that Wi-Fi signals drop
out cell signals drop out so to rely on
something like that is not is not good
the third form of infrastructure we we
use and I hate to call it infrastructure
but but it is that in the sense of
robots it's people so you could rely on
somebody to pilot you right and so the
question you want to ask is if there are
no pilots
there's no communications of any base
station if there's no knowledge of
position and if there's no a priori map
a priori knowledge of what the
environment looks like a priori model of
what might happen in the future can
robots navigate so that is true autonomy
right so that's that's true autonomous
and we're talking about you may
like military applications and drones
okay so what else is there you talk
about agile autonomous flying robots
aerial robots so that's a different kind
of it's not winged it's not big at least
its small so I used the word agility
mostly or at least we're motivated to do
agile robots mostly because robots can
operate and should be operating in
constrained environments and if you want
to operate the way a Global Hawk
operates I mean the kinds of conditions
in which you operate are and very very
restrictive if you go want to go inside
a building for example for search and
rescue or to locate an active shooter or
you want to navigate under the canopy in
an orchard to look at health of plants
or to look for to count to count fruits
to measure the tree the tree trunks
these are things we do by the way as
cool agriculture stuff you've shown in
the past is really alright so in those
kinds of settings you do need that
agility agility and does not necessarily
mean you break records for the hundred
meters - what it really means is you see
the unexpected and you're able to
maneuver in a safe way and in a way that
that gets you the most information about
the thing you're trying to do by the way
you may be the only person who in a TED
talk has used a math equation which is
amazing people should go see what
actually it's very interesting because
the Ted curator Chris Anderson told me
you can't show math and you know I
thought about it but but that's who I am
and that's that's what that's our work
and so I felt compelled to give the
audience a taste for at least some math
so on that point simply what does it
take to make a thing with four motors
fly a quadcopter one of these little
flying robots you know how hard is it to
make it fly how do you coordinate them
four motors what's how do you convert
there's those motors into actual
movement so this is an interesting
question we've been trying to do this
since 2000 it is a commentary on the
sensors that were available back then
the computers that were available back
then and a number of things happened
between 2000 and 2007 one is the
advances in computing which is and so we
all know about Moore's law but I think
2007 was a tipping point the year of the
iPhone the year of the cloud lots of
things happen in 2007 but going back
even further inertial measurement units
as a sensor really matured again lots of
reasons for that certainly there's a lot
of federal funding particularly DARPA in
the US but they didn't anticipate this
boom in I amuse but if you look
subsequently what happened is it every
year every car manufacturer had to put
an airbag in which meant you had to have
an accelerometer onboard and so that
drove down the price to performance
ratio oliver's so I should know this
that's very interesting yeah it's very
interesting the connection there and
that's why research is very it's very
hard to predict the outcomes and again
the federal government spent a ton of
money on things that they thought were
useful for resonators but it ended up
enabling these small UAVs yeah which is
great because I could have never raised
that much money and told you no soul
this project hey we want to build these
small UAVs can you can you actually fund
the development of low-cost dire news so
why do you need an IMU and so so so I
was I'll come back to that but but so in
2007 2008 we were able to build these
and then the question you're asking was
a good one how do you coordinate the
motors to develop this but over the last
10 years everything is commoditized a
high school kid today can pick up a
Raspberry Pi kit and build us all the
low levels functionality is all
automated but basically at some level
you have to drive the motors at the
right rpms the right
velocity in order to generate the right
amount of thrust in order to position it
and orient it in a way that you need to
in order to fly the feedback that you
get is from onboard sensors and the IMU
is an important part of it the IMU tells
you what the acceleration is as well as
what the angular velocity is and those
are important pieces of information in
addition to that you need some kind of
local position or velocity information
for example when we walk we implicitly
have this information because we kind of
know how how would our stride length is
we also are looking at images fly past
our retina if you will and so we can
estimate velocity we also have
accelerometers in our head and we're
able to integrate all these pieces of
information to determine where we are as
we walk and so robots have to do
something very similar you need an IMU
you need some kind of a camera or other
sensor that's measuring velocity and
then you need some kind of a global
reference frame if you really want to
think about doing something in a world
coordinate system and so how do you
estimate your position with respect to
that global reference frame that's
important as well so coordinating the
RPMs of the four motors is what allows
you to first of all fly and hover and
then you can change the orientation and
the velocity of the and so on exactly
exactly bunch of degrees of freedom six
degrees of freedom but you only have
four inputs the four motors and and it
turns out to be a remarkably versatile
configuration you think at first well I
only have four motors how do I go
sideways but it's not too hard to say
well if I tell myself I can go sideways
and then you have four motors pointing
up how do i how do I rotate in place
about a vertical axis
well you rotate them at different speeds
and that generation reaction moments in
that allows you to turn so it's actually
a pretty
it's an optimal configuration from
from engineer standpoint it's it's very
simple very cleverly done and and very
versatile so if you could step back to a
time so I've always known flying robots
as the to me it was natural that the
quadcopter should fly but when you first
started working with it
like how surprised are you that you can
make do so much with the four motors how
surprising is e this thing fly first of
all you can make it hover then you can
add control to it firstly this is not
the four motor configuration is not ours
you could it has at least a hundred year
history and with various people various
people try to get quad rotors to fly
without much success as I said we've
been working on this since 2000 our
first designs were well this is way too
complicated why not we try to get an
omnidirectional
flying robots or so our early designs we
had eight folders and so these eight
rotors were arranged uniformly on a
sphere if you will so you can imagine a
symmetric configuration and so you
should be able to fly anywhere but the
real challenge we had is the strength to
weight ratio is not enough and of course
we didn't have the sensors and so on so
everybody knew or at least the people
who work with rotor crafts knew four
rotors we get it done so that was not
our idea but it took a while before we
could actually do the onboard sensing
and the computation that was needed for
the kinds of agile maneuvering that we
wanted to do in our little aerial robots
and that only happened between 2007 and
2009 in our life yeah and you have to
send the signal may hundred times a
second so the compute there is
everything has to come down in price and
what are the steps of getting from point
A to point B so you just talked about
like local control but if all the kind
of cool
dancing in the air that I've seen you
show how do you make it happen it would
have trajector make a trajectory
first of all okay figure out a
trajectory so planet trajectory and then
how do you make that trajectory happen I
think planning is a very fundamental
problem in robotics I think you know 10
years ago it was an esoteric thing but
today with self-driving cars you know
everybody can understand this basic idea
that a car sees a whole bunch of things
and it has to keep a lane or maybe make
a right turn or switch lanes it has to
plan a trajectory it has to be safe it
has to be efficient so everybody's
familiar with that that's kind of the
first step that that you have to think
about when you when you when you when
you say autonomy and so for us it's
about finding smooth motions motions
that are safe so we think about these
two things one is optimality we want a
safety clearly you don't you cannot
compromise safety so you're looking for
safe optimal motions the other thing you
have to think about is can you actually
compute a reasonable trajectory in a
fast manner in a small amount of time
because you have a time budget so the
optimal becomes suboptimal but in our
lab we we focus on synthesizing smooth
trajectory that satisfy all the
constraints in other words don't violate
any safety constraints and is as
efficient as possible and when I say
efficient it could mean I want to get
from point A to point B as quickly as
possible
or I want to get to it as gracefully as
possible or I want to consume as little
energy as possible but always staying
within the safety constraints but yes
always finding a safe trajectory so
there's a lot of excitement and progress
in the field of machine learning yes and
reinforcement learning and the neural
network variant of that with deep
reinforcement learning DS do you see a
role of machine learning in so a lot of
the successful flying robots did not
rely on machine learning
except for maybe a little bit of the
perception the computer vision side on
the control side and the planning do you
see there's a role in the future for
machine learning so let me disagree a
little bit with you I think we never
perhaps called out and my work called
out learning but even this very simple
idea of being able to fly through a
constrained space the first time you try
it you'll invariably you might get it
wrong even if the task is challenging
and the reason is to get it perfectly
right you have to model everything in
the environment and flying is
notoriously hard to model there are
aerodynamic effects that we constantly
discover even just before I was talking
to you I was starting to a student about
how blades flap when they fly well and
that ends up changing how a rotor craft
is accelerated in the angular direction
this is like micro flaps or something is
smooth it's not microfiber you assume
that each blade is is rigid but actually
it flaps a little bit oh it bends
interesting yeah and so the models rely
on the fact on the on an assumption that
they're they're actually rigid but
that's not true if you're flying really
quickly these effects become significant
if you're flying close to the ground you
get pushed off by the ground right
something which every pilot knows when
he tries to land or she tries to land
this is called a ground effect something
very few pilots think about is what
happens when you go close to a ceiling
well you get sucked into a ceiling there
are very few aircrafts that fly close to
any kind of ceiling likewise when you go
close to close to a wall there are these
wall effects and if you've gone on a
train and you pass another train that's
traveling in the opposite direction you
feel the buffeting and so these kinds of
microclimates effect our UAVs
significantly so impossible to model
essentially if I wouldn't say they're
impossible to model but the level of
sophistication you would need
in the model and the software would be
tremendous plus to get everything right
would be awfully tedious so the way we
do this is over time we figure out how
to adapt to these conditions
so we've early on we use the form of
learning that we call iterative learning
so this idea if you want to perform a
task there are a few things that you
need to change and iterate over few
parameters that over time you can you
can you can figure out so I could call
it policy gradient reinforcement
learning but actually this is iterative
learning learning and so this was their
way back I think what's interesting is
if you look at autonomous vehicles today
learning occurs could occur in two
pieces one is perception understanding
the world second is action taking
actions everything that I've seen that
is successful is on the perception side
of things so in computer vision we've
made amazing strides in the last ten
years so recognizing objects actually
detecting objects classifying them and
and tagging them in some sense
annotating them this is all done through
machine learning on the action side on
the other hand I don't know if any
examples where there are fielded systems
where we actually learn the right
behavior outside a single demonstration
of successfully you know in the
laboratory this is the holy grail can
you do end-to-end learning can you go
from pixels to motor block mode
occurrence this is really really hard
and I think if you look go forward the
right way to think about these things
is data driven approaches learning based
approaches in concert with model-based
approaches which is the traditional way
of doing things by so I think there's a
piece there's a role for each of these
methodologies so what do you think just
jumping out in topic since you mention
autonomous vehicles what do you think
are the limits and the perception sighs
so I've talked to Elon Musk and they're
on the perception side they're using
primarily computer vision to
see the environment in your work with
because you work with a real world a lot
in the physical world what are the
limits of computer vision do you think
you can solve autonomous vehicles focus
on the perception side focusing on
vision alone and machine learning so you
know we also have a spin-off company X
and technologies that that works
underground in mines you go into mines
there they're dark they're dirty you fly
in a dirty area there's stuff you kick
up from by the propellers the downwash
kicks up dust I challenge you to get a
computer vision algorithm to work there
yeah so we used lighters in that setting
indoors and even outdoors when we fly
through fields I think there's a lot of
potential for just solving the problem
using computer vision alone but I think
the bigger question is can you actually
solve or can you actually identify all
the corner cases using a sense single
sensing modality and using learning
alone so what's your intuition there so
look if you have a corner case and your
algorithm doesn't work your instinct is
to go get data about the corner case and
patch it up learn how to deal with that
corner case but at some point this is
going to saturate this approach is not
viable
so today computer vision algorithms can
detect 90% of the objects or can detect
objects 90% of the time classify them
90% of the time cats on the Internet
I probably can do 95 percent on here but
to get from 90% to 99% you need a lot
more data and then I tell you well
that's not enough because I have a
safety critical application I want to go
from 99% to 99.9 percent that's even
more data so I think if you look at
wanting accuracy on the x-axis and look
at the amount of data on the y-axis I
believe that curve is an exponential
curve Wow okay it's even hard if it's
linear
it's hard if it's linear totally but I
think it's exponential and the other
thing you have to think about is that
this process is a very very power hungry
process to run data farms or solar power
you mean literally power literally power
literally power so in 2014 five years
ago and I don't have more recent data
two percent of US electricity
consumption was from data forms so we
think about this as an information
science and information processing
problem actually it is an energy
processing problem and so unless we
figured out better ways of doing this I
don't think this is viable so talking
about driving which is a safety critical
application and some aspect the flight
is safety critical maybe philosophical
question maybe an engineering one what
problem do you think is harder to solve
autonomous driving or autonomous flight
that's a really interesting question I
think autonomous flight has several
advantages that autonomous driving
doesn't have so look if I want to go
from point A to point B I have a very
very safe trajectory go vertically up to
a maximum altitude fly horizontally to
just about the destination and then come
down vertically this is pre-programmed
the equivalent of that is very hard to
find in a self-driving car car world
because you're on the ground you're in a
two-dimensional surface and the
trajectories in the two-dimensional
surface are more likely to encounter
obstacles I mean this in an intuitive
sense but mathematically true that's
mathematically as well that's true
there's another option on the 2g space
of platooning or because there's so many
obstacles you can connect with those
obstacles and all these those exist in
the three-dimensional space is wrong so
they do so the question also implies how
difficult are obstacles in the
three-dimensional space in flight so so
that's the downside I think in
three-dimensional space you're modeling
three-dimensional world not just just
because you want to avoid it but you
want a reason about it
you want to work in the
three-dimensional environment and that's
significantly harder so that's one
disadvantage I think the second
disadvantage is of course anytime you
fly you have to put up with the
peculiarities of aerodynamics and
they're complicated environments how do
you negotiate that so that's always a
problem if you see a time in the future
where there is you mentioned them
there's an agriculture application so
there's a lot of applications of flying
robots but do you see a time in the
future where there is tens of thousands
or maybe hundreds of thousands of
delivery drones they fill the sky a
delivery flying robots I think there's a
lot of potential for the last mile
delivery and so in crowded cities I
don't know if you go if you go to a
place like Hong Kong just crossing the
river can take half an hour and while a
drone can just do it in in five minutes
at most
I think you look at a delivery of
supplies to remote villages I work with
a nonprofit called weave robotics
they work in the Peruvian Amazon where
the only highways are rivers and to get
from point A to point B may take five
hours while with a drone you can get
there in 30 minutes
so just delivering drugs retrieving
samples for for testing vaccines
I think there's huge potential here so I
think if the challenges are not
technological that the challenge is
economical the one thing I'll tell you
that nobody thinks about is the fact
that we've not made huge strides in
battery technology yes it's true
batteries are becoming less expensive
because we have these mega factories
that are coming up but they're all based
on lithium based technologies and if you
look at the energy density and the power
density those are two fundamentally
limiting numbers so power density is
important because for a UAV to take off
vertically into the air which most
drones do they're not they don't have a
runway you consume roughly two and
watts per kilo at the small size that's
a lot right in contrast the human brain
consumes less than 80 watts the whole of
the human brain so just imagine just
lifting yourself into the air is like
two or three lightbulbs which makes no
sense to me yes so you're going to have
to at scale solve the energy problem
then charging the batteries storing the
the energy and so on and then the
storage is the second problem but
storage limits the range but you know
you you you have to remember that you
you have to you have to burn a lot of it
for a given time so the turning which is
the pop which is a power question yes
and do you think just your intuition
there there are breakthroughs in
batteries on the horizon how hard is
that problem look there are a lot of
companies that are promising flying cars
but there are autonomous and that are
clean
I think there over-promising the
autonomy piece is durable the clean
piece I don't think so
there's another company that I work with
called jet otra they make small jet
engines hmm and they can get up to 50
miles an hour very easily and left 50
kilos but their jet engines they're
efficient there are little louder than
electric vehicles but they can bail
flying cars so your sense is that
there's a lot of pieces that have come
together so on this crazy question if
you look at companies like Kitty Hawk
working on electrics of clean I'm
talking to Sebastian Thrun right it's a
it's a crazy dream you know but you work
with flight a lot
you've mentioned before that manned
flights are carrying of the human body
is very difficult to do so how crazy is
flying cars do you think there will be a
day when we have vertical takeoff and
landing vehicles that are sufficiently
affordable that we're going to see a
huge amount of them and they would look
like something like we dream of when we
think about flying cars yeah like the
Jetsons The Jetsons yeah so look there
are a lot of smart people working on
this and you never say something is not
possible when you're people like
Sebastian Thrun working on it
so I totally think it's viable I
question again the electric piece they
let you pee on it and again for short
distances you can do it and there's no
reason to suggest that these are all
just have to be rotorcraft you take off
vertically but the new morph into a
forward flight I think there are a lot
of interesting designs the question to
me is is are these economically viable
and if you agree to do this with fossil
fuels that instinct immediately becomes
viable that's a real challenge do you
think it's possible for robots and
humans to collaborate successfully on
tasks so a lot of robotics folks that I
talk to and work with
I mean humans just add a giant mess to
the picture so it's best to remove them
from consideration when solving specific
tasks it's very difficult to model
there's just a source of uncertainty in
your work with these agile flying robots
do you think there's a role for
collaboration with humans or is it best
to model tasks in a way that that
doesn't have a human in the picture well
I I don't think we should ever think
about robots without human in the
picture ultimately robots are there
because we want them to solve problems
for humans right but there is no general
solution to this problem I think if you
look at human interaction and how humans
interact with robots you know we think
of these in three different ways one is
the human commanding the robot the
second is the human collaborating with
the robot so for example we work on how
a robot can actually pick up things with
a human will carry things that's like
true collaboration and third we think
about humans as by standards so driving
cars what's the human's role and how
does how do self-driving cars
acknowledge the presence of humans so I
think all of these things are different
scenarios it depends on what kind of
humans were kind of tasks and I think
it's very difficult to say that there's
a general theory that we all have for
this but at the same time it's also
silly to say that we we should think
about robots independent of humans so to
me human robot interaction is almost a
mandatory aspect of everything we do yes
so but the Jewish to agree so with your
thoughts if you jump to autonomous
vehicles for example there's a there's a
big debate between what's called level 2
and level 4 so semi autonomous and
autonomous vehicles instead of the Tesla
approach currently at least has a lot of
collaboration between human and machine
so the human is supposed to actively
supervise the operation of the robot the
part of the safety
a definition of how safe a robot is in
that case is how effective is the human
and monitoring it do you think that's
ultimately not a good approach in sort
of having a human in the picture not as
a bystander or part of the
infrastructure but really as part of
what's required to make the system safe
this is harder than it sounds yes I
think you know if you if you if I mean
I'm sure you've driven the driven before
and highways and so on it's it's really
very hard to have to relinquish controls
to a machine and then take over when
needed so I think Tesla's approach is
interesting because it allows you to
periodically establish some kind of
contact with the car Toyota on the other
hand is thinking about shared autonomy
as a collaborative autonomy as a
paradigm if I may argue these are very
very simple ways of human-robot
collaboration because the task is pretty
boring you sit in a vehicle you go from
point A to point B I think the more
interesting thing to me is for example
search-and-rescue I've got a human first
responder robot first responders I got
to do something it's important I have to
do it in two minutes the building is
burning there's been an explosion it's
collapsed how do I do it I think to me
those are the interesting things where
it's very very unstructured and what's
the role of the human was the role robot
clearly there's lots of interesting
challenges and as a field I think we're
gonna make a lot of progress in this
area yeah it's an exciting form of
collaboration you're right in the town
was driving the main enemy is just
boredom of the human yes as opposed to
in rescue operations
it's literally life and death and the
collaboration enables the effective
completion of the mission so exciting in
some sense you know we're also doing
this you think about the human driving a
car and almost invariably the humans
trying to estimate the state of the cars
estimate the state of the environment so
on but what if the car were to estimate
the state of the human
so for example I'm sure you have a
smartphone the smartphone tries to
figure out what you're doing and send
you reminders and often times telling
you to drive to a certain place although
you have no intention of going there
because it thinks that that's why you
should be a cause of some gmail calendar
entry or something like that and and you
know it's trying to constantly figure
out who you are what you're doing if a
car were to do that maybe that would
make the driver safer because the car's
trying to figure out is the driver
paying attention
looking at his or her eyes looking at
saccadic movements so I think the
potential is there but from the reverse
side it's not robot modeling but it's
human modeling it's more in the human
right and I think the robots can do a
very good job of modeling humans if you
if you really think about the framework
that you have a human sitting in a in a
cockpit surrounded by sensors all
staring at him in addition to be staring
out staring outside but also staring at
him I think there's a real synergy there
yeah I love that problem because it's a
new 21st century form of psychology
actually AI enabled psychology a lot of
people have sci-fi inspired fears of
walking robots like those from Boston
Dynamics if you just look at shows on
Netflix and so on or flying robots like
those you work with how would you how do
you think about those fears
how would you alleviate those fears do
you have Inklings echos of those same
concerns you know anytime we develop a
technology meaning to have positive
impact in the world there's always a
worry that you know somebody could
subvert those technologies and use it in
an adversarial setting and robotics is
no exception right so I think it's very
easy to weaponize robots I think we
talked about swarms one thing I worry a
lot about is so you know for us to get
swamps to work and do something reliably
it's really hard but suppose I have
there's this challenge of trying to
destroy something and I have a swarm of
robots well only one out of the swarm
needs to get to its destination so that
suddenly becomes a lot more doable
worry about you know this gentle idea of
using autonomy with lots and lots of
agents
I mean having said that looked a lot of
this technology is not very mature my
favorite saying is that if somebody had
to develop this technology
wouldn't you rather the good guys do it
so the good guys have a good
understanding of the technology so they
can figure out how this technology is
being used in a bad way or could be used
in a bad way and try to defend against
it so we think a lot about that so we
have a were doing research on how to
defend against swarms for example that's
a there's in fact a report by the
National Academies on counter UAS
technologies this is a real threat but
we're also thinking about how to defend
against this and and knowing how swarms
work knowing how autonomy works is I
think very important so it's not just
politicians you think engineers have a
role in this discussion absolutely I
think the days where politicians can be
agnostic to technology are gone
III think every tech politician needs to
be literate in technology and I often
say technology is the new liberal art
understanding how technology will change
your life I think is important and every
human being needs to understand that and
maybe we can elect some engineers to
office as well on the other side what
are the biggest open problems in
robotics and you said we're in the early
days in some sense what are the problems
we would like to solve in robotics I
think there are lots of problems right
but I would phrase it in the following
way if you look at the robots or a
building they're still very much
tailored towards doing specific tasks
and specific settings I think the
question of how do you get them to
operate in much broader settings with
where things can change you know
unstructured environments is up in the
air so you know think of a self-driving
cars today we can build a self-driving
car in a parking lot we can do level
fire autonomy in a parking lot but can
you do level five autonomy in the
streets of Napoli in Italy or Mumbai in
India no no so in some sense when we
think about robotics we have to think
about where they're functioning what
kind of environment what kind of a task
we have no understanding of how to put
both those things together so we're in
the very early days of applying it to
the physical world and I was just
enables actually and that's there's
levels of difficulty in complexity
depending on which area you're applying
it to I think so we don't have a
systematic way of understanding that you
know everybody says just because
computer can now beat a human at any
board game we suddenly know something
about intelligence that's not true a
computer board game is very very
structured it is the equivalent of
working in a Henry Ford factory where
things parts come you assemble move on
it's a very very very structured setting
that's the easiest thing and we know how
to do that
so you've done a lot of incredible work
at the University of Pennsylvania grasp
ah you know Dean of engineering at UPenn
what advice do you have for a new
bright-eyed undergrad interested in
robotics or AI or engineering well I
think there's really three things one is
one is you have to get used to the idea
that the world will not be the same in
five years or four years whenever you
graduate right which is really hard to
do so this this thing about predicting
the future every one of us needs to be
trying to predict the future always not
because you'll be any good at it but by
thinking about it I think you sharpen
your senses and you become smarter so
that's number one number two it's a
corollary of the first piece which is
you really don't know what's going to be
important so this idea that I'm going to
specialize in something which will allow
me to go in a particular direction it
may be interesting but it's important
also to have this breadth so you have
this jumping-off point I think the third
thing and this is where I think Penn
excels I mean we teach engineering but
it's always in the context of the
liberal arts it's always in the context
of society as engineers we cannot afford
to lose sight of that so I think that's
important but I think one thing that
people underestimate when they do
robotics is the important of
mathematical foundations important of
represent importance of representations
not everything can just be solved by
looking for Ross packages on the
internet or to find a deep neural
network that works
I think the representation question is
key even to machine learning where if
you hope ever hope to achieve or get to
explainable AI somehow there need to be
representations that you can understand
so if you want to do robotics you should
also do mathematics and you said liberal
arts a little literature if you want to
build it all I should be reading
Dostoyevsky I agree with that very good
the v-j thank you so much for talking
today was an honor thank you it's just
exciting conversation thank you
you