Kind: captions
Language: en
today I'd like to talk about the state
of the art of autonomous vehicles how I
see the landscape how others see the
landscape what we're all excited about
ways to solve the problem and what to
look forward to in 2019 as we also get
to hear from the different perspectives
and the various leaders in industry and
autonomous vehicles in the next few next
couple of weeks and next few days so the
problem the mission the dream the thing
that we're trying to solve for many may
be about entrepreneurial possibilities
of making money and so on but really
it's about improving access to mobility
moving people around in the world that
don't have that ability whether it has
to do with age or purely access of where
you live we want to increase the
efficiency of how people move about the
ability to be productive in the in the
time we spend in traffic and
transportation one of the most hated
things in terms of stress emotion the
thing in our lives
that if we could just with a snap of a
finger remove as traffic so the ability
to convert that into efficiency into a
productive aspect into a positive aspect
of life and really the most important
thing at least for me and for many of us
working in the space is to save lives
prevent crashes the lead to injuries
prevent crash so the lead to fatalities
here's a counter every 23 seconds
somebody in the world dies in a car auto
crash it should be a sobering it is for
me thing that I think about every single
day you go to bed you wake up you work
on all the deep learning letters all the
all the different papers are publishing
everything we're trying to push forward
is really to save lives at the at the
beginning and at the end that is the
main goal so with that groundwork with
that idea with that base the mission
that we're all working towards from the
different ideas and different
perspectives the I would like to review
what happened in 2018 so first way mo
has done incredible work in deploying
and testing their vehicles and various
dome
and having October reached the mark of
10 million miles German autonomously
which is an incredible accomplishment
it's truly a big step for fully
autonomous vehicles in terms of
deployment and obviously is growing and
growing by day and we'll have we'll have
Drago here from way more to talk about
their work there then on the l2 on the
semi-autonomous side that's the pair
that's the mirror side of this equation
the other incredible number that's
perhaps less talked about is the 1
billion mile mark reached by Tesla in
the semi autonomous driving of autopilot
autopilot is a system that's able to
control its position in the lane Center
itself in the lane it's able to control
the longitudinal movement so not follow
a vehicle when there's a vehicle in
front and so on but the degree of its
ability to do so is the critical thing
here is the ability to do so for many
minutes at a time even hours at a time
especially on highway driving that's the
critical thing and the fact that they've
reached 1 billion with a B miles is an
incredible accomplishment all of that
from the machine learning perspective is
data that's data and all of the
autopilot models are driven with the
primary sensor being a camera that's
computer vision now how does computer
vision work the modern day especially
with the second iteration of auto pilot
hardware there's a neural network
there's a set of neural networks behind
it that's super exciting that is
probably the largest deployment of
neural networks in the world that has a
direct impact on a human life that's
able to decide that's able to make life
critical decisions many times a second
over and over that's incredible you go
from the step of image classification on
image net and you sit there with
tensorflow
and you're very happy they were able to
achieve a 99.3 accuracy with a state of
the art algorithm you
to take from that a step towards there's
a human life your parents driving your
grandparents driving this your children
driving the system and there's a neural
network making the decision of whether
they'll ever live so that one building
mark is an incredible accomplishment and
on the sobering side and from various
perspectives the fatalities there's been
two fatalities that happen in March of
2018 one in the fully autonomous side of
things with uber in Tempe Arizona
hitting a pedestrian and leading to a
pedestrian fatality and on the
semi-autonomous side with Tesla
autopilot the third fatality that Tesla
autopilot led to and the one in 2018 is
in Mountain View California when Tesla
slammed into a divider killing his
driver now the two aspects here that are
sobering and really important to think
about as as we talk about the
progression of autonomous vehicles
proliferation in our world is our
response as a public is from the general
public to the engineers to the media and
so on how we think about these
fatalities and obviously there's a
disproportionate amount of attention
given to these fatalities and that's
something as engineers you have to also
think about that the bar is much higher
on every level in terms of performance
so in order to success as I'll argue in
order to design successful autonomous
vehicles those vehicles will have to
take risks and when the risks don't pan
out the public if the public doesn't
understand the general problem they were
tackling the gold emission that those
risks when they don't with the the risks
that are taken can
have significant detrimental effect to
the progress in this autonomous vehicle
space so that's something we really have
to think about that's our role as
engineers and so on question yeah so the
question was do we know the the rate of
fatalities per mile of vehicle driven
which is at the crudest level how people
think about safety so there's about 8090
a hundred million miles driven in
manually controlled cars at every
fatality so one fatality per depending
on which numbers you look at 80 to 100
million miles and the Tesla vehicle for
example is the fatality is what we could
just take the 1 billion and divided by 3
now there's a it's apples and oranges in
comparison and that's something actually
that we're working on to make sure that
we compare it correctly
compare the the aspects of manual models
that directly are comparable to the
autopilot models so Otto Paulo is a
modern vehicle this much safer Tesla is
a modern vehicle that's much safer than
the general population of manually
driven vehicles Otto Pallas driven on
only a particular kinds of roads on the
highway primarily most of the models the
kinds of people that drive autopilot all
these kinds of factors need to be
considered when you compare the two but
when you just look at the numbers
Tesla autopilot three times safer that
manually driven vehicles but that's not
the right way to look at it and for
anyone that's ever taken a statistics
class three fatalities is not does not
it's not a large number by which to make
any significant conclusions nevertheless
that doesn't stop the media the New York
Times and everybody from responding to a
single fatality which PR and marketing
aspects of these different companies are
very sensitive to which is of course
troubling and concerning for an engineer
that wants to save lives
but it's something that we have to think
about ok 2018 in review continued the
there's been a lot of announcements or
rather actual launches of public testing
of autonomous taxi services so companies
that on public roads have been
delivering real people from one location
to another now there's a lot of caveats
in many of these cases it's very small
scale just a few vehicles in most cases
it's very low speed in a constrained
environment in a constrained community
and almost always really always with a
safety driver there's a few exceptions
for demonstration purposes but there's
always an actual driver in the seat some
of the brilliant folks representing
these companies will speak in this
course is voyage doing it in an isolated
community awesome work they're doing in
villages in Florida optimist ride here
in Boston doing and the community and
Union Point Drive AI in Texas main
mobility expanding beyond Detroit but
really most operation Detroit way mo has
launched its service way more one that's
gotten some publicity in Phoenix Arizona
that neuro doing DRA zero occupancy
deliveries of groceries autonomously so
we didn't say has to be delivering
humans delivering groceries autonomously
uber is quietly or not so quietly
resumed its autonomous vehicle taxi
service testing in Pittsburgh in a very
careful constrained way active after
acquiring Carling yema is in autonomy
has been doing extensive large-scaled
taxi service testing everywhere from
Vegas to Boston here to Pittsburgh and
in Singapore of course Aurora that spoke
here last time the head of Tesla
autopilot launched Aurora and the Chris
Urmson behind this this young upstart
company is doing testing in San
Francisco and Pittsburgh and then Cruz
Kyle will be here to talk from GM is
doing testing in San Francisco Arizona
and Michigan so when we talk about
predictions I'll talk about a few people
predicting when we're going to have
autonomous vehicles and when you
yourself think about what it means when
will they be here
when will autonomous vehicles arise such
that that uber that you call will be
autonomous and not with a populated by a
driver so the the thing we have to think
about is what we think about what how we
define autonomous what that experience
looks like and most importantly in these
discussions we have to think about scale
so we here at MIT
our group MIT human centered autonomous
vehicle we have a fully autonomous
vehicle that people can get in if you
would like and it will give you a ride
in a particular location but that's one
vehicle it's not a service and it only
works on particular roads it's extremely
constrained in some ways it's not much
different than most of the companies
that we were talking about today now
scale here there's a magic number I'm
not sure what it is but for this the
purpose of this conversation says 10,000
where there's a meaningful deployment
when it's truly going beyond that
prototype demo mode to where everything
is under control to where it's really
touching the general population in a
fundamental way scale is everything here
and it starts let's say a 10,000 just to
give you for reference there's 46,000
active viewers in New York City so
that's what 10,000 feels like some - you
know 25 30 % of the uber drivers in New
York City all of a sudden are become
passengers so the predictions I'm not a
marketing PR person so I don't
understand what everybody has to have
make a prediction but they all seem to
although major automakers have made a
prediction of one they'll have a deploy
when they will be able to deploy
autonomous vehicles Tesla has made in in
early 2017 a prediction that will have
on Thomas vehicles 2018 in 2018 they've
now adjusted the prediction to 2019
Nissan Honda Toyota have made prediction
for 2020 under certain constraints in
highway urban Hyundai and Volvo has in
2021 BMW and Ford Ford saying at scale
so a large scale deployment 2021 and
Chrysler 21 and Daimler saying in the
early 20s so there is the the
predictions that are extremely
optimistic that are perhaps driven by
the instinct that the company has to
declare that they're at the cutting edge
of innovation and then there is many of
the leading engineers behind the leading
these teams including Carl and Yama and
Gill Pratt from MIT who in injects a
little bit of caution and grounded ideas
about how difficult it is to remove the
human from the loop of automation so
Carl says that basically teleoperation
kind of gives us analogy of an elevator
you know and the elevators fully
autonomous but there is still a button
to call for help
if something happens and that's how he
thinks about autonomous vehicles even
with greater and greater degree of
automation they're still going to have
to be a human in the loop they're still
going to be a way to contact a human to
get help and Gill Pratt and Toyota and
they're making some analysis was CES
basically saying that the human in the
loop is the fundamental aspect that we
need to approach this problem and
removing the human from consideration
is really really far away and guilt
who's historically and currently is one
of the sort of the great roboticists in
the world that defined a lot of the
DARPA challenges and a lot of our
progress historically speaking up to
this point so they're really the full
spectrum we can think of it as the the
Elon Rodney spectrum of optimism versus
pessimism the Elon Musk who's extremely
bold and optimistic about his
predictions I often connect with this
kind of thinking because sometimes you
have to believe the impossible is
possible in order to make it happen and
then there is Rodney one also one of the
great roboticists the former head of the
of C cell they a a laboratory here is a
little bit on the pessimistic side
so for Elon now fully autonomous vehicle
will be here in 2019 for Rodney the
vehicles are really fully autonomous or
beyond 2050 but there it believes in the
30s there will be a significant a major
city will be able to allocate a
significant region of significant region
of that city where manual driving is
fully banned which is the way he
believes those vehicles could autonomous
vehicle really proliferate when you ban
manually German vehicles in certain
parts and then in the 40s 2045 or beyond
majority of US cities will ban manually
driven vehicles of course the the quote
from Elon Musk in 2017 is that my guess
is that in probably 10 years it will be
very unusual for cars to be built that
are not fully autonomous so we also have
to think about the long tail of the the
fact that many people drive cars that
are 10 years old 20 years old so even
when you have every cars built as fully
autonomous it's still going to take time
for that dissipation of vehicles to
happen and so my own view beyond
predictions to to take a little pause
into the ridiculous and the fun to
explain the view
yes that is me playing guitar in our
autonomous vehicle now the the point of
this ridiculous video and embarrassing I
should've never played it yep okay I
think it's gonna be over soon
now for those of you born in the 90s
that's classic rock so the point I'm
trying to make beyond predictions is
that autonomous vehicles will not be
adopted by human beings in the near term
in the next 10-15 years because they're
safer safety is not going to they may be
safer but that is they're not going to
be so much safer that the that that's
going to be the reason you adopt it's
not gonna be because they get you to the
location faster everything we see with
autonomy is they're going to be slower
until majority of the fleet is
autonomous they're cautious and
therefore slower and therefore more
annoying in the way we think about
actually how we navigate this world we
take risk we drive assertively with
speed over the speed limit all the time
that is not how autonomous vehicles
today operate so there's now gonna get
us there faster and for every promise
every hope that they're going to be
cheaper really there's still significant
investment going into them and there is
not good economics in the near term of
how to make them obviously significantly
cheaper what I think
uber and lyft has taken over the taxi
service because of the human experience
in the same way autonomy will only take
over if not take over BBB adopted by
human beings if it creates a better
human experience if there's something
about the experience that you enjoy the
heck out of this video and many others
that were putting out shows that in the
natural language communication the
interaction with the car the ability of
the car to sense everything you're doing
from the activity of the driver to the
driver's attention and being able to
transfer control back and forth in a
playful way but really in a serious way
also that that's personalized to you
that's really the human experience
the efficiency of the human experience
the richness of the human experience
that is what we need to also saw that's
something you have to think about
because many of the people they'll be
speaking at this class and many of the
people that are working on this problem
are not focused on the human experience
it's a kind of afterthought that once we
saw the autonomous vehicle problem
it'll be fun as hell to be in that car I
believe you first have to make it fun as
hell to be in the car and then solve the
autonomous vehicle problem jointly so in
the language that we're talking about
here there are several levels of
autonomy that are defined from level 0
to level 4 level 0 no automation 4 & 5
level 3 4 & 5 increasing automation so
level 2 is when the driver is still
responsible level 3 4 5 is when there's
less and less responsibility but really
in 3 4 5 there's parts of the driving
where the liabilities on the car so
there's only really two as far as I'm
concerned levels human center autonomy
and full autonomy human centered means
the human is responsible full full
autonomy means the car is responsible
both on the legal side the experience
side and the algorithm side that means
full autonomy does not allow for
teleoperation
so it doesn't allow for the human to to
step in and remotely control the vehicle
because that means the human is still in
the loop it doesn't allow for the ten
second rule that it's going to be fully
autonomous but once it starts warning
you you only you have ten seconds to
take over no it's not fully autonomous
we cannot guarantee safety in any
situation it has to be able to if the
driver doesn't respond in ten seconds
has to be able to find safe harbor it
has to be able to pull off to the side
of the road without hurting anybody else
to find safety so that that's the fully
autonomous challenge and so how do we
envision these two levels of automation
proliferating society getting deployed
at a mass scale the ten thousand ten
million beyond on the fully autonomous
side
the way to think about it with the
predictions that we're talking about
here is there several different
possibilities of how to deploy these
vehicles one is less mild delivery of
goods and services like the groceries
these are zero occupancy vehicles
delivering groceries or delivering human
beings at the last mile what the last
mile means is it's slow-moving transport
to the destination where most of the
tricky driving along the way is done
manually and then the last mile delivery
in the city in the urban environment is
done by zero occupancy autonomous
vehicles trucking on the highway
possibly with platooning where a
sequence of trucks follow each other so
in this what people think about it as a
pretty well-defined problem of highway
driving with lanes well marked well
mapped routes throughout the United
States and globally on the highway
driving is automatable the specific
urban routes kind of like what a lot of
the these companies are working on
defining this taxi service and
personalized public transport does you
get this certain pickup locations you
log to go to there are certain drop-off
locations that's it it's kind of like
taking the train here but as opposed to
getting on the train with a hundred
other people you're getting or bus
you're getting on the car with when
you're alone with one other person the
closed communities something
Oliver Cameron with voyage is working on
defining and defining and optimist ride
defining a particular community that you
now have a monopoly over that you define
the constraint defining the customer
base and then you just deliver the
vehicles you map the entire road you
have slow-moving transport that gets
people from A to B anywhere in that
community the and then there's the world
of zero occupancy ride-sharing delivery
so the uber that comes to you as opposed
to having you drive it yourself and it
comes to you autonomously with nobody in
there and then you get in
drive it to imagine a world where we
have empty vehicles driving around
delivering themselves to you
semi-autonomous side is
thinking about a world where
teleoperation plays a really crucial
role where there's it's fully autonomous
under certain constraints in the highway
but a human can always step in high
autonomy on the highway kind of like
what Tesla is working towards most
recently it's on ramp to off-ramp now
the driver is still responsible a lot of
liability wise and in terms of just
observing the vehicle and
algorithmically speaking but the
autonomy is pretty high level to a point
where much of the highway driving could
be done fully autonomously and low
autonomy under strict to travel as an
driver assistance advanced driver
assistance system meaning that the car
kind of like the Tesla the Volvo s90 SR
the super cruising and the Cadillacs all
these kinds of l2 systems that are able
to keep you in the lane you know 10 to
30% of the miles that you drive in some
fraction of the time me take take some
of the stress of driving off and then
there is some out there ideas right the
idea of connected vehicles vehicle to
vehicle communication and vehicles
infrastructure communication enabling us
to navigate for example intersection
efficiently without stopping removing
all traffic lights so here shown on the
bottom is our conventional approach of
there's a queuing system that that forms
because of traffic lights that turn red
green yellow and with without traffic
lights and with communication to the
infrastructure in between the vehicles
you can actually optimize that to
significantly increase the traffic load
through a city of course there's the the
boring solution
of tunnels under cities layers of
tunnels under cities tunnels all the way
down autonomous vehicles basically by
the design of the tunnel constraining
the problem to such a degree that at a
time I mean the idea of autonomy just is
completely transformed that you're
basically a car is able to transform
itself into a mini train into many
public transit entity for a particular
period of time so you get into that
tunnel you drive at 200 miles an hour
and or not not necessarily drive be
driven 200 miles an hour and then you
get out of the tunnel of course there's
the flying cars personalized flying car
vehicles I will not I mean
Rodney as I mentioned before
believe that we'll have them in 2050
there's a lot of people that are
seriously actually thinking about this
this problem is there's a level of
autonomy obviously that's required here
for a regular person like I don't know
somebody without a pilot's license for
example to be able to take off and land
you know making that experience
accessible to regular people means that
there's going to be a significant amount
of autonomy involved one of the people
really want one of the companies really
seriously working on this is uber with
the uber elevate uber air I think it's
called and the idea is that you would
meet your vehicle not on the street but
other roof and take it elevator you meet
them at the roof of the of a building
this is this videos from from uber
they're seriously addressing this
problem many of the great solutions to
the world's problems have been laughed
at at some point so that's not a that's
not laugh too loud and these
possibilities back in my day we used to
drive in the street okay so aha 10,000
vehicles if that's the bar I sort of out
of curiosity asked did a little public
poll 3,000 people who responded asked
who who will be first to deploy 10,000
fully autonomous cars operating on
public roads without a safety driver and
several options percolated with Tesla
getting 50% 57% of the vote and way more
gaining 21 percent of the vote and 14
percent someone else and 8% the the
curmudgeons and the engineers saying no
one in the next 50 years will do it and
again in 1998 when Google came along the
leaders of the space were asked jeez and
the Infoseek and excite all services
I've used them probably some people in
this room have used like oh yeah Yahoo
obviously there were the leaders in the
space and Google disrupted that space
completely
so this poll shows the current leaders
but it's wide open to ideas and that's
why there's a lot of autonomous vehicle
companies some companies are taking
advantage of the hype and the fact that
there's a lot of investment in the space
but some companies like some of the
speakers visiting in this course are
really trying to solve this problem they
want to be the next Google the next
billion multi-billion next trillion
dollar company by solving the problem so
it's wide open but currently Tesla with
a human with the semi autonomous vehicle
approach working towards trying to
become fully autonomous and the way most
starting with the fully autonomous
working towards achieving scale at the
fully autonomous are the leaders in the
space given that ranking in 2019 let's
take a quick step back to 2005 with the
DARPA challenge when the story began to
race to the desert when Stanley from
Stanford won a race to the desert that
really captivated people's imagination
about what's possible and a lot of
people have said that the autonomous
vehicle problem is solved in 2005 they
really said you know the idea was
especially because in 2004 nobody
finished that race 2005 four cars
finished a race it was like well we
cracked it this is it and then you know
some critics said that urban driving is
really nothing comparable to to desert
driving doesn't it's very simple there's
no obstacles and so on it's really a
mechanical engineering problem it's not
a software problem it's not a
fundamentally it's not really an
autonomous driving problem as it would
be delivered to consumers and of course
in 2007 the DARPA put together urban
Grand Challenge and several people
finish that with CMU's boss winning and
so the thought was at that point that's
it we're done as our Ernest Rutherford a
physicist said that physics is the only
real science the rest is just stamp
collecting all the biology chemistry
certainly boy I wouldn't want to know
what he thinks about computer science
it's just all this stupid silly details
physics
the fundamentals and that was the the
idea in with the DARPA Grand Challenge
and solving that that we solved the
fundamental problem of autonomy and the
rest is just for industry to figure out
some of the details of how to make an
app and make a business out of it so
that that could be true in the
underlying beliefs there's that driving
is an easy task that you know it's it's
solvable the thing that we do as human
beings that it's pretty formalize able
it's pretty easily it's as easy to solve
with autonomy that the other idea is
that humans are bad at driving this is a
common belief not me not you but
everybody else nobody in this room but
everybody else is a terrible driver this
the kind of intuition that we have about
our experience of traffic leads us to
believe that humans are just really bad
at driving and from the human factors
psychology side there's been over 70
years of recent years of research
showing that humans are not able to
monitor maintain vigilance monitoring a
system so when you put a human in a room
with a robot and say watch that robot
they they start texting like 15 seconds
in so that's the fundamental psychology
there's thousands of papers on this
people are they tuned out that over
trust the system they misinterpret the
system and they lose vigilance those are
the three underlying beliefs if very
well could be true but what if it is not
so we have to consider that it is not
the driving task is easy because if you
think the driving task is easy and
formalized and solved by autonomous
vehicles you have to solve this problem
the subtle vehicle-to-vehicle vehicles
with the pedestrian nonverbal
communication that happens here in a
dramatic sense but really happens in the
subtle sense millions of times every
single day in Boston subtle nonverbal
communication between vehicles you go no
you go you have to solve all the crazy
road conditions where in a split seconds
you have to make
a decision about so in snowy icy weather
rain limited visibility conditions you
have a hundred 200 milliseconds to make
a decision your algorithm based on the
perception has to make a control
decision and then you have to deal with
a nonverbal communication with
pedestrians the these are unreasonable
irrational creatures us human beings you
have to not only understand what they're
the intent of the movement that they're
that's this anticipated so anticipating
the trajectory the pedestrian you also
have to assert yourself in a game
theoretic way as crazy might sound you
have to threaten yourself you have to
take a risk you have to take a risk that
if I don't slow down like that ambulance
didn't slow down that the pedestrian
will slow down algorithmically we're
afraid to do that the the idea that a
pedestrian that's moving
we anticipate their trajectory based on
the simple physics of the current
velocity of the momentum they're going
to keep going with some probability the
fact that by us accelerating we might
make that pedestrian stop it's something
that we have to incorporate into
algorithms and we don't today so that
and we don't know how to really so if
driving is easy we have to solve that
too and of course the thing I showed
yesterday with the coast runners and the
boat going around and all the ethical
dilemmas from the moral machine to the
more serious engineering aspects that
from the unintended consequences that
arise from having to formalize the
objective function under which a
planning algorithm operates if there's
any learning that as I showed yesterday
a boat on the left or him by a human
wants to finish the race the boat on the
right figures out there's this enough to
finish the race it can pick up turbos
along the way and gets watch more reward
so if the objective function is to
maximize the reward you can slam into
the wall over and over and over again
and that's actually the way to optimize
the reward and those are the unintended
consequences of an algorithm that has to
be formalized able to the objective
function without a human in the loop
humans are bad at driving as I showed
yesterday humans if they're bad at
anything it's about having a good
intuition about what's hard and what's
easy the fact that we have 540 million
years worth of data on our visual
perception system means we don't
understand how damn impressive it is to
be able to perceive and understand the
scene in a split second maintain context
maintain an understanding of performing
all the visual localization tasks about
anticipating the the physics of the
scene and so on and then there's a
control side the humans don't give
enough credit to ourselves we're
incredible a state state-of-the-art
soccer player on the left and the
state-of-the-art robot on the right I
think I think there's like four or five
times you scores it all right and this
is all the movement and so on of all do
that of course here that's the human
robot that's a really incredible work
that's done for the DARPA Robotics
Challenge with the humanoid robots on
the right and incredible work by the the
human people doing the same kind of
tasks much more impressive task I would
say so that's where we stand and the
ones on the right are actually not fully
autonomous there's still some human in
the loop there's just a noisy broken
communication so that humans are
incredible
in terms of our ability to understand
the world and in terms of our ability to
act in that world and the the fact that
humans the idea the view the popular
view grounded in the psychology that
humans and automations don't don't mix
well over trust misunderstanding loss of
visual vigilance that command and so on
that's not an obvious fact it happens a
lot in the lab most of the
pyramids are actually in the lab this is
the difference you put you put a many of
you and uh you put a undergrad grad
student in a lab and say here watch this
screen and wait for the dot to appear
they'll to not immediately but when it's
your life and you're on the road it's
just you in the car it's a different
experience it's not completely obvious
the vigilance will be lost and it's not
a complete when it's just you and the
robot it's not completely obvious what
the psychology what the attentional
mechanism with the vigilance that it
looks like so one of the things we did
is we instrumented here 22 Tesla's and
observed people now over a period of two
years of what they actually do when
they're driving on a pilot driving these
systems in red shown manually controlled
vehicles and Sian showed vehicle control
autopilot now there's there's a lot of
details here and with a lot of
presentations on this Barilla the
fundamentals are is that they drive 34%
large percentage of the miles in
autopilot and in twenty six thousand
moments of transfer of control there you
are always vigilant there's not a moment
once in this data set where they respond
too late to to a critical situation to a
challenging role situation now the data
set twenty two vehicles
that's a 0.1 percent or less than the
full Tesla fleet that has auto polity
but it's still an inkling it's not
obvious that it's not possible to build
a system that works together with a
human being and that system essentially
looks like this
some percentage ninety percent maybe
less maybe more when it can solve the
problem of autonomous driving it solves
it and when he needs human help it asks
for help that's the trade-off that's the
balance on the fully autonomous side on
the right it has to solve here with
citations and there's reference is
always on the bottom all the problems
have to be solved exceptionally
perfectly from mapping localization to
the scene perception to control to play
to being able to find safe harbor at any
moment to also being able to do external
HMI communication with the other
pedestrians the vehicles in the scene
and then there's teleoperation vehicle
to vehicle vehicle to I you have to
solve those perfectly if you want to
solve the fully autonomous problem as I
said including all the crazy things that
happen in driving and if you approach
the shared autonomous side the semi
autonomous where you're only responsible
for a large percentage but not a hundred
percent of the driving then you have to
solve the human side the human
interaction the the sensing what the
driver is doing the collaborating
communicated with the driver and the
personalization aspect that learns with
the driver
I'll like weave as I said you can go
online we have a lot of demonstrations
these kinds of ideas but the natural
language the communication I think is
critical for all of us as we're tweeting
as all of us do so it's as simple as so
this is just demonstration of Eco taking
control when the attention over time and
that the driver is being just okay we
got it thank you okay so basically a
smartphone use which has gone up year by
year and we're doing a lot of analysis
on that it's really what people do in
the car is they use the phone
whether it's manual or autonomous
driving or semi autonomous driving so
being able to manage that to communicate
with the driver about when they should
be paying attention which may not be
always you sort of balancing the time
one is it's a critical time to pay
attention when it's not and
communicating effectively learning with
the driver that problem is a fundamental
machine learning problem there's a lot
of data visible light everything about
the driver and it's a psychology problem
so we have data we have human
complicated human beings and it's a
human robot interaction problem that
deserves solving but as you'll hear on
the beyond the human side looking
looking out into the world people that
are trying to solve the fully autonomous
vehicle it's really a to approach
consideration one approach is vision
cameras and deep learning right collect
a huge amount of data so cameras have
this aspect that they they're the
highest resolution of information
available it's rich texture information
and there's a lot of it which is exactly
what you know networks love right so to
be able to cover all the crazy edge
cases the the vision data camera data
visible light data is the exactly the
kind of data you need to collect a huge
amount of to be able to generalize over
all the crazy countless edge cases that
happen
it's also feasible all the major data
sets all the in terms of cost interest
scale all the major data sets of visible
light cameras that's another Pro and
they're cheap and the world as it
happens whoever designed the simulation
the we're all living in made it such
that our the our world our roads and our
world is designed for human eyes so eyes
is the way we perceive the world and so
the lane mark is also on is visual most
of the road textures that you use to
navigate to drive are visible are are
made for human eyes the cons are that
without a ton of data and we don't know
how much the they're not accurate you
make errors because driving is
ultimately about 99.99999% accuracy and
so that's what I mean by not accurate
you do it's really difficult to reach
that that that level and then the second
approach is lidar taking a very
particular constraint set of roads
mapping the heck out of them
understanding them fully in a different
weather conditions and so on and then
using the most accurate sensors
available a sweetest one sensors but
really lidar at the forefront being able
to localize yourself effectively the
pros there that it's consistent
especially when machine learning is not
involved it's consistent and reliable
and it's explainable if it fails you can
understand why you can account for those
situations it's not so much true for
machine learning methods it's not so
much explainable why it fails in a
particular situation the accuracy is
hires we'll talk about the cons of lidar
is that it's expensive and most of the
approaches in perceiving the world using
lidar primarily are not deep learning
based and therefore they're not learning
over time and if they were deep learning
based there's a reason they're not it's
because you need a lot of car you don't
need a lot of lidar data and there's
only a tiny percentage of cars in the
world
quite obviously are equipped with lidar
in order to collect that data so quickly
running through the sensors radar is the
it's kind of like the offensive line of
football they're actually the ones that
do all the work and they never get the
credit
so radar is that it's it's always behind
to catch to actually do the detection in
terms of obstacle the most critical
safety critical obstacle avoidance it's
cheap it does extremely well and it does
well in extreme weather but it's lower
resolution so it's cannot stand on its
own as an to achieve any kind of degree
of high autonomy now on the lighter side
it's expensive it's extremely accurate
depth information 3d cloud point cloud
information its resolution is much
higher than radar but still lower than
visible light and there is depending on
the sensor or 360 degree visibility
that's built in so there's a difference
in resolution here visualized lighter on
the right radar on the left the
resolution is just much higher and is
improving and the cost is going down and
so on now on the camera side is cheap
everybody got one the resolution is
extremely high in terms of the the
amount of information transferred per
frame and everybody you know really the
scale of the the number of vehicles that
have this equipped is humongous so the
it's it's ripe for application of deep
learning and the challenge is its noisy
it's bad a depth estimation and it's not
good in extreme weather so if we kind
use this plot to look to compare these
sensors to compare these different
approaches so lidar works in the dark
variable lighting conditions that's
pretty good resolution has pretty good
range but it's expensive it's huge and
it doesn't provide rich textural
contrast information and it's also
sensitive to fog and rain conditions now
ultrasonic sensors catch a lot of those
problems they're better at detecting
proximity the their their high
resolution in objects that are close
which is why they're often used for
parking but they can still also be
integrated in the sensor fusion package
for an autonomous vehicle they they
really catch a lot of the problems that
radar has they complement each other
well and radar cheap tiny detect speed
and has pretty good range but has
terrible resolution there's very little
information being provided and then
cameras a lot of rich information there
are cheap there's small range is great
the best range actually of all the
sensors and works in bright conditions
but doesn't work in the dark in extreme
conditions and it's just susceptible to
all these kinds of problems and doesn't
detect speed unless you do some tricky
structure from motion kind of things so
here's where sense of fusion steps in
and you the everybody works together to
build an entire picture if you do that's
how this plot works you can stack it on
top of each other so if you look at a
suite that for example Tesla is using
which is ultrasonic radar and camera and
you compare it to just lidar and see how
these paths compare that actually the
suite of camera radar and ultrasonic are
comparable to lidar so that those are
the two comparisons that we have you
have the costly non machine-learning way
of lidar and you have the
but needs a lot of data and is not
explainable reliable in the near term
vision based approach and those are the
two competing approaches
now of course huevos will talk about
they're trying to use both but
ultimately the question is who catches
who is the failsafe in the semi
autonomous way when there's a camera
based method the human is the failsafe
when you say oh crap I don't know what
to do the human catches in the fully
autonomous mode so what way Mo's working
on and others the failsafe is lidar
failsafe is maps that you can't rely on
the human but you know this roads so
well that if the camera is freaked out
if there's any of the sensors freaked
out that you're able to you have such
good Maps you have such good accurate
sensors that the fundamental problem of
obstacle avoidance which is what safety
is about is can be solved the question
is what kind of experience that creates
in the meantime as the people debate try
to make money start companies there's
just lots of data Ford f-150 still the
most popular car in America manually
driven cars are still happening so
there's a lot of data happening semi
autonomous cars every company is now
releasing more and more semi autonomous
technology so that's all data and what
that boils down to is the two paths
they're walking towards his vision
versus lidar L 2 versus L 4 semi
autonomous this is fully autonomous
Tesla and the semi autonomous front has
reached 1 billion miles weigh mode the
leader on the autonomous front as me ten
million miles the pros and cons I've
outlined them one Division one though
and I'm obviously very excited about and
and machine learning researchers excited
about which fundamentally relies on huge
data and deep learning the the neural
networks that are running inside the
Tesla and with their new as they its
kind of the same kind of path as Google
is taking from the GPU to the GPU
Tesla's taking from
Vidia drive px2 system sort of more
general GPU based system to creating
their own ASIC and having a ton of
awesome neural networks running on their
car that kind of path that others are
beginning to embrace is really
interesting to think about for machine
learning region years and then people
that are maybe more grounded and
actually wanna a really value safety
reliability and serve from the
automotive world I thinking what we need
machine learning is not explainable it's
difficult to work with is it's it's it's
not reliable and so in that sense we
have to have a sense of suite that
extremely reliable those are the two
paths yep question the the question is
there's all kinds of things you need to
perceive stop signs and traffic lights
pedestrians and so on some of them if
you hit them it's a problem some of them
are a bag flying through the air and all
have different visual characteristics
all have different characteristics for
all the different sensors some so lidar
can detect of solid-body objects camera
is better detecting as last year sascha
our new talked about I think fog or
smoke these are interesting things they
might look like an object to certain
sensors or not to others but the the
traffic light detection problem luckily
is with VIP with cameras is it's pretty
solved at this point so that that's the
that's luckily the easy part the hard
part is when you have a green light and
there's a drunk drugged drowsy or
distracted the four DS that nits online
pedestrian trying to cross what to do
that's that's the hard part so the road
ahead for us as engineers the science is
the thing I'm super excited about the
possibility of artificial intelligence
having a huge impact is taking this step
from having these even if they're large
toy datasets toy problems toy benchmarks
of imagenet classification in cocoa all
the all the exciting deep RL stuff that
we'll talk about
future weeks really our toy examples the
game of go and chess and so on but
taking those algorithms and putting them
in cars where they can save people's
lives and they actually directly touch
and impact our entire civilization
that's actually the defining problem for
artificial intelligence in the 21st
century is AI that touches people in a
real way and I think cars autonomous
vehicles is one of the big ways that
that happens we get to deal with the
psychology the philosophy the sociology
aspects of it how we associate think
about it to the robotics problem to the
perception problem it's a fascinating
space to explore and we have many guest
speakers exploring that different ways
and that's really exciting to see how
these people are trying to change the
world so with that I'd like to thank you
very much go to deep learning that MIT
died edu and the code is always
available online
[Applause]
[Music]
you