Kind: captions
Language: en
today we have sterling Anderson he's the
co-founder of Aurora an exciting new
self-driving car company previously he
was the head of the Tesla auto pilot
team that brought both the first the
second generation auto pilot to life
before that he did his PhD at MIT
working on shared human machine control
of ground vehicles the very thing I've
been harping on over and over in this
class and now he's back at MIT to talk
with us please give him a warm welcome
[Applause]
thank you it's good to be here I was
telling Lex just before I think it's
been a little while since I've been back
after the Institute and it's great to be
here I want to apologize in advance I've
just landed this afternoon from Korea
via Germany where I've been spending the
last week and so I may speak a little
slower than normal please bear with me
if I become incoherent or slurred my
speech
somebody flag at 2:00 and Lola will try
to make corrections so tonight I thought
I'd chat with you a little bit about my
journey over the last decade it's been
just over ten years since I was at MIT a
lot has changed a lot has changed for
the better in the self-driving community
and I've been privileged to be a part of
many of those changes and so I wanted to
talk with you a little bit about some of
the things that I've learned some of the
things that I've experienced and then
maybe end by talking about sort of where
we go from here and and what the next
steps are both for you know the industry
at large but also for the company that
we're building that as Lex mention is
called Aurora to start out with and
there are a few sort of key phases or
transitions in my journey over the last
10 years as Lex mentioned when I started
MIT I worked with Carly on Yemma Amelio
Fazoli's John Leonard a few others on
some of these sort of shared adaptive
automation approaches I'll talk a little
bit about those
from there I spent some time at Tesla
where I first led the Model X program as
we both finish the development and
ultimately launched I took over the
autopilot program where we introduced a
number of new both active safety but
also sort of you know enhanced
convenience features from auto steer to
adaptive cruise control that were able
refine in a few unique ways and we'll
talk a little bit about that and then
from there in December of last year of
2016 I guess now we started a new
company called Aurora and I'll tell you
a little bit about that so to start out
with when I KN OIT was 2007 the DARPA
urban challenge is were well underway at
that stage and one of the things that we
wanted to do is find a way to address
some of these safety issues in human
driving earlier than potentially full
self-driving Qadeer and so we developed
what became known as the intelligent
co-pilot what you see here is a
simulation of that operating I'll tell
you a little bit more about that in just
a second but to explain a little bit
about the the methodology the innovation
the key approach that we took that was
slightly different from what in
traditional planning control theory we
were doing was instead of designing in
path space for the robot we instead
found a way to identify plan optimize
and design a controller subject to a set
of constraints rather than paths and so
what we were doing is looking for Hama
top Eastern environment so imagine for a
moment an environment that's pockmarked
by objects by their vehicles by
pedestrians etc if you were to create
the Voronoi diagram through that
environment you would have a set of each
unique set of paths or Hama top is
continuously deformable paths that will
take you from one one location to
another through it
if you then turn that into its dual
which is the de'longhi triangulation of
set environment presuming that you've
got convex obstacles you can then tile
those together rather trivially to
create a set of homotopy sand
transitions across which those paths can
can stake out sort of a given set of
options for the human eye turns out
humans tend to this tends to be a more
intuitive way of imposing certain
constraints on human operation rather
than enforcing that the ego vehicle
stick to some arbitrary position within
you know some distance of a safe path
you instead look to enforce only that
the that the state of the vehicle remain
within a constraint bounded and
dimensional tube in state space those
constraints being spatial imagine for a
moment edges of the roadway or you know
circumventing various objects in the
roadway imagine them also being dynamic
right so limits of tire tire friction
imposed limits on side slip angles and
so using that what we did is found a way
to create those Hammurabi's forwards
simulate the trajectory of the vehicle
given its current state and some optimal
set of controls inputs that would
optimize its stability through that we
use model creative control in that work
and then taking that forward simulated
trajectory computing some metric of
threat for instance if the objective
function for that minimize the or
maximize stability or minimize some some
of these parameters like wheel side slip
then wheel side slip is a fairly good
indication of how threatening that
optimal maneuver is becoming and so what
we did is then use that in a modulation
of control between the human and the car
such that should the car ever find
itself in a state where that forward
simulated optimal trajectory is very
near the limits of what the vehicle and
it can actually handle we will have
transition control fully to the to the
vehicle to the automated system so that
it can avoid an accident
and then it transitions back in some
manner and we played with a number of
different methods of transitioning this
control to ensure that that we didn't
throw off the human mental model which
was which was one of the key concerns we
also wanted to make sure that we were
able to arrest accidents before they
happen what you see here is a simulation
that was fairly faithful to the behavior
we saw in test drivers up at Dearborn in
Dearborn Michigan Ford provided was
provided us with a Jaguar s-type to test
this on and what we did so what you see
here is there's a blue vehicle in the
gray vehicle both in both cases we have
a poorly tuned driver model in this case
if your pursuit controller with a fairly
short look ahead shorter than would be
appropriate given this scenario in these
dynamics the grey vehicle is without the
intelligent copilot in the loop
you'll notice that obviously the driver
becomes unstable loses control and
leaves the safe roadway the co-pilot
remember is in is interested not in
following any given path it doesn't care
where the vehicle lands on this road why
provided it remains inside the road in
the blue vehicles case it's the exact
same human driver model now with the
copilot in the loop you'll notice that
as as this scenario continues what you
see here on the left is the green is in
this green bar is the portion of
available control authorities being
taken by the automated system you'll
notice that it never exceeds half of the
available control which is to say that
the steering inputs received by the
vehicle end up being a blend of what the
human and what the automation are
providing and what what results is a
path for the blue vehicle that actually
better tracks the humans intended
trajectory then even the copilot
understood right again the copilot is
keeping the vehicle stable it's keeping
it on the road the human is healing to
the centerline of that roadway so there
was some very interesting things that
came out of this there were a lot of we
did a lot of
work in understanding what kind of
feedback was most natural to provide to
a human our biggest concern was if you
throw off a human's mental model by
causing the vehicles at behaviors to
deviate from what they expect it to do
in response to British control inputs
that could be a problem so we tried
various things from you know adjusting
for instance one of the one of the key
questions that we had early on was if we
couple the computer control and the
human control via planetary gear and
allow the human to feel a actually a
backwards torque to what the vehicle is
doing so the car starts to turn right
human will feel the wheel turn left
they'll see it start to turn left
is that more confusing or less confusing
they're human and it turns out it
depends on how experienced a human is
some some drivers will modulate their
input space on the torque feedback that
they feel through the wheel and it for
instance a very experienced driver
expects to feel the wheel pull left when
they're turning right however less
experienced drivers in response to
seeing the wheel turning opposite to
what the what the car supposed to be
doing this for a rather confusing
experience so there were a lot of really
interesting human interface challenges
that we were dealing with here we ended
up working through a lot of that
developing a number of sort of micro
applications for it one of those at the
time Gill Pratt was leading a DARPA
program focused on what they call the
time maximal mobility manipulation we
decided to see what this system could do
in application to unmanned ground
vehicles so in this case what you see is
a human driver sitting at a remote
console as one would when operating an
unmanned vehicle for instance in the
military what you see on the left top
left is the top-down view of what the
vehicle sees I should have played this
in repeat mode with bounding boxes
bounding various cones and what we did
is we set up about 20 drivers 2020 test
subjects looking at this this
troll screen and operating the vehicle
through this track and we set this up as
a race with prizes for the winners as
one would expect and penalize them for
every barrel they hit if they knocked
over the barrel I think they got a
five-second penalty if they brushed a
barrel they got a one-second penalty and
they were to cross they work across the
field as fast as possible they couldn't
they had no line-of-sight connection the
vehicle and we played with some things
on their interface we did you know we
caused it to drop out occasionally we
delayed it as one would realistically
expect in the field and then we either
engaged or didn't engage the copilot to
try to understand what effect that had
on their performance and their
experience and what we found was not
surprisingly the incidence of collisions
declined it climbed by about 72% when
the copilot was engaged versus when it
was not we also found that you know even
with that seventy-two percent decline in
collisions the speed increased by I'm
blanking on the the amount but it was
you know 20 to 30 percentage finally in
perhaps the most interesting to me after
every run I would ask the driver and
again these were blind tests they didn't
know if the copilot was active or not
and I would ask them how much control
did you feel like you had over the
vehicle and I found that there was a
statistically significant increase of
about 12% when the copilot was engaged
in that is to say drivers reported
feeling more control of the vehicle 12%
more of the time when the copilot was
engaged and when it wasn't and then
noticed the statistics it turns out they
actually at the average level of control
the the copilot was taking was 43% so
they were reporting that they felt more
in control when in fact there were 43
percent less in control which was which
was interesting and I think a bears a
little bit on sort of the human psyche
in terms of you know they were reporting
the vehicle was doing what I wanted to
do maybe not what I told it to do which
was which was kind of fun observation
and and fun too I think I think the most
enjoyable part of this was getting
together with the with the whole group
at the end of the study and presenting
some of this and seeing some of the
reaction
so from there you know we looked at a
few other areas my Carl um and I looked
at a few different opportunities to
commercialize this again this was years
ago and the industry was in a very
different place than it is today we
started a company first called gimlet
then another called ride this is the
logo it may look familiar to you we
turned that into we at the time it
intended to roll this out across various
automakers in their operations at the
time very few saw self-driving as a
technology was really gonna impact their
business going forward they were in fact
even even ride-sharing at the time was a
fairly new concept that was I think to a
large degree viewed as unproven so as I
mentioned December of last year i
co-founded aurora with a couple of folks
who have been making significant
progress in this space for many years at
Chris Urmson who formerly led Google's
self-driving car group at drew back now
as a professor at Carnegie Mellon
University exceptional machine learning
in apply machine learning was one of the
founding members of Ober self-driving
car team and led autonomy and perception
there we felt like we had a unique
opportunity at the convergence of a few
things one the automotive world has
really come into the full-on realization
that self-driving and particularly
self-driving and ride-sharing and
vehicle electrification are three
vectors that will change the industry
that was something that didn't exist ten
years ago two significant advances have
been made in you know some of these
machine learning techniques in
particular deep learning and other
neural network network approaches in the
computers that run them and the
availability of you know low-power GPU
and TPU options to really do that well
in sensing technologies
in high-resolution radar and a lot of
the light our development so it's really
a unique time in the self-driving world
a lot of these things are really coming
together now and we felt like by
bringing together an experienced team we
had an interesting opportunity to build
from a clean sheet a new platform a new
self-driving architecture that leverage
the latest advances in most Reichman fly
machine learning together with our
together with our experience of where
some of the pitfalls tend to be down the
road as you develop these systems
because you don't tend to see them early
on they tend to express themselves as
you get into the long tail of corner
cases that you end up needing to resolve
so we've built that team we have offices
in Palo Alto California and Pittsburgh
Pennsylvania
we've got fleets of vehicles operating
in both pallet on Pennsylvania a couple
of weeks ago we announced that
Volkswagen Group one of the largest
automakers in the world
Ondine Motor Company also one of the
largest automakers in the world have
both partnered with Aurora we will be
developing and are developing with them
a set of platforms and ultimately will
will scale that our technology on their
vehicles across the world and one of the
important the important elements of
building Lexus is Lex before coming out
here what this group would be most
interested in hearing one of the things
that he mentioned was what does it take
to build a self-driving you know build a
new company in a space like this one of
the things that we found very important
was a business model that was
non-threatening to others we recognized
that our strengths and our experience
over the last in my case a decade in
Chris's case almost two really lies in
the development of the self-driving
systems not in building vehicles that I
have had some experience there but but
in developing the self-driving so our
our feeling was if our mission is to get
a technology to market as quickly as
broadly as safely as possible that
mission is best served by playing our
position and working well with others
who can play theirs which is why you see
the model that we've adopted and is now
you'll start to see some of the fruits
of that it through the
partnerships with some of these
automakers so the end of the day our
aspiration in our hope is that this
technology that that is so important the
world in increasing safety in improving
access to transportation in improving
efficiency in the utilization of our
roadways in our cities I mean I this is
maybe the first stock I've ever given
where I didn't start by rattling off
statistics about safety and all the
these other things if you haven't heard
them yet
you should look them up there they're
stark right the fact that most vehicles
in the United States today have an
average on average three parking space
as space is allocated to them the amount
of land that's taken up across the world
in housing vehicles that are used less
than 5% of the time the number of people
I think in the United States the
estimate has spent somewhere between 6
and 15 million people don't have access
to the transportation they need either
the because they're elderly or disabled
or you know one of many other factors
and so this technology is potentially
one of the most impactful for our
society in the coming years it's a
tremendously exciting technological
challenge and you know the confluence of
those two things I think is a really
unique opportunity for engineers and
others who are not engineers who really
want to get involved to play a role in
changing our changing our world going
forward so with that maybe I'll maybe
I'll stop with this and we can go to go
to questions
I am Wayne - hello thanks for coming um
I'm a question a lot of self-driving car
companies are making extensive use of
lidar but you don't see a lot of that
with Tesla wanted to know if you had any
thoughts about that yeah I don't want to
talk about Tesla too much in terms of
our specific any anything that wasn't
public information I'm not going to get
into you I will say that for Aurora we
believe that the right approach is
getting the market quickly and you get
to market and doing so safely and you
get to market most quickly and safely if
you leverage multiple modalities
including layer these are the
just to clarify what's running the
background these are all just aurora
videos of our cars driving on various
test routes yeah hi I'm Luke ramzan from
the stone school a lot of so a lot of
customers have visceral type connections
to their automobile I was wondering how
you see that market the car enthusiast
market being affected by AVS and then
vice versa how the how the AVS will be
designed around those type of oh yeah
customers yeah it's a good question
thanks for asking but I am one of those
enthusiasts I very much appreciate being
able to drive a car in certain settings
I very much don't appreciate driving in
others right I remember distinctly
several evenings I almost literally
pounding my steering wheel sitting in
Quogue in in Boston traffic you know on
my way to somewhere I do the same in San
Francisco I think the opportunity really
is to turn that it turned sort of
personal vehicle ownership and driving
into more of a sport and something you
do for leisure I see it a gentleman
some time ago asked me to talk hey don't
you think this is a problem for the
country I think you meant the world if
people don't learn how to drive that's
just something a human should know how
to do my perspective is it's as much of
a problem as people not intrinsically
knowing how to ride a horse today if you
want to know how to ride a horse go ride
a horse if you want to you want to race
a car go to a racetrack or go out to you
know a mountain road that's been
allocated for it ultimately I think I
think there is an important place for
that because I certainly agree with you
I'm very much a vehicle enthusiast
myself but I think there is so much
opportunity here in alleviating some of
these other problems particularly in
places where it's not fun to drive that
I think there's a place for both yeah
yeah yeah
congratulations on the partnership that
was announced recently I think so I have
a two-part question the first one is so
we heard last week from I think there
was a gentleman from talking about
how long they have been working on this
autonomous car technology and you simply
have rammed up extremely fast so is
there a licensing model that you have
taken that I mean how are you able to
commercialize the technology in one year
so just to be clear we're not actually
commercializing we're just to
distinguish we are partnering and
developing vehicles and Walter may be
running pilots as we announced you know
we could to ago with the Moya shuttles
we are however I will distinguish that
from broad commercialization of the
technology and I don't want to get too
much into you know the nuances of that
business model I will say that it is is
one that's done in very close
partnership with our automotive partners
because you know they at the end of the
day they understand their cars they
understand their customers they have
distribution networks they are you know
our automotive partners are fairly well
positioned it provided they have the
right support in developing a
self-driving technology the fairly
fairly well positioned to you know roll
it out of the scale so the second part
of my question is again looking at this
you know pace of adoption and the
maturity of technology do you see like
an open source model for autonomous you
know cars as they become more and more
unclear I am not convinced that an open
source model is what gets to market most
quickly in the long run it's not clear
to me what will happen I think there
will be a handful of successful
self-driving stacks that will make it
nowhere near the number of self-driving
companies today but a handful I think
two questions one is in invariably a new
product development there's typically
two types of bottlenecks there's a
technological bottleneck and an economic
bottleneck right so technological
bottleneck might be a you know the
sensors aren't good enough or the
machine learning algorithms aren't good
enough and so on I'd be interested to
hear and it'll shift obviously over time
so I'd be interested to know what you
would say is the current thing that if
hey yeah if this part of the of the
architecture was ten times better we
would and that on the economic side I'd
be interested to know you know gee if if
sensors were 100 times cheaper then so
it'd be interested to hear your
perspective on that's a great question
let me start with the economic side of
it and just to get that at the wake is a
little bit quicker answer the economics
of operating a self-driving vehicle and
a shared network today would close that
that business case closes even with high
costs of sensors that is not that is not
what's stopping us and that's part of
why the the gentleman earlier who asked
you know should use lighter or not if
your target is to initially deploy these
in fleets you would be wise to start at
the top end of the market develop and
deploy a system that's as capable as
possible as quickly as possible and then
costs it down over time and you can do
that as computer vision and precision
recall increase today they're not good
enough right and so so economically
depending on your model of going to
market and we believe that the right
model is through
mobility services you can cost out your
cost down the center inevitably you know
there's no unobtainium in light our
units today
there's no reason fundamentally that he
should conserve a light our unit will
lead you to a seventy thousand dollar
price point right however if you build
anything in low enough volumes is going
to be expensive
many of these things will work their way
into the standard automotive process
they'll work their way into Tier one
suppliers and when they do the
automotive community has shown
themselves to be exceptional at driving
those costs down and so I expect them to
come way down to your other question
technological bottlenecks and challenges
one of the key challenges of
self-driving rima is and remains that of
forecasting the intent and B and future
behaviors of other actors both in
response to one another but also in
response to your own decisions in motion
that's a perception problem but it's
something more than a perception problem
it's also a you know prediction and you
know there there are a number of
different things that come together to
have that have to come together to solve
this we're excited about some of the
tools that we're using and interleaving
various of modern machine learning
techniques throughout the system to do
things like project our own behaviors
that were learned for the ego vehicle on
others and assume that they'll behave as
we would had we been in that situation
like an expert system kind of approach
yeah yeah you you assume nominal
behavior and you guard against off
nominal right but it's it's very much
it's not a solved problem I wouldn't say
it's it's very much as you get into that
really long tail of development when
you're no longer you know putting out
demonstration videos but you're instead
just putting your head down and eking
out those you know fine on lines that's
the kind of problem you tend to deal
with again so this question isn't
necessarily about the development of
self-driving cars but more like an
ethics question when you're putting
human lives into like the hands of
software isn't there always the
possibility
for like outside agents with malicious
intent to use it for their own gain and
how do you guys if you do have a plan
how do you intend to protect against
yeah so security is a very real
aspect
so we saw it's a constant game of cat
and mouse and so I think it just
requires a very good
you know team and a concerted effort
over time I I don't think I don't think
you solve at once and I certainly
wouldn't pretend to have a plan that
solves it and is done with it we're we
we try to leverage best practices where
we can in the fundamental architecture
of the system to make it less exposed
and in particular key parts of the
system was exposed to nefarious actions
of others but at the end of the day it's
just a constant is a constant
development effort thank you for being
here so I had a question about what
opportunities self-driving cars open up
since driving has kind of been designed
around like a human being at the center
since the beginning if you put a
computer at the center what you know
society-wide differences and maybe even
like within individual car differences
that open up like you know could cars go
150 miles an hour on the highway and get
places much faster what cars be like
like look differently when a human
doesn't need to be paying attention and
stuff like that yeah I think the answer
is yes the and that's that something is
very exciting right so one of the I
think one of the unique opportunities
that automakers in particular have when
self-driving technology gets
incorporated into their vehicles is they
can do things like play like
differentiate the user experience they
can provide services
you know augmented reality services or
you know location services many other
sort of it opens a new window into an
entirely new market that automakers
haven't historically played in and it
allows them to change the the very
vehicles themselves as you've mentioned
the interior can change as we validate
some of these self-driving systems and
confirm that they do in fact reduce the
collision the the rate of collisions is
we hope they will you can start to pull
out a lot of the extra you know mass and
other things that we've added to
vehicles to make them more passively
safe right
roll cages crumple zones airbags you
know a lot of these things you know
presumably in a world where we don't
crash there is there is much less need
for passive safety systems so yes I have
a question about the go no-go tests that
you conduct for certain features like
you mentioned the throttle control where
you slow down the throttle assuming that
the driver has pressed the wrong wrong
pedal when you test when you decide to
launch that feature how do you know it's
definitely going to work in all
scenarios because your data set might
not be oh it's a it's a it's a
statistical evaluation every case right
you're right there you will this is this
is part of the art of self-driving
vehicle development is you will never
have comprehensively captured every case
every scenario that is as my some of you
may want to correct me on this I think
that's an unbounded set it may in fact
be bounded at some point but I think
it's on and so you'll never you know
there actually have characterized
everything what you will have done
hopefully if you do it right is you will
have established with a reasonable
degree of confidence that you can
perform at a level of safety that's
better than the average human driver and
once you've reached that threshold and
you're confident that you've reached
that threshold I think it the
opportunity to launch is is real and you
should seriously consider it so thank
you for your talk today first and my
question is self-driving seems to be
able to ultimately take over the world
to some extent but just like other
technologists today they open up new
opportunities but also bring in adverse
effects so how do you respond to fear
and nected effects that may come in one
day and especially what do you see as
the positive and active implications of
future day self-driving positive and
negative implications
so the positive ones like kind of listed
and you'll find your favorite press
article and they'll list them as well
the negative ones in the near term I do
I do worry a little bit about the
displacement of jobs not a little bit
this will happen it happens with every
technology like this I think it's
incumbent on us to find a good way of
transitioning those who are employed in
some of the transportation sectors that
will be affected into better work right
there are a few opportunities that are
interesting in that regard but I think
it's an important thing to start
discussing now because it's gonna take
you know a few years and you know by the
time we got these self-driving systems
on the roads really starting to place
that labor I'd really like to have a new
home for it now I I'm kasha from the
Sloan School my question was more about
your business model again with
partnering with both VW and he and a and
you're just perspective and how you were
able to effectively do that
did not one of them want to go sort of
exclusive with you and what was your
sort of thought process about that yeah
so our our mission as I mentioned used
to get the technology to market broadly
and quickly and safely we are you know
have been and remain convinced that the
right way to do that is by providing it
to as much the industry as possible
to every automaker who shares our vision
in our approach and we were pleased to
see that both Volkswagen Group and I'm
assuming you all know the scope of
Volkswagen right this is a massive
automaker Hyundai Motor also very large
across Hyundai Kia and Genesis they both
shared our vision of how we should do
this which was important to us they both
shared you know a a keen interest in
making a difference at scale through
their platforms Volkswagen has you know
I can give very admirable set of
initiatives around electric and vehicle
electrification a few other things Honda
is doing similar things and so you know
for us it was important that we enable
everyone and that was kind of what
Aurora was started to do hi I had a
question now that I see a lot of
companies are coming up with
self-driving cars right so most of the
cars are pretty much all the technology
is bound only to the car so would we see
something like an open network where car
communicate with each other regardless
of which company they come from and
would this in any way you know increase
the safety or the performance of
vehicles and stuff like that yeah I
think I think you're getting it vehicle
to vehicle vehicle infrastructure type
communication there there efforts
ongoing in that and it certainly it's
it's only positive right the having that
information available to you
can only make things better the
challenge has historically been with
vehicle the vehicle and back to
particular vehicle to infrastructure or
vice versa
it doesn't scale well one and two it's
been slow it's been much slower and
coming than our development and so when
we develop these systems we develop them
without the expectation that those that
those communication protocol are
available to us will certainly protect
for them and it will certainly be you
know a benefit once or once they're here
but until then many of the hard problems
that I would have welcomed 10 years ago
to have a beacon on every traffic light
that just told me at state rather than
having to perceive it I would have
certainly used those ten years ago
now they're less significant because
we've kind of worked our way through a
lot of the problems that would have
solved thank you for your talk my
question is what's your opinion about
cooperation of self-driving vehicles so
maybe I think if you can control a group
of self-driving vehicles at the same
time you can achieve a lot of benefits
to the traffic yes that is where one of
the that is where a lot of the benefits
come from and infrastructure utilization
or and is in ride-sharing with
autonomous vehicles and and specifically
you know the better we understand demand
patterns people movement goods movement
the better we can sort of optimally
allocate these vehicles and at locations
where they're needed so yes that's that
certainly that that coordination this is
where as I mentioned these three vectors
of vehicle electrification ride-sharing
autonomy or transfer mobility as a
service and autonomy
really come together with a unique value
proposition yeah okay thank you yeah
thank you so much for a great talking