Kind: captions
Language: en
the following is a conversation with
convoked he is the president and the CTO
of Cruz Automation leading an effort to
solve one of the biggest robotics
challenges of our time vehicle
automation he's a co-founder of two
successful companies twitch and crews
that have each sold for a billion
dollars and he's a great example of the
innovative spirit that flourishes in
Silicon Valley and now is facing an
interesting and exciting challenge of
matching that spirit with the mass
production and the safety centric
culture of a major automaker like
General Motors this conversation is part
of the MIT artificial general
intelligence series and the artificial
intelligence podcast if you enjoy it
please subscribe on youtube itunes or
simply connect with me on twitter at Lex
Friedman spelled Fri D and now here's my
conversation with Kyle vote
grew up in Kansas right yeah and I just
saw that picture you had you know
there's them a little bit a little bit
worried about that yeah so in high
school in Kansas City you joined Shawnee
Mission North High School Robotics team
yeah now that wasn't your high school
that's right that was that was the only
high school in the area that had a like
a teacher who was willing to sponsor a
FIRST Robotics team I was gonna troll
you a little bit jog your mess
trying to look super cool and intense
because you know this was BattleBots
it's a serious business
so we're standing there with a welded
steel frame and looking tough so go back
there what is that jury to robotics well
I think I've been trying to figure this
out for a while but I've always liked
building things with Legos and when I
was really really young I wanted the
Legos I had motors and other things and
then you know Lego Mindstorms came out
and for the first time you could program
Lego contraptions and I think things
just sort of snowballed from that but I
remember seeing you know the battle bots
TV show on Comedy Central and thinking
that is the coolest thing in the world I
want to be a part of that and not
knowing a whole lot about how to build
these 200-pound fighting robots so I
sort of obsessively pored over the
internet forums where all the creator's
for battle bots would sort of hang out
and talk about you know document their
build progress and everything and I
think I read I must have read like you
know tens of thousands of forum posts
from from basically everything that was
out there on what these people were
doing and eventually like sort of
triangulated how to how to put some of
these things together and and ended up
doing battle bots which was you know I
was like 13 or 14 which is pretty
awesome I'm not sure if the show is
still running but the battle bots is
there's not an artificial intelligence
component it's remotely controlled and
yeah it's an almost like a mechanical
generic challenge yeah I think things
that can be broken they're
radio-controlled so and I think that
they allowed some limited form of
autonomy but you know in a two-minute
match you're in and the way these things
ran you're really doing yourself a
disservice by trying to automate it
versus just you know do the practical
thing which is drive it yourself
the entertainment aspect just going on
YouTube there's like and some of them
wield an axe some of them I mean there's
that fun so what drew you to that aspect
it wasn't the mechanical engineering was
it the dream to create like Frankenstein
and sentient being I was just like the
Lego you like tinkering with stuff I
mean that that was just building
something I think the the idea of you
know this this radio-controlled machine
that that can do various things if it
has like a weapon or something was
pretty interesting I agree it doesn't
have the same appeal as you know
autonomous robots which I which I you
know sort of gravitated towards later on
but it was definitely an engineering
challenge because everything you did in
in that competition was pushing
components to their limits so we would
buy like these $40 DC motors that came
out of a winch like on the front of a
pickup truck or something and we'd power
the car with those and we'd run them at
like double or triple their rated
voltage so they immediately start
overheating but for that 2-minute match
you can get you know a significant
increase in the power output of those
motors before they burn out and so
you're doing the same thing for your
battery packs all the materials in the
system and I think there was something
something intrinsically interesting
about just seeing like where things
break and did you all fly and see where
they break did you take it to the
testing point like how did you know two
minutes or was there a reckless let's
just go with it and see we weren't very
good at BattleBots we lost all of our
matches that woody first round like the
one I built first both of them were
these wedge-shaped robots because a
wedge even though it's sort of boring to
look at is extremely effective you drive
towards another robot and the front edge
of it gets under him and then they sort
of flip over kind of like a door stopper
and the first one had a pneumatic
polished stainless steel spike on the
front that would shoot out about eight
inches the purpose of which is what
pretty pretty ineffective actually but
it looks cool and was it helpful to lift
no it was it was just to try to poke
holes in the other robot and then the
second time I did it which is the
following I think maybe 18 months later
we had well a titanium axe with a with a
hardened steel tip on it that was
powered by a hydraulic cylinder which we
were
activating with liquid co2 which was had
its own set of problems so great so
that's kind of on the hardware side I
mean at a certain point there must have
been born a fascination on the software
side so what was the first piece of coal
you've written go back there see what
language was it what what was that was a
Emacs vim was it a more respectable
modern ID do you remember any of this
yeah well I remember I think maybe when
I was in third or fourth grade school I
was at elementary school had a bunch of
Apple 2 computers and we'd play games on
those and I remember every once in a
while something mood would would crash
it wouldn't start up correctly and it
would dump you out to what I later
learned was like sort of a command
prompt and my teacher would come over
and type actually remember this to this
day for some reason like PR number six
or PR pound six which is peripheral 6
which is the disk drive which would fire
up the disk and load the program and I
just remember thinking wow she's like a
hacker like teach me these these codes
these error codes what I called him at
the time but she had no interest in that
so it wasn't until I think about fifth
grade that I had a school where you
could actually go on these Apple twos
and learn to program and so it's all in
basic you know where every line you know
the line numbers are all number that
every line is numbered and you have to
like leave enough space between the
numbers so that if you want to tweak
your code you go back and the first line
was 10 and the second line is 20 now you
have to go back and insert and 15 and if
you need to add code in front of that
you know 11 or 12 and you hope you don't
run out of line numbers and have to redo
the whole thing
there's go-to statements yeah go to and
it's very basic maybe it's a name but a
lot of fun and that was like that was
you know it's fun that's when you know
when your first program you see the
magic of it it's like it just just like
this world opens up with you know
endless possibilities for the things you
could build or or accomplish with that
computer so you got the bug then so even
starting with basic and then what C++
throughout what did you it was a
computer program in computer science
classes in high school not not where I
went so it was a self-taught but I did a
lot of programming the thing that
you know sort of pushed me in the path
of eventually working on self-driving
cars is actually one of these really
long trips driving from my house in
Kansas to I think Las Vegas where we did
the Battle Watts competition and I had
just gotten my I think my learner's
permit or early driver's permit and so I
was driving this you know 10 hour
stretch across western Kansas where it's
just you're going straight on a highway
and it is mind-numbing ly boring and I
remember thinking even then with my sort
of mediocre programming background that
this is something that a computer can do
right let's take a picture of the road
let's find the yellow lane markers and
you know steer the wheel and you know
later I've come to realize this had been
done
you know since since the 80s or the 70s
or even earlier but I still wanted to do
it and sort of immediately after that
trip switched from sort of BattleBots
which is more radio-controlled
machines to thinking about building you
know autonomous vehicles of some scale
start off with really small electric
ones and then you know progress to what
we're doing now so what was your view of
artificial intelligence at that point
what did you think so this is uh before
there's been ways in artificial
intelligence right the the current wave
with deep learning makes people believe
that you can solve in a really rich deep
way the computer vision perception
problem but like in before the deep
learning craze you know how do you think
about how would you even go about
building a thing that perceives itself
in the world local as itself in the
world moves around the world like when
you were younger and yeah as what was
your thinking about it well prior to
deep neural networks our convolutional
neural as these modern techniques we
have or at least ones that are in use
today it was all heuristic space and so
like old-school image processing and I
think extracting you know yellow lane
markers out of an image of a road is one
of the problems that lends itself
reasonably well to those heuristic based
methods you know like just do a
threshold on the color yellow and then
try to fit some lines to that using a
Hough transform or something and then go
from there
traffic like detection and then stop
signs detection red yellow green and I
think you can you could I mean if you
wanted to do a full I was just trying to
make
thing that would stay in between the
lanes on a highway but if you wanted to
do the full the full you know set of
capabilities needed for a driverless car
I think you could and we done this at
cruise you know in the very first days
you can start off with a really simple
you know human written heuristic just to
get the scaffolding in place for your
system traffic light detection probably
a really simple you know color threshold
injustice system up and running before
you migrate to you know a deep learning
based technique or something else and
you know back in when I was doing this
my first one it was on Pentium 203 233
megahertz computer in it and I I think I
wrote the first version in basic which
is like an interpreted language it's
extremely slow because that's the thing
I knew at the time and so there was no
no chance at all of using there was no
computational power to do any sort of
reasonable deep nets like you have today
so I don't know what kids these days are
doing our kids these days you know at
age 13 using neural networks in their
garage I mean I also I get emails all
the time from you know like 11 12 year
old saying I'm having you know I'm
trying to follow this tensorflow
tutorial and I'm having this problem
and their general approach in the deep
learning community is of extreme
optimism of as opposed to you mentioned
like heuristics you can you can separate
the autonomous driving problem into
modules and try to solve it sort of
rigorously or you just do it end to end
and most people just kind of love the
idea that you know us humans do a tenth
and we just perceive and act we should
be able to use that do the same kind of
thing when you're on that's and that
that kind of thinking you don't want to
criticize that kind of thinking because
eventually they will be right yeah and
so it's exciting and especially when
they're younger to explore that is a
really exciting approach but yeah it's
it's changed the the language the kind
of stuff you turned green with it it's
kind of exciting to see when they
seniors grow up yeah I can only imagine
if you if your starting point is you
know Python and tensorflow at age 13
where you end up you know after 10 or 15
years of that that's that's pretty cool
because of github because this
they're tools for solving most of the
major problems and artificial
intelligence are within a few lines of
code for most kids and that's incredible
to think about also on the
entrepreneurial side and and and at that
point was there any thought about
entrepreneurship before you came to
college is sort of doing your building
this into a thing that impacts the world
on the large scale yeah I've always
wanted to start a company I think that's
you know just a cool concept of creating
something and exchanging it for value or
creating value I guess so in high school
I was I was so trying to build like you
know a servo motor drivers little
circuit boards and sell them online or
other other things like that and
certainly knew at some point I wanted to
do a startup but it wasn't really I'd
say until college until I felt like I
had the
I guess the right combination of the
environment the smart people around you
and some free time and a lot of free
time at MIT so you came to MIT as an
undergrad 2004 that's right and that's
when the first DARPA Grand Challenge was
happening yeah the the timing of that is
beautifully poetic so how did you get
yourself involved in that one originally
there wasn't a official entry yeah
faculty sponsored thing and so a bunch
of undergrads myself included I started
meeting and got together and tried to
haggle together some sponsorships we got
a vehicle donated a bunch of sensors and
tried to put something together and so
we had our team was probably mostly
freshmen and sophomores you know which
was not really a fair fair fight against
maybe the you know postdoc and
faculty-led teams from other schools but
we we got something up and running we
had our vehicle drive by a wire and you
know very very basic control and things
but on the day of the qualifying for pre
qualifying round the one and only
steering motor that we had purchased the
thing that we had you know retrofitted
to turn the steering wheel on the truck
died and so our vehicle was just dead in
the water couldn't steer so we didn't
make it very far on the hardware side so
was there a software component was there
like how did your view of autonomous
vehicles in terms of artificial
intelligence evolve in this moment I
mean you know like you said from the 80s
has been autonomous vehicles but really
that was the birth of the modern wave
the the thing that captivated everyone's
imagination that we can actually do this
so what how were you captivated in that
way so how did your view of autonomous
vehicles change at that point I'd say at
that point in time it was it was a
the curiosity as in like is this really
possible and I think that was generally
the spirit and the the purpose of that
original DARPA Grand Challenge which was
to just get a whole bunch of really
brilliant people exploring the space and
pushing the limits and and I think like
to this day that DARPA challenge with
its you know million dollar prize pool
was probably one of the most effective
you know uses of taxpayer money dollar
for dollar that I've seen you know
because that that small sort of
initiative that DARPA put put out sort
of in my view was the catalyst or the
tipping point for this this whole next
wave of autonomous vehicle development
so that was pretty cool so let me jump
around a little bit on that point they
also did the urban challenge where I was
in the city but it was very artificial
and there's no pedestrians and there's
very little human involvement except a
few professional drivers yeah do you
think there's room and then there was
the Robotics Challenge with humanoid
robots right so in your now role is
looking at this you're trying to solve
one of the you know autonomous driving
one of the harder more difficult places
of San Francisco is there a role for
DARPA to step in to also kind of help
out they challenge with new ideas
specifically a pedestrians and so on all
these kinds of interesting things well I
haven't I haven't thought about it from
that perspective is there anything DARPA
could do today to further accelerate
things and I would say my instinct is
that that's maybe not the highest and
best use of their resources in time
because like kick starting and spinning
up the flywheel is I think what what
they did in this case for a very very
little money but today this has become
this has become like commercially
interesting to very large companies and
the amount of money going into it and
the amount of people like going through
your class and learning about these
things and developing these skills is
just you know orders of magnitude more
than it was back then and so there's
enough momentum and inertia and energy
and investment dollars into this space
right now that I don't I don't I think
they're I think they're they can just
say mission accomplished and move on to
the next area of technology that that
needs help
so then stepping back to MIT you left on
my teaching a junior year what was that
decision like as I said I always wanted
to do a company in or start a company
and this opportunity landed in my lap
which was a couple guys from Yale we're
starting a new company and I googled
them and found that they had started a
company previously and sold it actually
on eBay for about a quarter million
bucks which was a pretty interesting
story but so I thought to myself these
guys are you know rock star
entrepreneurs they've done this before
they must be driving around in Ferraris
because they sold their company and you
know I thought I could learn a lot from
them so I teamed up with those guys and
you know went out during went out to
California during IIP which is my tease
month off on one on one way ticket and
basically never went back we were having
so much fun we felt like we were
building something and creating
something and it was going to be
interesting that you know I was just all
in and got completely hooked and that
that business was justin.tv which is
originally a reality show about a guy
named Justin
which morphed into a live video
streaming platform which then morphed
into what is twitch today so that was
that was quite a an unexpected journey
so no regrets no looking back it was
just an obvious
I mean one-way ticket I mean if we just
pause on that for a second
there was no how did you know these were
the right guys this is the right
decision you didn't think it was just
follow the heart kind of thing well I
didn't know but you know just trying
something for a month during IEP he
seems pretty little risk right right and
then you know well maybe I'll take a
semester off and my teas pretty flexible
about that you can always go back right
and then after two or three cycles of
that I eventually threw in the towel but
you know I think it's
I guess in that case I felt like I could
always hit the undo button if I had to
right but it never lasts from from when
you look in retrospect I mean it seems
like a brave decision that you know it's
difficult it would be difficult for a
lot of people to make it wasn't as
popular I'd say that the general you
know flux of people out of MIT at the
time was mostly into you know financier
consulting jobs in Boston or New York
and very few people were going to
California to start companies but today
I'd say that's it's probably inverted
which is just a sign of a sign of the
times I guess yeah
so there's a story about midnight of
March 18 2007 where whether we're
TechCrunch I guess and I was just in TV
earlier than was supposed to a few hours
the site didn't work I don't know if any
of this is true you can tell me and I
you and one of the folks adjusted to e
I'm a shear coated through the night can
you take me through that experience so
let me let me say a few nice things that
the article I read quoted Justin Kahn
said that you were known for mural
coding through problems and being a
creative quote creative genius so on
that night what was going through your
head or maybe I put another way how do
you solve these problems what's your
approach to solving these kinds of
problems were the line between success
and failure seems to be pretty thin
that's a good question well first of all
that's that's a nice of Justin to say
that I think you know I would have been
maybe twenty-one years old then and not
very experienced at programming but as
with with everything in a start-up
you're sort of racing against the clock
and so our plan was the second we had
this live streaming camera backpack up
and running where Justin could wear it
and no matter where he went in a city it
would be streaming live video and this
is even before the iPhones this is like
hard to do back then we would launch and
so we thought we were there and and the
backpack was working and then we sent
out all the emails to launch the launch
the company and do the press thing and
then you know we weren't quite actually
there and then we thought oh well you
know they're not going to announce it
until maybe 10 a.m. the next morning and
it's I don't know it's 5 p.m. now so how
many hours do we have left what is that
like you have 17 hours to go and and and
that was that was gonna be fine was the
problem obvious did you understand what
could possibly like how complicated was
the system at that point it was it was
pretty messy so to get a live video feed
that looked decent working from anywhere
in San Francisco I put together the
system where we had like three or four
cell phone data modems and they were
like we take the video stream and you
know sort of spray it across these three
or four modems and then try to catch all
the packets on the other side you know
with unreliable cell phone networks
pretty low level networking yeah and and
putting his like you know sort of
protocols on top of all that to
reassemble and reorder the packets and
have time buffers and error correction
and all that kind of stuff and the night
before it was just staticky every once
in while the image would would go
staticky and there would be this
horrible like screeching audio noise
because the audio was also corrupted and
this would happen like every five to ten
minutes or so and it was a really you
know off-putting to the viewers
how do you tackle that problem what was
the just freaking out behind a computer
there's the word are there other other
folks working on this problem like we
behind a whiteboard were you doing uh
yes a little hair coding it has a little
only because there's four of us working
on the company and only two people
really wrote code and Emmett wrote the
website in the chat system and I wrote
the software for this video streaming
device and video server and so I you
know it's my sole responsibility to
figure that out yeah and I think I think
it's those you know setting setting
deadlines trying to move quickly and
everything where you're in that moment
of intense pressure that sometimes
people do their best and most
interesting work and so even though that
was a terrible moment I look back on it
fondly because that's like you know
that's one of those character defining
moments I think
so in 2013 October you founded cruise
automation yeah so progressing forward
another exception successful company was
acquired by GM in 16 for 1 billion
dollars but in October 2013 what was on
your mind what was the plan how does one
seriously start to tackle one of the
hardest robotics
most important impact for robotics
problems of our age after going through
twitch twitch was was and it is today
pretty successful but the the work was
the result was entertainment mostly like
the the better the product was the more
we would entertain people and then you
know make money on them ad revenues and
other things and that was that was a
good thing it felt felt good to
entertain people but I figured like you
know what is really the point of
becoming a really good engineer and
developing these skills other than you
know my own enjoyment and I realized I
wanted something that scratched more of
an existential itch like something that
that truly matters and so I basically
made this list of requirements for a new
if I was going to do another company and
the one thing I knew in the back of my
head that twitch took like eight years
to become successful and so whatever I
do I better be willing to commit you
know at least ten years to something and
when you think about things from that
perspective
you certainly I think raised the bar on
weight you choose to work on so for me
the three things where it had to be
something where the technology itself
determines the success of the product
like hard really juicy technology
problems because that's what motivates
me and then it had to have a direct and
positive impact on society in some way
so an example would be like you know
healthcare self-driving cars because
they save lives other things where
there's a clear connection to somehow
improving other people's lives and the
last one is it had to be a big business
because for the positive impact to
matter it's got to be a large scale
scale yeah and I was thinking about that
for a while and I made like I tried
writing a gmail clone and looked at some
other ideas and then it just sort of
light bulb went off like self-driving
cars like that was the most fun I had
ever had in college working on that and
like well what's the state of the
technology has been ten years maybe
maybe times have changed and maybe now
is the time to make this work and I
poked around and looked at the only
other thing out there really at the time
was the Google self-driving car project
and I thought surely there's a way to
you know have an entrepreneur mindset
and sort of solve the Minimum Viable
Product here and so I just took the
plunge right then in there and said this
this is something I know I can commit
ten years to it's the probably the
greatest applied AI problem of our
generation it's right and if it works
it's going to be both a huge business
and therefore like probably the most
positive impact I can possibly have on
the world so after that light bulb went
off I went all in on crews immediately
and got to work did you have an idea how
to solve this problem which aspect of
the problem to solve you know slow like
what we just had Oliver for voyage here
slow-moving retirement communities urban
driving highway driving did you have
like did you have a vision of the city
of the future or you know the
transportation is largely automated that
kind of thing or was it sort of more
fuzzy and gray area than that my
analysis of the situation is that Google
is putting a lot it had been putting a
lot of money into that project that a
lot more resources and so
and they still hadn't cracked the fully
driverless car you know this is 20 2013
I guess so I thought what what can I do
to sort of go from zero to you know
significant scale so I can actually
solve the real problem which is the
driverless cars and I thought here's the
strategy we'll start by doing a really
simple problem or solving a really
simple problem that creates value for
people so eventually ended up deciding
on automating highway driving which is
relatively more straightforward as long
as there's a backup driver there and
I'll you know the go-to-market will be
able retrofit people's cars and just
sell these products directly and the
idea was we'll take all the revenue and
profits from that and use it to do the
social reinvest that in research for
doing fully fabulous cars and that was
the plan
the only thing that really changed along
the way between then and now is we never
really launched the first product we had
enough interest from investors in enough
of a signal that this was something that
we should be working on that after about
a year of working on the highway
autopilot we had it working you know on
a prototype stage but we just completely
abandoned that and said we're gonna go
all in on driverless cars now is the
time can't think of anything that's more
exciting and if it works more impactful
so we're just gonna go for it the idea
of retrofit is kind of interesting yeah
being able to it's how you achieve scale
it's a really interesting idea is it's
something that's still in the in the
back of your mind as a possibility not
at all I've come full circle on that one
trying to build a retrofit product and
I'll touch on some of the complexities
of that and then also having been inside
in OEM and seeing how things work and
how a vehicle is developed and validated
when it comes to something that has
safety critical implications like
controlling the steering and the other
control inputs on your car it's pretty
hard to get there with with a retrofit
or if you did even if you did it it
creates a whole bunch of new
complications around liability or how
did you truly validate that or you know
something in the base vehicle fails and
causes your system to fail whose fault
is it
or if the cars anti-lock brake systems
or other things kick in or the software
has been it's different in one version
of the car you retrofit versus another
and you don't know because the
manufacturer has updated it behind the
scenes there's basically an infinite
list of longtail issues that can get you
and if you're dealing with a safety
critical product that's not really
acceptable that's a really convincing
summary of why it's really challenging
but I didn't at the time so we tried it
anyway but it's a pitch also at the time
it's a really strong one yes that's how
you achieve scale and that's how you
beat the current the the leader at the
time of Google or the only one in the
market the other big problem we ran into
which is perhaps the biggest problem
from a business model perspective is we
had kind of assumed that we'd we started
with an Audi s4 as the vehicle we
retrofitted with his highway driving
capability and we had kind of assumed
that if we just knock out like three
make and models of vehicle that'll cover
like eighty percent of a San Francisco
market doesn't everyone there drive I
don't know a BMW or a Honda Civic or one
of these three cars and then we surveyed
our users we found out that it's all
over the place we would to get even a
decent number of units sold we'd have to
support like you know 20 or 50 different
models and each one is a little
butterfly that takes time and effort to
maintain you know that retrofit
integration and custom hardware and all
this so is it there's a tough business
so GM manufactures and sells over nine
million cars a year and what you with
crews are trying to do some of the most
cutting-edge innovation in terms of
applying AI and so hot out of those
you've talked about a little bit before
but it's also just fascinating to me
we'll work a lot of automakers you know
the difference between the gap between
Detroit and Silicon Valley
let's say just to be sort of poetic
about it I guess what how do you close
that gap how do you take GM into the
future where a large part of the fleet
would be autonomous perhaps I want to
start by acknowledging that that GM is
made up of you know tens of thousands of
really brilliant motivated people who
want to be a part of the future and so
it's pretty fun to work within the
attitude inside a car company like that
is you know embracing this this
transformation and change rather than
fearing it and I
think that's a testament to the
leadership at GM and that's flown all
the way through to to everyone you talk
to even the people in this in blue
plants working on these cars so that's
really great so that starting from that
position makes a lot easier so then when
the the people in San Francisco at Cruz
interact with the people at GM at least
we have this common set of values which
is that we really want this stuff to
work because we think it's important and
we think it's the future
not to say you know those two cultures
don't clash they absolutely do there's
different different sort of value
systems like in a car company the thing
that gets you promoted and so the reward
system is following the processes
delivering the the program on-time and
on-budget so any sort of risk-taking is
discouraged in many ways because if a
program is late or if you shut down the
plant for a day it's you know you can
count the millions of dollars that burn
by pretty quickly whereas I think you
know most Silicon Valley companies and
crews in the methodology we were
employing especially around the time of
the acquisition the reward structure is
about trying to solve these complex
problems in any way shape or form or
coming up with crazy ideas that you know
90% of them won't work and and so so
meshing that culture of sort of
continuous improvement and
experimentation with one where
everything needs to be you know
rigorously defined upfront so that you
never slip a deadline or miss a budget
was a pretty big challenge and that
we're over three years in now after the
acquisition and I'd say like you know
the investment we made in figuring out
how to work together successfully and
who should do what and how we bridge the
gaps between these very different
systems and way of doing engineering
work is now one of our greatest assets
because I think we have this really
powerful thing but for a while it was
both both GM and crews were very steep
on the learning curve yes I'm sure it
was very stressful it's really important
work because that's that's how to
revolutionize the transportation it
really to revolutionize any system you
know you look at the healthcare system
or you look at the legal system I have
people like lawyers come up to me all
the time like everything they're working
on can easily be automated but then
that's not a good feeling yeah that was
it's not a good feeling but also there's
no way to automate because the the the
entire infrastructure is really you know
based is older and it moves very slowly
and so how do you close the gap between
I haven't how can I replace of course
lawyers don't wanna be replaced with an
app but you could replace a lot of
aspect when most of the data is still on
paper
and so the same thing was with
automotive I mean it's fundamentally
software so it's is basically hiring
software engineers it's thinking a
software world I mean I'm pretty sure
nobody in Silicon Valley's ever hit a
deadline so and then it's probably true
yeah and GSI is probably the opposite
yeah so that's that culture gap is
really fascinating so you're optimistic
about the future of that yeah I mean
from what I've seen it's impressive and
I think like especially in Silicon
Valley it's easy to write off building
cars because you know people have been
doing that for over a hundred years now
in this country and so it seems like
that's a solved problem but that doesn't
mean it's an easy problem and I think it
would be easy to sort of overlook that
and think that you know we're Silicon
Valley engineers we can solve any
problem you know building a car it's
been done therefore it's you know it's
it's it's not it's not a real
engineering challenge but after having
seen just the sheer scale and magnitude
and industrialization that occurs inside
of an automotive assembly plant that is
a lot of work that I am very glad that
we don't have to reinvent to make
self-driving cars work and so to have
you know partners who have done that for
a hundred years now these great
processes and this huge infrastructure
and supply base that we can tap into is
just remarkable because the scope in
surface area of the problem of deploying
fleets of self-driving cars is so large
that we're constantly looking for ways
to do less so we can focus on the things
that really matter more and if we had to
figure out how to build an assemble in
you know test and build the cars
themselves I mean we work closely with
Jim on that but if we had to develop all
that capability in-house as well you
know that that would just make make the
problem really intractable I think mmm
so yeah just like your first entry mit
DARPA challenge when there was what the
motor that failed and somebody that
knows what they're doing with the motor
did it that would have been nice if you
focus on the software and not the
hardware platform yeah right so from
your perspective now you know there's so
many ways that autonomous vehicles can
impact Society in the next year five
years ten years what do you think is the
biggest opportunity to make money in
autonomous driving sort of make it a
financially viable thing in the
near-term what do you think would be the
biggest impact there well the things
that that drive the economics for fleets
of self-driving cars or they're sort of
a handful of variables one is you know
the cost to build the vehicle itself so
the material cost how many you know
what's the cost of all your sensors plus
the cost of the vehicle and every all
the other components on it another one
is the lifetime of the vehicle it's very
different if your vehicle drives one
hundred thousand miles and then it falls
apart versus you know two million
and then you know if you have a fleet
it's kind of like an airplane where or
airline where once you produce the
vehicle you want it to be in operation
as many hours a day as possible
producing revenue and then a you know
the other piece of that is how are you
generating revenue I think that's kind
what you're asking and I think the
obvious things today are you know the
ride-sharing business because that's
pretty clear that there's demand for
that there's existing markets you can
tap into and larger urban areas that
kind of thing yeah yeah and and and I
think that there are some real benefits
to having cars without drivers compared
to through the status quo for people who
use ride share services today you know
you get privacy consistency
hopefully significant improve safety all
these benefits versus the current
product but it's it's a crowded market
and then other opportunities which
you've seen a lot of activity in the
last really in last six to twelve months
is you know delivery whether that's
parcels and packages food or or
groceries those are all sort of I think
opportunities that are that are pretty
ripe for these you know once you have
this core technology which is the fleet
of autonomous vehicles there's all sorts
of different business opportunities you
can build on top of that but I think the
important thing of course is that
there's zero monetization opportunity
until you actually have that fleet of
very capable driverless cars that are
that are as good or better than humans
and that's sort of where the entire
industry is sort of in this holding
pattern right now yeah the trend
achieved that baseline so but you said
sort of rely not reliability consistency
it's kind of interesting I think I heard
you say somewhere I'm not sure if that's
what you meant but you know I can
imagine a situation where you would get
an autonomous vehicle and you know when
you get into an uber or lyft
you don't get to choose the driver in a
sense that you don't get to choose the
personality of the driving do you think
there's a there's room to define the
personality of the car the way drives
you in terms of aggressiveness for
example in terms of sort of pushing the
bomb the one of the biggest challenges
in Toms driving is the is a trade-off
between sort of safety and
and do you think there's any room for
the human to take a role in that
decision to accept the liability I guess
we III wouldn't it no I'd say within
reasonable bounds as in we're not gonna
I think it'd be highly unlikely we did
expose any nob that would let you you
know significantly increase safety risk
I think that's that's just not something
we'd be willing to do but I think
driving style or like you know are you
gonna relax the comfort constraints
slightly or things like that all of
those things make sense and are
plausible I see all those is you know
nice optimizations once again we get the
core problem solved and these fleets out
there but the other thing we've sort of
observed is that you have this intuition
that if you sort of slam your foot on
the gas right after the light turns
green and aggressively accelerate you're
gonna get there faster but the actual
impact of doing that is pretty small you
feel like you're getting there faster
but so that so the same would be true
for ABS even if they don't slam there
you know the pedal to the floor when the
light turns green they're gonna get you
they're within you know if it's a
15-minute trip within 30 seconds of what
you would have done otherwise if you
were going really aggressively so I
think there's this sort of
self-deception that that my aggressive
driving style is getting me there faster
well so that's you know some of the
things I study some things I'm
fascinated by the psychology of that I
don't think it matters that it doesn't
get you there faster it's it's the
emotional release driving is is a place
being inside or a car somebody said it's
like the real world version of being a
troll so you have this protection this
mental protection you're able to sort of
yell at the world like release your
anger whatever is but so there's an
element of that that I think autonomous
vehicles would also have to you know
have giving an outlet to people but it
doesn't have to be through through
through driving or honking or so on
there might be other outlets but I think
to just sort of even just put that aside
the baseline is really you know that's
the focus that's the thing you need to
solve and then the fun human things can
be solved after but so from the baseline
of just solving autonomous driving and
you're working in San Francisco one of
the more difficult cities to operate in
what
what is what is the any of you currently
the hardest aspect of autonomous driving
and negotiated with pedestrians is that
edge cases of perception is it planning
is there a mechanical engineering is it
data
fleet stuff like what are your thoughts
on the challenge the more challenging
aspects there that's a good that's a
good question I think before before we
go to that though I just wanted I like
what you said about the psychology
aspect of this because I think one
observation I made is I think I read
somewhere that I think it's maybe
Americans on average spend you know over
an hour a day on social media like
staring at Facebook and so that's just
you know 60 minutes of your life you're
not getting back and it's probably not
super productive and so that's 3,600
seconds right and that's that's time you
know it's a lot of time you're giving up
and if you compare that to people being
on the road if another vehicle whether
it's a human driver or autonomous
vehicle delays them by even three
seconds they're laying in on the horn
you know even though that's that's you
know one one thousandth of the time they
waste looking at Facebook every day so
there's there's definitely some you know
psychology aspects of this I think that
are pre interesting road rage in general
and then the question of course is if
everyone is in self-driving cars do they
even notice these three-second delays
anymore because they're doing other
things or reading or working or just
talking to each other so it'll be
interesting to see where that goes
in a certain aspect people people need
to be distracted by something
entertaining something useful inside the
car so they don't pay attention to the
external world and then and then and it
can take whatever psychology and bring
it back to Twitter and then focus on
that as opposed to sort of interacting
sort of putting the emotion out there
into the world so it's a it's an
interesting problem but baseline
autonomy I guess you could say
self-driving cars you know at scale will
lower the collective blood pressure of
society probably by a couple points yeah
without all that road rage and stress so
that's a good good externality so back
to your question about the technology in
the the I guess the biggest problems and
I have a hard time answering that
question because you know we've been at
this
like specifically focusing on driverless
cars and all the technology needed to
enable that for a little over four and a
half years now and even a year or two in
I felt like we had
completed the functionality needed to
get someone from point A to point B as
in if we need to do a left turn maneuver
or if we need to drive around a you know
a double parked vehicle into oncoming
traffic
or navigate through construction zones
the the scaffolding and the building
blocks where it was there pretty early
on and so the challenge is not any one
scenario or situation for which you know
we fail at 100% of those it's more you
know we're benchmarking against a pretty
good or pretty high standard which is
human driving all things considered
humans are excellent at handling the
edge cases and unexpected scenarios
whereas computers the opposite and so
beating that that baseline set by humans
is the challenge and so what we've been
doing for quite some time now is
basically
it's this continuous improvement process
where we find sort of the the most you
know uncomfortable or the things that
that could lead to a safety issue other
things all these events and then we sort
of categorize them and rework parts of
our system to make incremental
improvements and do that over and over
and over again and we just see sort of
the overall performance of the system
you know actually increasing in a pretty
steady clip but there's no one thing
there's actually like thousands of
little things and just like polishing
functionality and making sure that it
handles you know every version
impossible permutation of a situation by
either applying more deep learning
systems or just by you know adding more
tests coverage or new scenarios that
that we develop against and just
grinding on that it's we're sort of in
the the unsexy phase of development
right now which is doing the real
engineering work that it takes to go
from prototype to production
you're basically scaling the the
grinding so has sort of taking seriously
that the process of all those edge cases
both with human experts and machine
learning methods to cover to cover all
those situations yeah and the exciting
thing for me is I don't think that
grinding ever stops right because
there's a moment in time where you you
cross that threshold of human
performance and become superhuman but
there's no reason there's no first
principles reason that AV capability
will tap out anywhere near humans like
there's no reason it couldn't be 20
times better whether that's you know
just better driving or safer driving a
more comfortable driving or even a
thousand times better given enough time
and we intend to basically chase that
you know forever to build the best
possible product better and better and
better and always new educators come up
and you experiences so and you want to
automate that process as much as
possible mhm so what do you think in
general in society when do you think we
may have hundreds of thousands of fully
autonomous vehicles driving around so
first of all predictions nobody knows
the future you're a part of the leading
people trying to define that future but
even then you still don't know but if
you think about a hundreds of thousands
of heat
so a significant fraction of vehicles in
major cities are autonomous do you think
I would Rodney Brooks who is 2050 and
beyond are you more with Elon Musk who
is we should have had that two years ago
well I mean I don't want me to have it
two years ago but we're not there yet so
I guess the the way I would think about
that is let's let's flip that question
around so what would prevent you to
reach hundreds of thousands of vehicles
and that's a goodness a good rephrasing
yeah so the
I'd say the it seems the consensus
among the
people developing self-driving cars
today is to sort of start with some form
of an easier environment whether it
means you know lacking inclement weather
or you know mostly sunny or whatever it
is and then add add capability for more
complex situations over time and so if
you're only able to deploy in areas that
that meet sort of your criteria or that
the current domain you know operating
domain of the software you developed
that may put a cap on how many cities
you could deploy in
but then as those restrictions start to
fall away like maybe you add you know
capability to drive really well and and
safely in heavy rain or snow you know
that that probably opens up the market
by - two or three fold in terms of the
cities you can expand into and so on and
so the real question is you know I I
know today if we wanted to we could
produce that that many autonomous
vehicles but we wouldn't be able to make
use of all of them yet because we would
sort of saturate the demand in the
cities in which we would want to operate
initially so if I were to guess like
what the timeline is for those things
falling away and reaching hundreds of
thousands of vehicles maybe a range is
but I would I would say less than five
years that's in five years yeah and of
course you're working hard to make that
happen
so you started two companies that were
eventually acquired for each for a
billion dollars so you're pretty good
person to ask what does it take to build
a successful startup mmm-hmm I think
there's there sort of survivor bias here
a little bit but I can try to find some
common threads for the the things that
worked for me which is
you know in in both of these companies
it was really passionate about the core
technology I actually like you know lay
awake at night thinking ab