Kind: captions
Language: en
the following is a conversation with
Gustav Sorum he's the chief research and
development officer Spotify leading
their product design data technology and
engineering teams as I've said before in
my research and in life in general I
love music listening to it and creating
it and using technology especially
personalization through machine learning
to enrich the music discovery and
listening experience that is what
Spotify has been doing for years
continually innovating defining how we
experience music as a society in the
digital age that's what Gustav and I
talk about among many other topics
including our shared appreciation of the
movie true romance in my view one of the
great movies of all time this is the
artificial intelligence podcast if you
enjoy it subscribe on YouTube give it
five stars on iTunes support on patreon
or simply connect with me on twitter at
lux Friedman spelled Fri D ma n and now
here's my conversation with Gustav Soros
from Spotify has over 50 million songs
in its catalogue so let me ask the
all-important question I feel like
you're the right person to ask what is
the definitive greatest song of all time
it varies for me personally she can't
speak definitively for everyone
I wouldn't believe very much in machine
learning if I did right because everyone
had the same taste so for you what is
you have to pick what is the song
alright so it's it's pretty easy for me
there is this song called you're so cool
Hans Zimmer soundtrack to true romance
it was a movie that made a big
impression on me and it's kind of been
following me through my life actually
had it play at my wedding I start with
the organist and help them play it on an
organ which was a pretty pretty
interesting experience that is probably
my I would say top 3 movie of all time
yeah it's just an incredible yeah and
then it came out during my formative
years and
as I've discovered in music you shape
your music taste during those years so
it definitely affected me quite a bit
did it affect you in any other kind of
way well the movie itself affected me
back then it was a big part of culture I
didn't really adopt any characters from
the movie about it it was a it was a
great story of love fantastic actors and
and you know really I didn't even know
who Hans Zimmer was at a time but
fantastic music and so um that song has
followed me and the movie actually
follow me throughout my life that was a
Quentin Tarantino actually I think
director director produced that her so
it's not stairway to heaven or Bohemian
Rhapsody so those are those are great
they're not my personal favorites but uh
but they're I realized that people have
different tastes and that's it's a big
part of what we do well for me I have to
stick with stairway to heaven so 35,000
years ago I looked this up on Wikipedia
flute like instruments started being
used in caves as part of hunting rituals
then primitive cultural gatherings
things like that this is the birth of
music since then we had a few folks
Beethoven Elvis Beatles Justin Bieber
of course Drake so in your view let's
start like high level philosophical what
is the purpose of music on this planet
of ours I think music has many different
purposes I think there's there's
certainly a big purpose which is the
same as multiple attainment which is
ESCA pisum and to be able to live in
some sort of other mental state for a
while
but I also think you have the the
opposite of escaping which is to help
you focus on something you are actually
doing and so I think people use music as
a tool to to tune the brain to the
activities that they are actually doing
and it's kind of like in one sense maybe
it's the rawest signal if you if you
think about the brain that's known that
works it's maybe the most efficient hack
we can do to actually actively tune it
into some state that you want to be you
can do it in other ways you can tell
stories to put people in a certain mood
but music is probably very F
to get to a certain mood very fast and
you know there's uh there's a social
component historically to music where
people listen to music together I was
just thinking about this that to me you
mentioned machine learning but to me
personally music is a really private
thing I'm speaking for myself I listen
to music like almost nobody knows the
kind of things I have in my library
except people who are really close to me
and they really only know a certain
percentage there's like some weird stuff
that I'm almost probably embarrassed but
by right it's called to give the
pleasure everyone I said the guilty
pleasures yet hopefully they're not too
bad but it's just the ice for me it's
personal do you think of music is
something that's social or as something
that's personal this is a very so I
think it's the same it's the same answer
that you use it for for both we we've
thought a lot about this during these 10
years at Spotify obviously in one sense
as you said music is incredibly social
you go to concerts and so forth on the
other hand it is your your escape and
everyone has these things are very
personal to them so what we've found is
that when it comes to to most people
claim that they have a friend or two
that they are heavily inspired by and
that they listen to so I actually think
music is very social but in a smaller
group setting it's in it's an intimate
form of of it's an intimate relationship
it's not something that you necessarily
share broadly now at concerts you can
argue you do but then you've gathered a
lot of people that you have something in
common with I think this broadcast
sharing on music it's something we tried
on social networks and so forth but it
turns out that people aren't super
interested in is what their friends
listen to they're interested in
understanding if they have something in
common crabs with a friend but not you
know not just as information right that
that that's really interesting
that was just
thinking about this morning listening to
Spotify I really have a pretty intimate
relationship was modified with my
playlists right I've had them for many
years now and they've grown with me
together there's there's an intimate
relationship you have with a library of
music you've developed and we'll talk
about different ways to play with that
can you do the impossible task and try
to give a history of music listening
from your perspective from before the
Internet and after the Internet and just
kind of everything leading up to
streaming Spotify I'll try it it could
be a 100 year podcast yeah I'll try to
do a brief version there are some things
that that I think are very interesting
during the history of music which is
that before recorded music you to be
able to enjoy music you actually had to
be where the music was produced because
he couldn't he couldn't record it and
time-shifted write creation and
consumption had to happen at the same
time basically concerts and so you
either had to get to the nearest village
to listen to music and while that was
cumbersome and it severely limited the
distribution of music it also had some
different qualities which was that the
Creator could always interact with the
audience
it was always live and also there was no
time cap on the music so I think it's
not a coincident that these early
classical works they're much longer than
the three minutes the three minutes came
in as a restriction of the first wax
disc that could only contain a three
minute singsong on one side right so
actually the recorded music severely
limited there or could constraint I
won't say limit I mean constraints often
good but it put very hard constraints on
the music format so you kind of said
like instead of doing these this opus
like many tens of minutes or something
now you get three and a half minutes
because then you're out of wax on this
disc but in return you get in amazing
distribution you reach will widen right
just on that point real quick without
the mass scale distribution there's a
scarcity component where you kind of
look forward
what we had that it's like the Netflix
versus HBO Game of Thrones you like wait
for the event because you can't really
listen to it see you're like look
forward to it and then it's you derive
perhaps more pleasure because it's more
rare for you to listen to particular
piece you think there's value to that
scarcity yeah I think that that is
definitely a thing and there's always
this component of if you have something
in infinite amount so will you value it
as much probably not humanity is always
seeking some is relative so you're
always seeking something you didn't have
and when you have it you don't
appreciate as much I think that's
probably true but I think that's why
concerts exist so you can actually have
both but I think net if you couldn't
listen to music in your car driving that
that'd be worse that cost will be bigger
than the benefit of of the anticipation
I think that you would have so yep it
started with live concerts then it's
being able to you know the the
phonograph invented right you start to
be able to record music exactly so then
then you got this massive distribution
that that made it possible to create two
things I think first of all cultural
phenomenons they probably need
distribution to be able to happen but it
also opened access to you know for a new
kind of artists so you started to have
these phenomenons like Beatles inhale
this and so forth that would really a
function of distribution I think
obviously of talent and innovation but
there was also taking a component and of
course the next big innovation that come
along was was radio broadcast radio and
I think radio is interesting because it
started not as a music medium and
started us as an information medium for
for news and then radio need to define
something to fill the time wid so that
they could honestly play more ads and
make more money and music was free so so
then you had this massive distribution
we could program to people I think those
things that ecosystem is what created
the ability for for for hits but it was
also very broadcast medium so you would
tend to get these massive massive hits
but maybe not such a lot
tail in terms of choice of everybody
listening to the same stuff yeah and as
you said I think there are some social
benefits to that yeah I think for
example there is there's a high
statistical chance that if I talk about
the latest episode of Game of Thrones we
have something to talk about yeah just
statistically in the age of individual
choice maybe some of that goes away so I
I do see the value of like you know
shared cultural components but I also
obviously love personalization and so
let's catch us up to the Internet
so maybe Napster well first of all
there's like mp3's exact tape CDs there
was a digitalization of music with a CD
really it was physical distribution but
the music became did you don't yeah and
so they were files but basically boxed
software and use the software analogy
and then you could start downloading
these files and I think there are two
interesting things that happen back to
music used to be longer before it was
constrained by the distribution medium I
don't think that was a coincidence and
then really the only music genre to have
developed mostly after music was a file
again on the Internet is EDM and EDM is
often much longer than the traditional
music I think I think it's interesting
to think about the fact that music is no
longer constrained in minutes per song
or something it's it's a it's a legacy
of our own distribution technology and
you see some of this new music that that
breaks the format not so much as I would
have expected actually by now but but it
still happens so first of all I don't
really know what EDM is electronic dance
music yeah right you could say Avicii
was one of the biggest in this genre so
the main constraint is of time something
like three four or five minute songs
songs there were eight minutes ten
minutes and so forth because the you
know it started as a digital product
that you downloaded so you didn't have
this this constraint anymore so I think
it's something really interesting that I
don't think has fully happened yet we're
kind of jumping ahead a little bit to
where we are but I think there's there's
tons of formal innovation in music that
should happen now
that couldn't happen when you needed to
really adhere to the distribution
constraints if you didn't adhere to that
you will get no distribution so so jerk
for example
Icelandic artist she made a full I pad
app as an album that's very expensive
you know even though the App Store has
great distribution she gets nowhere near
the distribution versus staying within
the 3-minute format so I think now that
music is fully digital inside these
streaming services there is there is the
opportunity to change the format again
and allow creators to be much more
creative without limiting their their
distribution ability that's interesting
that you're right it's surprising we
don't see that taking advantage more
often it's almost like the constraints
of the distribution from the 50s and 60s
have molded the culture to where we want
the five three to five minutes on that
anything else not just so we want the
song as consumers and as artists like I
cuz I write a lot of music and I never
even thought about writing something
longer than 10 minutes that's it's
really interesting that those
constraints because all your training
data has been three minutes right it's
right okay so yes digitization of data
later than mp3s
yeah so I think you had this file then
that was distributed physically but then
you had the components of digital
distribution and then the internet
happened and there was this vacuum where
you had a format that could be digitally
shipped but there was no business model
and then all these pirate networks
happen Napster and in pirate in Sweden
Pirate Bay which was one of the biggest
and it you know I think from a consumer
point of view which which kind of leads
up to the inception of of Spotify from a
consumer point of view consumers for the
first time had this access model to
music where they could without kind of
any marginal costs they could they could
try different tracks you could use music
in in new ways there was no marginal
cost and that was a fantastic consumer
experience that
just all the music ever made I think was
fantastic but it was all so horrible for
artists because there was no business
model around it so they didn't make any
money so the user need almost drove the
user interface before there was a
business model and then there were these
download stores that allowed you to
download files which was a solution but
it didn't solve the access problem there
was still a marginal cost of 99 cents to
try one more track and I think that that
heavily limits how you listen to music
the example always give this you know in
Spotify a huge amount of people listen
to music while they sleep while they go
to sleep and while they sleep if that
costed you $0.99 per three minutes you
probably wouldn't do that and you would
be much less adventurous if there was a
real dollar cost to exploring music so
the access model is interesting in that
it changes your music behavior you can
be you can take much more risk because
there's no marginal cost to it maybe let
me linger on piracy for a second because
I I find especially coming from Russia
piracy is something that's very
interesting to me not me of course ever
but my friends who partook in piracy of
music software TV shows sporting events
and usually to me what that shows is not
that they're they can actually pay the
money and they're not trying to save
money they're choosing the best
experience so what to me piracy shows is
a business opportunity in all these
domains and that's where I think you're
right spot if I stepped in is basically
piracy was is an experience you can
explore was fine music you like and
actually the interface of piracy isn't
as horrible because it's I mean it's not
metadata yeah that metadata is long
download times all kinds of stuff and
what Spotify does is basically
first rewards artists and second makes
the experience of exploring music much
better I mean the same is true I think
for movies and so on this piracy reveals
in the software space for example I'm a
huge user and fan of Adobe products and
the there was much more incentive to
pirate Adobe products before they went
to a monthly subscription plan and now
all of the sudden
that you used to pirate Adobe products
that I know now actually pay gladly for
the monthly subscription I think you're
right I think it's in it's a sign of an
opportunity for product development and
that sometimes the there's a product
market fit before there's a business
model fit in product development I think
that that is that's a sign of it in in
Sweden I think was a bit of both there
was there was a culture where even had a
political party called the pirate party
and this was during the time when when
people said that you know information
should be free it's not was somehow
wrong to charge for ones and zeros so I
think people felt that artists should
probably make some money somehow else
and you know concerts or something so at
least in Sweden it was part really
social acceptance even at the political
level and that but that also forced
Spotify to compete with with free which
which I don't think would actually could
have happen anywhere else in the world
the music industry needed to be doing
bad enough to take that risk
and Sweden was like a perfect testing
ground it had government-funded
high-bandwidth low-latency broadband
which meant that the product would work
and it was also there was no music
revenue anyway so they were kind of like
I don't think this is going to work but
why not
so this product is one that I don't
think could have happened in America
there was large music market for example
so how do you compete with free because
that's an interesting world of the
Internet where most people don't like to
pay for things so Spotify steps in and
tries to yes compete with free how do
you do it so I think two things one is
people are starting to pay for things on
the Internet
I think one way to think about it was
that advertising was the first business
model because no one would put a credit
card on internet transactional with
Amazon was the second and maybe
subscription is their third and if you
look offline subscription is the biggest
those so that may still happen I think
people are starting to pay but
definitely back then we needed to
compete with free and the first thing
you need to do is obviously to lower the
price to free and then you need to be
better somehow and the way that Spotify
was better was on the user experience on
the on the actual performance the
latency of you know even if even if you
had high bandwidth broadband it would
still take you thirty Seconds to a
minute to download one of these tracks
so the Spotify experience of starting
within the perceptual limit of immediacy
about 250 milliseconds meant that the
the whole trick was that felt as if you
had downloaded all the part that it was
on your harddrive it was that fast even
though it wasn't and it was still free
but somehow you were actually still
being a legal citizen now that was the
trick that's what if I managed to to
pull off so yeah I've actually heard you
say this to write this and that was
surprised that wasn't aware of it
because I just took it for granted you
know whenever an awesome thing comes
along you just like of course it has to
be this way that that's exactly right
that it felt like the entire world's
libraries at my fingertips because of
that of that latency being reduced what
was the technical challenge in reducing
Olli so there was a group of really
really talented engineers one of them
called Ludwig freakiest he wrote the
actually from Gothenburg he wrote the
initial the uterine client which is kind
of an interesting backstory to Spotify
you know that we have one of the top
developers from from BitTorrent clients
as well so he wrote utorrent the world's
smallest BitTorrent clients and then he
he was acquired very early by daniel and
martin who found it spotify and they
actually sold the u torrent client to
BitTorrent but kept living so Spotify
had a lot of experience within
peer-to-peer networking so the original
innovation wasn't was a distribution
innovation where Spotify built an
end-to-end media distribution system up
until only a few years ago we actually
hosted all the music ourselves so we had
both the server side in the cloud
and that meant that we could do things
such as having a peer-to-peer solution
to use local caching on the client-side
because back then the world was mostly
desktop but we could also do things like
hack the TCP protocols
things like niggles algorithm for kind
of exponential back-off or ramp up and
just go full throttle and optimize for
latency at the cost of bandwidth and all
of this end-to-end control
meant that we could do an experience
that felt like a step change these days
we actually are on on GCP we don't host
our own stuff and everyone is really
fast these days so that was the initial
competitive advantage but then obviously
you have to move on over time and that
was I was over 10 years ago right that
was in 2008 the product was launched in
Sweden it was in a beta I think 2007 and
it was on the desktop right so his
desktop only there's no phone there was
no phone the iPhone came out in 2008 but
the App Store came out one year later I
think so the writing was on the wall but
there was no phone yet you've mentioned
that people would use Spotify to
discover the songs they liked and then
they would torrent those songs just so
they can copy it to their phone just
hilarious because I'm not torrent quiet
it seriously piracy does seem to be and
like a good guide for business models
video content as far as I know Spotify
doesn't have video content well we do
have music videos and we do have videos
on the on the service but the way we
think about ourselves is that we're an
audio service and we think that if you
look at the amount of time that people
spend on audio it's actually very
similar to the amount of time that's
people spend on video so the opportunity
should be equally big but today is not
at all valued videos value much higher
so we think it's basically completely
undervalues we think of ourselves as an
audio service but within that audio
service I think video can make a lot of
sense I think for when you're when
you're discovering an artist you
probably do want to see them and
understand who they are to understand
their identity you won't see the video
every time now 90% of the time the phone
is gonna be in your pocket
for podcasters you use video I think
that can make a ton of sense so we do
have video but we're an audio service
where think of it as we call it
internally background able video video
that is helpful but isn't isn't the
driver of the narrative I think also if
we look at YouTube the way people
there's quite a few folks who listen to
music on YouTube
so in some sense YouTube was a bit of a
competitor to to Spotify which is very
strange to me that people use YouTube to
listen to music they play essentially
the music videos right but don't watch
the videos and put it in their pocket
well I think I think it's similar to to
what strange I mean it's similar to what
we were for the piracy networks know
where YouTube for historical reasons
have a lot of music videos so you use
people use YouTube for a lot of the
discovery part of the process I think
but then it's not a really good sort of
quote unquote mp3 player because it
doesn't even background then you have to
keep the app in the foreground so so the
consumption on a good consumption tool
but it's a decently good discoveries I
mean I think YouTube is fantastic
products and I use it for all kinds of
purposes so if I were to admit something
I do use YouTube a little bit for the
discovery to assistant discovery process
of songs and then if I like it I'll add
it just fine that's okay that's okay
with that ok so sorry we're jumping
around a little bit so the it's kind of
incredible you look at Napster you look
at the early days of Spotify how do you
one fascinating points how do you grow a
user base see their ins in Sweden you
have an idea I saw the initial sketches
that look terrible how do you grow user
base from all from a few folks to
millions I think there are a bunch of
tactical answers so first of all I think
you need a great product I don't think
you take a bad product and and market it
to be successful so you need a great
product but sorry to interrupt but it's
a totally new way to listen to music too
so it's not just did people realize
immediately that Spotify is a great
product
I think they did so back to the point of
pyrazine it was a totally new way to
listen to music illegally but people had
been used to the access model in Sweden
and the rest of the world for a long
time through piracy so one way to think
about Spotify it was just legal and fast
piracy yeah and so people have been
using it for a long time so they weren't
alien to it they didn't really
understand how it could be legal because
it was seemed too fast and too good to
be true yeah which i think is a great
product proposition if you can be too
good to be true but what I saw again and
again was people showing each other
clicking the song showing how fast it
started and saying I can't believe this
yeah so I really think it was about
speed then we also had an invite product
program that was there was really meant
for scaling because we hosted our own
service we needed as a control scaling
but that built a lot of expectation and
I don't want to say hype because I hype
implies that it was that it wasn't true
excitement around the product and we've
replicated that when we launched in the
in the US we also built up and it might
only program first there are lots of
tactics but I think you need a you need
a great product that solves some problem
and B basically the key innovation there
was technology but on a method level the
innovation was really the access model
versus the ownership model and that was
tricky a lot of people said that they I
mean they wanted to own their music they
would never kind of rent it or borrow it
but I think the fact that we had a free
tier which meant that you get to keep
this music for life as well
helped quite a lot so this is an
interesting psychological point maybe
you can speak to it was a big shift for
me like I get to it's almost like a I go
to therapy for this is uh I think I
would describe my early listening
experience and I think a lot of my
friends do is basically hoarding music
is your like slowly one song by one song
or maybe albums gathering a collection
of music that you love and you own it
it's like awful especially with CDs or
tape you like physically had it and and
what Spotify what I had to come to grips
with it was kind of liberating actually
is to throw away all the music I've had
this therapy session yes people and I
think the mental trick is so actually we
seen the user data once what if I
started a lot of people did the exact
same thing they started hoarding as if
the music would disappear right almost
the equivalent of downloading and so you
know we had these playlists that had
limits of like a few hundred thousand
tracks which we no one will ever like
well they do needs and hundreds and
hundreds of thousands of tracks and to
this day you know some people want to
actually save code and coordinate play
the entire catalog but I think that the
therapy session goes something like
instead of throwing away your music if
you took your files and you store them
in the locker at Google it'd be a
streaming service it's just that in that
locker you have all the world's music
now for free so instead of giving away
your music you got all the music it's
yours it's a you could think of it at
having a copy of the world's catalogue
that forever so you actually got more
music instead of less it's just that you
just took that hard disk and you sent it
to to someone who stored it for you and
once you go through that mental journey
I'm like still my files they're just
over there and I just have 40 million
other 50 million or something now then
people are like okay that's good
the problem is I think because you paid
us a subscription if we hadn't had the
free tier where you would feel like even
if I don't want to pay anymore I still
get to keep them you keep your playlist
forever they don't disappear even though
you stopped paying I think that was
really important if we would have
started us you know you can put in all
this time but if you stopped paying you
lose all your work I think that would
have been a big challenge and what's the
big challenge for a lot of our
competitors that's another reason why I
think the free tier is really important
that people need to feel the security
that the work they put in it will never
disappear even if they decide not to pay
I like how you put the work you put in I
she stopped even think of it that way I
just actually Spotify taught me to just
enjoy music I'm sorry as opposed as
opposed to what I was doing before which
is like in an unhealthy way hoarding
music which I found that because I was
doing that I was listening to a small
selection of songs way too much to our
where I was getting sick of them whereas
Spotify the more liberating kind of
approaches I was just enjoying of course
I listened to stairway to heaven over
and over but because of the extra
variety I don't get as sick of them
there is an interesting statistic I saw
that so
Spotify has maybe you can correct me but
over 50 million songs tracks and over
three billion playlists so yes a million
songs and three billion playlists 60
times more playlists what do you make of
that yeah so the way I think about it is
that from a from is that the station or
machine learning point of view you have
all these if you only thing about
reinforcement learning where you have
this state space of all the tracks and
you can take different journeys through
this through this world and these I
think of these is like people helping
themselves and each other creating
interesting vectors through this space
of tracks and then it's not so
surprising that across you know many
tens of millions of kind of atomic units
there will be billions of paths that
make sense and we're probably pretty
quite far away from having found all of
them so kind of our job now is users
when Spotify started it was really a
search box that was for that time pretty
powerful and then I'd like to refer to
that this programming language called
play listing where if you as you
probably were pretty good at music
you knew your new releases you knew your
backyard law you knew your stairway to
heaven you could create a soundtrack for
yourself using this playlist thing - oh
that's like meta programming language
for music - sounds like your life and
people who were good at music it's back
to how do you scale the product for
people who are good at music that wasn't
actually enough if you had the catalog
in a good search tool and you can create
your own sessions you could create
really good a soundtrack for your entire
life probably perfectly personalized
because you did it yourself but the
problem was most people many people
aren't that good at music they just
can't spend the time even if you're very
good at news it's gonna be hard to keep
up so what we did to try to scale this
was to essentially try to build you can
think of them as a
instead there's this friend that some
people had that helped them navigate
this music catalog that's what we're
trying to do for you but also there is
something like 200 million active users
on Spotify so there it's okay so from
the machine learning perspective you
have these 200 million people plus
they're creating it's really interesting
to think of playlist as I mean I don't
know if you meant it that way but it's
almost like a programming language it's
a released a trace of exploration of
those individual agents of the the
listeners and you have all this new
tracks coming in so it's a fascinating
space that is ripe for machine learning
so that is there is it is it possible
how can playlist be used as data in
terms of machine learning and just to
help Spotify organize the music so we
found in our data not surprising that
people who play listed lots they retain
much better they had a great experience
and so our first attempt was to playlist
for users and so we acquired this
company called tune ego of editors and
professional playlist errs and kind of
leverage the maximum of human
intelligence to help to help build kind
of these vectors through the track space
for four people and that that broaden
the product then the obvious next and we
you know use statistical means where
they could see what when they created a
playlist how did that play this perform
you know they could see skips of the
songs they could see how the songs
perform and they manually iterated the
playlist to maximize performance for a
large group of people but there were
never enough editors to playlist for you
personally so the promise of machine
learning was to go from kind of group
personalization using editors and tools
into statistics to individualization and
then what's so interesting about the 3
billion playlist we have is we ended the
truth is we lucked out this was not
a priori strategy as is often the case
it looks really smart in hindsight was
as dumb luck we looked at these
playlists and we had some people in the
company a person named their grandson it
was really good at machine learning
already back in in back then in like
2007-2008 back then it was mostly
collaborative filtering you so forth but
we realized that what what this is is
people are grouping tracks for
themselves that have some semantic
meaning to them and then they actually
label it with a playlist name as well so
in a sense people were grouping tracks
along semantic dimensions and labeling
them and so could you could you use that
information to find that that latent
embedding and so we started playing
around with collaborative filtering and
we saw tremendous success with it
basically trying to extract some of
these some of these dimensions and and
if you think about it's not surprising
at all it'd be quite surprising if
playlists were actually random if they
had no semantic meaning for most people
they group these tracks for some reason
so we just happen to cross this
incredible data set where people are
taking taken these tens of millions of
tracks and group them along different
semantic vectors and the semantics being
outside the individual users it's some
kind of universal there's a universal
embedding that holds across people on
this earth yes I do think that the
embeddings you finally gonna be
reflective of the people who play listed
so if if you have a lot of indie lovers
who playlist your embed is going to
perform better there but what we found
was that yes there were these these
latent similarities they were very
powerful and we we had them it was
interesting because I think that the
people who play listed the most
initially were this so-called music
aficionados who who really into music
and they often had a certain they're
tasteful stuff is often certain geared
towards a certain type of music and so
what surprised us if you look at the
problem from the outside you might
expect that
the algorithms would start performing
best with mainstreamers first because it
somehow feels like an easier problem to
solve mainstream tastes than really
particular tastes it was the complete
opposite for us the recommendations
performed fantastically for people who
saw themselves as having very unique
taste that's probably because all of
them playlist ed and they didn't perform
so well for mainstream is they actually
thought they were a bit too particular
and unorthodox so we had the complete
opposite of what we expected success
within the hardest problem first and
then had to try to scale to more
mainstream recommendations so you've
also acquired echo nests that analyze a
song data so in your view maybe you can
talk about so what kind of data is there
from a machine learning perspective
there's a like a huge amount what we're
talking about playlists thing and just
user data of what people are listening
to the playlists are constructing and so
on and then there's the the actual data
within a song what makes a song I don't
know the actual waveforms right is there
any how do you mix the two how much
values are in each to me it seems like
user data is well it's a romantic notion
that the song itself would contain
useful information but if I were to
guess user data would be much more
powerful like playlists would be much
more powerful yeah so we use both our
biggest success initially what was with
playlist data without understanding
anything about the structure of this
song but when we acquire the echo nest
they had the inverse problem they
actually didn't have any play data they
were just they were a provider of
recommendations but they didn't actually
have any play data so they they looked
at the structure of songs sonically and
they looked at Wikipedia for cultural
references and so forth right cool and
did a lot of NLU and so forth so we got
that skill into the company and combine
kind of our user data with their with
their kind of content-based so you can
think of as we were used to based and
they were content based in their
recommendations and we combine those two
and for some cases where you have a new
there's no no play date obviously you
have to try to go by either you know who
the artist is or or the sonic
information in the song or what it's
similar to so there's definitely value
in in both and we do a lot in both but I
would say yes the user data captures
things that that have to do with culture
in the greater society that you would
never see in the in the content itself
but that said we have seen we have a
research lab in Paris when you know we
can talk about more about that on kind
of machine layer on the creator side
what it can do for creators not just for
the consumers but where we looked at how
does the structure of a song actually
affect the listening behavior and it
turns out that there is a lot of we can
we can predict things like skips based
on we you know based on on the song
itself we could say that maybe you
should move that chorus a bit because
you're skippers gonna go up here there
is a lot of latent structure in the
music which is not surprising because it
is some sort of mind hack so there
should be structured that's probably
what we respond to you just blew my mind
actually for from the creator
perspective so that's really interesting
topic that probably most creators aren't
taking advantage of right so there's
I've recently got to interact with a few
folks youtubers who are like obsessed
with this idea of what do I do to make
sure people keep watching the video and
then like look at the analytics of which
point if people turn off and so on first
of all don't think that's healthy but
it's it's because you can do it a little
too much but it is a really powerful
tool for helping the creative process
you just made me realize you could do
the same thing for creation of music and
so is that something you've looked into
oh is it can you speak to how much
opportunity there is for that yeah I
think I listen to to the podcast with
Suraj yeah and I thought it was
fantastic and directed to do the same
thing where he said and he said he
posted something in the morning
yeah immediately watch the feedback
where the drop off was and then
responded to that in the afternoon
yeah which which is quite different from
how people make podcasts for example yes
exactly I mean the feedback loop is
almost non-existent it's very so if we
back out a one-level I think actually
both for music and podcasts which we
also do is let Spotify I think there's a
tremendous opportunity just for the
creation workflow and I think it's
really interesting speaking to you who
because you're a musician a developer
and a podcaster if you think about those
three different roles if you if you make
the leap as a musician if you if you
think about it as a software tool chain
really your door with the stems
that's the IDE right that's what you
work in source code formant with your
with with what you're creating then you
sit around and you play with that and
when you're happy you compile that thing
into some sort of you know AAC or mp3 or
something you do that because you get
distribution there's so many runtimes
for that mp3 across the world and
Carstairs and stuff so you kind of
compile this executable you ship it out
and kind of an old fashioned box
software analogy and then you hope for
the best right right but as a as a as a
software developer you'd never do that
first you go and get helping you
collaborate with other Creators yeah and
then you know you think it'd be crazy to
just ship one version of your software
without doing an a/b test without any
feedback loop and then HD tracking
exactly and then you would you would
look at the feedback loops and try to
optimize that thing right so I think if
you think of it as a as a very specific
software tool chain it looks quite
arcane you know the tools that a music
creator has versus what a software
developer has so that's kind of how we
think about it and why wouldn't a why
wouldn't a music creator have something
like github you could collaborate much
more easily so we have we bought this
company called sound trap which has a
kind of Google Docs for music approach
where you can collaborate with other
people on the kind of source code format
with stamps and I think introducing
things like AI tools there to help you
as you're creating music both in in
helping you you know
put accompaniment your music like drums
or something help you master and mix
automatically help you understand how
this track will perform exactly what you
would expect as a software developer I
think makes a lot of sense and I think
the same goes for a podcaster I think
podcasters will expect to have the same
kind of feedback loop that Siraj has
like why wouldn't you maybe maybe it's
not healthy but sorry I wanted to
criticize the fact cuz you can overdo it
because a lot of the each and we're in a
new era of that so you can become
addicted to it and therefore what people
say you become a slave to the YouTube
algorithm are sort of it's a it's always
a danger of a new technology as opposed
to say if you're creating a song
becoming too obsessed about the intro
riff to the song that keeps people
listening versus actually the entirety
of the creation process it's a balance
absolutely but the fact that there's
zero I mean you're blowing my mind right
now because you're completely right that
there is no signal whatsoever there's no
feedback whatsoever in the creation
process and music or podcasting almost
at all and are you saying that Spotify
is hoping to help create tools to not
tools but no tools actually actually
tools from traders absolutely so we have
we've remains micro stations the last
few years around music creation this
company called soundtrap which is the
door digital audio workstation but that
is browser-based and that their focus
was really the Google Docs approach
where you can collaborate with people
much more easily then you could in
previous tools so we have some of these
tools that we're working with that we
want to make accessible and then we can
connect it with our with our consumption
data we can create this feedback loop
where we could help you understand we
could help you create and help you
understand how you will perform we also
acquired this other company within
podcasting called anchor which is one of
the biggest podcasting tools mobile
focused so really focused on simple
creation or easy access to create
but that also gives us this feedback
loop and even before that we invested in
something called Spotify for artists and
Spotify for podcasters which is an app
that you can download you can verify
that you are that creator and then you
get you get things that you know
software developers have had for years
you can see where if you look at your
podcast for example on Spotify or or a
song that you released you can see how
it's performing which cities is
performing and who is listening to it
what's the demographic break up so
similar in the sense that you can
understand how you're actually doing on
the on the platform so we we definitely
want to build tools I think you also
interviewed the head of research for
Adobe and I think that's an item back to
photoshop that you like I think that's
an interesting analogy as well Photoshop
I think has been very innovative in
helping photographers and artists and I
think there should be the same kind of
tools for for music creators where you
could get you know AI assistants for
example that's you creating music as you
can do with with Adobe where you can I
want to sky over here and you can get
help creating that sky the really
fascinating thing is what Adobe doesn't
have is a distribution for the content
you create so you don't have the data of
if I create if I uh you know whatever
creation I'm making Photoshop a premiere
I can't get like immediate feedback like
I can on YouTube for example about the
way people are responding and if Spotify
is creating those tools that that's a
really exciting actually world but let's
talk a little about podcast it's so I
have trouble talking to one person so
it's a bit terrifying and kind of hard
to fathom but an average sixty to a
hundred thousand people will listen to
this episode okay so it's intimidating
it's intimidating
so I hosted on blueberry I don't know if
I'm pronouncing that correctly actually
it looks like most people listen to an
Apple podcast cashbox and pocket gas
and only about a thousand listen on
Spotify in just my podcast right so
where do you see a time when Spotify
will dominate this so Spotify is
relatively new into this podcasting talk
nesting site yeah in podcasting what's
the deal with podcasting and Spotify how
serious is Spotify about podcasting do
you see a time where everybody would
listen to you know probably a huge
amount of people majority perhaps listen
to music on Spotify do you see a time
when the same is true for podcasting
well I certainly hope so
that is our mission our mission as a
company is actually to enable a million
creators to live off of their art in a
billion people inspired by it and what I
think it is interesting about that
mission is it actually puts the crater's
first even though it's not as a consumer
focused company and it says to be able
to live off of their art not just make
some money or further art as well so
it's quite an ambitious project and so
we think about creators of all kinds and
we kind of expanded our mission from
being music - being audio a while back
and that's not so much because we think
we made that decision we think that my
decision was was made for us
we need the world made that decision
whether we like it or not when you put
in your headphones you're gonna make a
choice between music and new episode of
of your podcast or something else right
we're in that world whether we like it
or not
and that you know that's how radio work
so we decided that we think it's about
audio you can see the rise of audiobooks
and so forth we think audio is this
great opportunity so we decided to enter
it and and obviously Apple and Apple
podcast is absolutely dominating in
podcasting and we didn't have a single
podcast only like two years ago
what we did though was we we we looked
at this and said no can we bring
something to this you know we want to do
this but the back to the
Josefa we have to do something that
consumers actually value to be able to
do this and the reason we've gone from
not existing at all to being the the
record of what quite a wide margin the
second-largest podcast consumption still
still wide gap to iTunes but we're
growing quite fast I think it's because
when we when we looked at the consumer
problem
people said surprisingly that they
wanted their podcasts and music in the
same in the same application so what we
did was we took a little bit of a
different approach what we said instead
of building a separate podcast app we
thought it's their consumer problem to
solve here because the others are very
successful already and we thought there
was in making a more seamless experience
where you can have your podcast in your
music in the same application because we
think it's audio to you and that that
has been successful and that meant that
we actually had 200 million people to
offer this to instead of starting from 0
so I think we have a good chance because
we're taking a different approach than
the competition and back to the other
thing I mentioned about creators because
we're looking at the end-to-end flow I
think there's a tremendous amount of
innovation to do around podcast as a
format when we have creation tools and
consumption I think we could start
improving what podcasting is I mean
podcast is this this opaque big like 1/2
hour file that you're streaming which it
really doesn't make that much sense in
2019 that it's not interactive there's
no feedback loops nothing like that so I
think if we're gonna win it's gonna have
to be because we build a better product
for creators and for for consumers so
we'll see but it's certainly our goal we
have a long way to go
well the creators part is really
exciting you ready you got me hooked
there is the only stats I have a
blueberry just recently added the stats
of whether it's listened to the end or
not and that's like a huge improvement
but that's still nowhere to where you
could possibly go into her statistics
you just download this pot of five
podcasters up and verify and then then
you know where people dropped out in
this episode oh wow ok the moment I
started talking okay I might be
depressed by this but okay so one one
other question
the original Spotify for music and I
have a question about podcasting in this
line is the idea of albums I have what
did you use ik aficionados a friends who
are really big fans of music often
really enjoy albums listening to entire
albums of an artist and correct me if
I'm wrong but I feel like Spotify has
helped replace the idea of an album with
playlists so you create your own albums
it's kind of the way at least I've
experienced music and I have really
enjoy it that way one of the things that
was missing in podcasting for me I don't
know if it's missing I don't know it's
an open question for me but the way I
listen to podcasts is the way I would
listen to albums so I take Joe Rogan
experience and that's an album and I
listen you know I like I put that on and
I listen one episode after the next and
there's a sequence and so on is there a
room for doing what you did for music or
doing what about if I did for music but
creating playlists sort of this kind of
play listing idea of breaking apart from
podcasting from individual podcasts and
creating kind of this interplay or
Debbie thought about that space it's a
great question so I think in in music
you're right basically you bought an
album so it was like you bought a small
catalog of like 10 tracks right it was
it was again it was actually a lot of a
lot of consumption you think it's about
what you like but it's based on the
business model right so you paid for
this 10 track yeah service and then you
listened to that for a while
and then when everything was flat priced
you tended to listen differently now so
so I think the I think the album is
still tremendously important that's why
we have it and you can save albums and
so forth and you have a huge amount of
people who really listen according to
albums and I like that because it is a
creator format you can tell a longer
story over several tracks and so some
people listen to just one track some
people actually want to hear that whole
story now in podcast I think
I think it's different you can argue
that podcasts might be more like shows
on Netflix you have like a full season
of of narcos and you're probably not
going to do like one episode of Marcos
and then one of house of cards like you
know that there's a narrative there and
you you you love the cast and you love
these chairs so I think people will
people love shows and I think they will
they will listen to those shows I do
think you follow a bunch of shows at the
same time so there's certainly an
opportunity to bring you the latest
episode of you know whatever that 5 6 10
things that that you're into but but I
think I think people are gonna listen to
specific hosts and love those hosts for
a long time because I think there's
something different with podcast where
this format of the the experience of the
of the audience is actually sitting here
right between us whereas if you look at
something on TV the audio actually would
come from
you would sit over there the order would
come to you from both of us as if you
were watching notice you were part of
the conversation so my experience is I
mean listen to podcasts like yours and
Joe Rogan is I feel like I know all of
these people they they have no idea who
I am but I feel like you know so many
artists and that it's very different
from me watching a watching like a TV
show or an interview so I think you you
kind of fall in love with people and
experience in a different way so I think
I think shows and hosts are gonna be
very very important I don't think that's
gonna go away in just some sort of thing
where well you don't even know who
you're listening to I don't think that's
gonna happen what I do think is I think
there's a tremendous discovery
opportunity in podcast because the
catalog is growing quite quickly and I
think podcast is only a few like five
six hundred thousand shows right now if
you look back to YouTube as another in
knowledge of creators no one really
knows if you would lift the lid on
YouTube but it's probably billions of
episodes and so I think the podcast
catalog would probably grow tremendously
because the creation tools are getting
easier and then you're gonna have this
discovery opportunity that I think is
really big so so a lot of people tell me
that they love their shows
but discovering podcasts kind of suck
it's really hard to get into new show
they're usually quite long it's a big
time investments I think there's plenty
of opportunity in the discovery part
yeah for sure a hundred percent in in
even the dumbest there's so many
low-hanging fruit too for example just
knowing what episode to listen to first
to try out a podcast exactly because
most podcasts don't have an order to
them they they can be listening to out
of order and sorry to say some are
better than others episodes so some
episodes of Joe Rogan are better than
others and it's nice to know which you
should listen to to try it out and
there's as far as I know almost no
information in terms of like up votes on
how good an episode is exactly so I
think part of the problem is you it's
kind of like music there isn't one
answer people use music for different
things and there's actually many
different types of music there's workout
music and there's classical piano music
and focus music and and and so forth I
think the same with podcasts some
podcasts are sequential they're supposed
to be listened to in in order it's
actually it's actually telling a
narrative some podcasts are one topic
kind of like yours but different guests
so you could jump in anywhere some potes
I have completely different topics and
for those podcasts it might be that I
want you know we should recommend one
episode because it's about AI yeah from
someone but then they talk about
something that you're not interested in
the rest of the episodes so I think or
well we're spending a lot of time on now
it's just first understanding the domain
and creating kind of the knowledge graph
of how do these objects relate and how
do people consume and I think we'll find
that it's gonna be it's gonna be
different I'm excited
is it your the Spotify is the first
people I'm aware of that are trying to
do this for podcasting podcasting has
been like a Wild West
up until now it's been a very we want to
be very careful though because it's been
a very good Wild West I think is this
fragile ecosystem
and I you we want to make sure that you
don't barge in and say like oh we're
gonna internalize this thing and you
have to think about the the creators you
have to understand how they get
distribution today who listens to how
they make money today started you know
make sure that their business model
works that they understand the I think
it's back to doing something improving
their products like feedback loops and
distribution so jumping back into terms
of this fascinating world of recommender
system and listening to music and using
machine learning to analyze things do
you think it's better to what currently
correct me if I'm wrong but currently
Spotify lets people pick what they
listen to the most part there's a
discovery process but you kind of
organize playlists is it better to let
people pick what they listen to or
recommend what they should listen to
something like stations by Spotify that
I saw that you're playing around with
maybe you can tell me what's the status
of that this is a Pandora style app that
just kind of as opposed to you select
the music you listen to it kind of feeds
you music you listen to what's the
status of stations by Spotify what's its
future the store is modify as we have
grown has been that we made it more
accessible to different different
audiences and stations is another one of
those where the question is some people
want to be very specific they actually
want to hear stairway to heaven right
now
that needs to be very easy to do and
some people or even the same person at
some point might say I want to feel up
beats or I want to feel happy or I want
songs to sing in the car alright so they
put in they put in the information in a
very different level and then we need to
translate that into that what that means
musically so stations is a test to to
create like a consumption input vector
that is much simpler where you can just
tune in a little bit and see if that
increases the overall reach but we're
trying to kind of serve the entire gamut
of super-advanced so-called music
aficionados all the way to to people who
they love listening to music but it's it
not their number-one priority in life
right they're not going to sit and
follow every new release from every new
artist they need to be able to to
influence music at a at a at a different
level so we're trying you can think of
it as different products and I think
when one of them one of the interesting
things to answer your question on if
it's better to lift the user to soar to
play I think the answer is the the
challenge when you when machine learning
kind of came along there was a lot of
thinking about what this product
development mean in a machine learning
context people like Andrew Aang for
example when I went to buy do I started
doing a lot of practical machine
learning went from academia and you know
he thought a lot about this and he had
this notion that you know product
manager designer an engine they used to
work around this wireframe I kind of
described what the product should look
like who some talk about when you're
doing like a chatbot or a playlist how
do you what are you gonna say like it
should be good that's not a good product
description so how do you how do you do
that he came up with this notion that
the test set is the new wireframe the
the job of the product manager is to
source a good test set that is
representative of what like if you say
like I want to play the status songs
missing in the car job the product
managers go in source like a good test
out of what that means then you can work
with engineering to have algorithms to
try to produce that right so we try to
think a lot about how to structure
product development for for a machine
learning age and and what we discovered
was that a lot of it is actually in the
expectation and you can go you can go
two ways so
let's say that if you if you set the
expectation with the user that this is a
discovery product like discover weekly
you're actually setting the expectation
that most of all we show you will not be
relevant when you're in the discovery
process you're gonna accept that
actually if you find one gem every
Monday that you totally love you're
probably gonna be happy even though the
statistical meaning one out of ten is
terrible or one out of twenty is
terrible from a user point of view
because the studying was discoveries
fine can I get a sorry to interrupt real
quick I just actually learned about
discover weekly which is a Spotify I
don't know it's it's a feature a Spotify
that shows you cool songs to listen I
maybe I can do issue tracking I couldn't
find them my Spotify app it's in your
library it's in the library it's in the
list of life cuz I was like whoa is just
cool I didn't know this existed and I
try to find it but I'll show it to you
back product yeah but yeah it's a so yes
I just uh just to mention the
expectation there is basically they
you're going to discover in your song
yes so then you can be quite adventurous
in in the recommendations you do but but
if you're but we have another product
called daily mix which kind of implies
that these are only going to be your
favorites so if you have one out of ten
that is good and nine out of ten that
doesn't work for you you're gonna think
it's a horrible product so actually a
lot of the product development we
learned over the years it's about
setting the right expectation so for
daily mix you know algorithmically we
would pick among things that feel very
safe in your taste space there's
discover weekly we go kind of well
because the expectation is most of this
is not gonna so a lot of that a lot of
times of your question there a lot of
should you let the user pick or not it
depends we have some products where the
whole point is that the user can click
play put the phone in the pocket and it
should be really good music for like an
hour we have other products where you
probably need to say like no no say no
no and it's very interactive I see that
makes sense and then the radio product
the station's product is one of these
like click Play put in your pocket four
hours that's really interesting so
you're thinking of different test sets
for differ
users interact create products that sort
of optimize optimize for those test sets
that represents a specific set of users
yes I think one thing that I think is
interesting is we we invested quite
heavily in editorial in in people
creating playlists using a statistical
data and I was successful for us and
then we also invested in in machine
learning and for the longest time you
know within Spotify and within the rest
of the industry there was always this
narrative of humans versus the machine
how go versus editorial and an editors
would would say like well if I had that
data if I could see your playlist in
history and I made a choice for you I
would have made a better choice and they
would have because they honest they're
much smarter than these algorithms the
human is incredibly smart compared to
our algorithms they can take culture
into account and so forth the problem is
that they can't make 200 million
decisions you know per hour for every
user that logs in so the algo may be not
as sophisticated but much more efficient
so there was this there was this
contradiction but then a few years ago
we started focusing on this kind of
human-in-the-loop thinking around
machine learning and we actually coined
an internal term for it called algo
toriel combination of algorithms and and
editors where if we take a concrete
example you think of the editor there's
this paid expert that we have there's
really good at something like so hip-hop
EDM something right there a true expert
no and one in the industry so they have
all the cultural knowledge you think of
them as the product manager and you say
that let's say that you want to create a
you think that there's a there's a
product need in the world for something
like songs to sing in the car or someone
sitting in the shower and I'm taking
that example because it exists people
love yeah The Scream songs in the car
when they drive right yeah so you want
to create that product and you have this
product manager if it's a musical expert
they create they come up with a concept
like I think this is a missing thing in
in humanity like upside it's called song
sitting in the car they they create the
framing the image the title and they
create a test set or they create a group
of songs like a few thousand songs
out of the catalogue that they manually
curate that our known songs that are
great to sing in the car and they can
take like to romance into account they
understand things that our algorithms do
not at all so they have this huge set of
tracks then when we deliver that to you
we look at your taste vectors and you
get the 20 tracks that are songs to sing
in the car in your taste so you have you
have personalization and editorial input
in the same process if that makes sense
yeah it makes total sense and I have
several questions around that this is a
this is like fascinating okay so first
it is a little bit surprising to me that
the world expert humans are
outperforming machines at specifying
songs to sing in the car so maybe you
could talk to that a little bit I don't
know if you can put it into words but
what is it how difficult is this problem
of tea do you really I guess what I'm
trying to ask is there how difficult is
it to encode the cultural references the
the context of the song the artists all
those things together can machine
learning really not do that I mean I
think machine learning is great at
replicating patterns if you have the
patterns but if you try to write with me
a spec of what songs greatest song
testing in the card definition is is it
is it loud there's many choruses
should've been in movies it's it quickly
gets incredibly complicated right yeah
and and a lot of it may not be in the
structure of the song or the title it
could be cultural references because you
know it was a history once so so the
definition problems quickly get and I
think that was the that was the insight
of Andrew Aang when he said the job of
the product manager is to understand
these things that that algorithms don't
and then define what that looks like and
then you have something to train towards
right then you have kind of the test set
and then so so today the editors create
this pool of tracks and then we
personalize you could easily imagine
that once you have this set you could
have some automatic exploration on the
rest of the catalogue because then you
understand what it is and then the other
side of it what machine learning
does help is this taste vector how hard
is it to construct a vector that
represents the things an individual
human likes it this human preference so
you can you know music isn't like it's
not like Amazon like things you usually
buy music seems more amorphous like it's
this thing that's hard to specify like
what what is what you know if you look
at my playlist what is the music that I
love it's harder it seems to be much
more difficult to specify concretely so
how hard is it to a build the taste
vector it is very hard in the sense that
you need a lot of data and I think what
we found was that so it's not so it's
not a stationery problem most it changes
over time and so we've gone through the
journey of if you've done a lot of
computer vision obviously I've done a
bunch of computer vision in my past and
and we started kind of with the
handcrafted heuristics for you know this
is kind of in the music this is this and
if you consume this you probably like
this so we have we started there and we
have some of that still then what was
interesting about the playlist data was
that you could find these latent things
that wouldn't necessarily even make
sense to you that could could even
capture maybe cultural references
because they Co occurred things that
they wouldn't have appeared
mechanistically either in the content or
so forth so I think that I think the
core assumption is that there are
patterns in in almost everything and if
there are patterns these these embedding
techniques are getting better and better
now if now as everyone else we're also
using kind of deep embedding so you can
encode binary values and so forth and
what I think is interesting is is this
process to try to find things that they
do not necessarily you wouldn't actually
have have guessed so it is very hard in
a in an engineering sense to find the
dimensions it's an incredible
scalability problem to do for hundreds
of millions of users and to update it
every day but in but in theory in theory
embeddings isn't that complicated the
fact that you try to find some principal
components or some like that
dimensionality reduction is also the
theory I guess is easy that the
practices is very very hard and it's a
it's a huge engineering challenge but
fortunately we have some amazing
research and engineering teams in this
space yeah I guess the the question is
all I mean it's similar I deal with it
with an autonomous vehicle spaces the
question is how hard is driving and here
is basically the question is of edge
cases so embedding probably works not
probably but I would imagine works well
in a lot of cases so there's a bunch of
questions that arise then so do song
preferences does your taste vector
depend on context like mood right so
there's different moods and absolutely
so how does that take in is it is it
possible to take that at consideration
or do you just leave that as a interface
problem that allows the user to just
control it so when I'm looking for
workout music I kind of specify it by
choosing certain players doing certain
search yeah so that's a great point it's
back to the product development you
could try to spend a few years trying to
predict which mood you're in
automatically open Spotify or you create
a tab which is happy and sad right and
you're gonna be right 100% of the time
with one click now it's probably much
better to let the user tell you if
they're happy or sad or if they want to
work out on the other hand if you use
the interface become 2,000 tabs you're
introducing so much friction so no one
will use the product so then you have to
get better so it's this thing where I
think it maybe was I remember who coined
it but it's called fault tolerant you is
right you build the UI that is tolerant
of being wrong and then you can be much
less right in your you know in your
algorithms
so we you know we've had to learn a lot
of that building the right UI that fits
where the where the machine learning is
and and a great discovery there which is
which was by the teams during one of our
hack days was this thing of taking
discovery packaging it into a playlist
and saying that these are new tracks
that we think you might like based on
this and studying the right expectation
made it made it a great product so I
think we have this benefit that for
example Tesla doesn't have that we can
we can we can change the expectation we
can we can build a fault-tolerant
setting it's very hard to be fault
tolerant when you're driving at you know
100 miles per hour or something and and
we we have the luxury of being able to
say that of being wrong if we have the
right UI which gives us different
abilities to take more risk so I
actually think the self-driving problem
is is much harder oh yeah for sure
it's much less fun because people die
exactly and in Spotify it's just such a
more fun problem because failure will
mean failures beautiful away at least
exploration so it's a really fun
reinforcement learning project the worst
case scenario is to get these WTF tweets
like how did I get this this song which
is a lot better than the self-driving
favor so what's the feedback that a user
puts the signal that a user provides
into the system so the the you mentioned
skipping what is like the strongest
signal is uh you didn't mention clicking
like so so we have a few signals that
are important obviously playing playing
through so so one of the benefits of
music actually even compared to podcast
or or movies is the object itself is
really only about three minutes so you
get a lot of chances to recommend and
the feedback loop is is every three
minutes instead of every two hours or
some things you actually get kind of
noisy but but quite fast feedback and so
you can see if people played through or
if they which is you know the inverse of
skip really that
an important signal on the other hand
much of the consumption happens when
your phone is in your pocket maybe
you're running or driving or you're
playing on a speaker and so you not
skipping doesn't mean that you love that
song it might be that it wasn't bad
enough that you would walk up and skip
so it's a noisy signal then then we have
the equivalent of the like which is you
saved it to your library that's a pretty
strong signal of affection and then we
have the more explicit signal of play
listing like you took the time to create
a playlist you put it in there there's a
very little small chance that if you
took all that trouble this is not a
really important track to you and then
we understand also what are the tracks
it relates to so we have we have the
playlist thing we have the like and then
we have the listening or skipped and you
have two very different approaches to
all of them because at different levels
of noise one one is very voluminous but
noisy and the others rare but you can
you can probably trust it yeah it's
interesting I think between those
signals captures all the information
you'd want to capture I mean there's a
feeling a shallow feeling for me that
there's sometimes I'll hear songs like
yes this is you know this was the right
song for the moment but there's really
no way to express that fact except by
listening through it all the way yeah
and maybe playing it again at that time
or something yeah
there's no need for a button that says
this was the best song I could have
heard at this moment well we're playing
around with that with kind of the thumbs
up concept saying like I really like
this just kind of talking to the
algorithm it's unclear if that's the
best way for you miss to interact maybe
it is maybe they should think of Spotify
has a person an agent sitting there
trying to serve you and you can say like
that's modified good Spotify right now
the analogy we've had is more you
shouldn't think of us we should be
invested oh and the feedback is if you
save it it's kind of you work for
yourself you do a playlist because you
think is great and we can learn from
that it's kind of back to back to Tesla
how they kind of have that shadow mode
they sit in what you drive we kind of
took the same analogy we stayed in watch
your playlist and then maybe we can we
can offer you an autopilot where you can
take over for a while or something like
that and then back off if you say like
that's not that's not good enough
but but I think it's interesting to
figure out what your mental model s if
Spotify is an AI that you talk to which
I think might be a bit too abstract for
for many consumers or if you still think
of it as it's my music app but it's just
more helpful and depends on the device
it's running on which brings us to smart
speakers so I have a lot of the Spotify
listening I do is on things like on
device they can talk to you whether it's
from Amazon Google or Apple what's the
role Spotify on those devices how do you
think of it differently then on the
phone or on the desktop there are few
things to say about the first of all
it's incredibly exciting they're growing
like crazy especially here in the in the
in the US and it's solving a consumer
need that I think is is you can think of
it as just remote interactivity you can
control this thing from from from across
the room and it might may feel like a
small thing but it turns out that
friction matters to consumers being able
to say play pause and so forth from
across the room is very powerful so
basically you made you made the living
room interactive now and what we see in
our data is that the the number one use
case for these speakers is music music
and podcasts so fortunately for us it's
been important to these companies to
have those use case covered so they
wanted Spotify on this we have very good
relationships with with them and we're
seeing we're seeing tremendous success
for them what what I think it's
interesting about them is it's already
working with we kind of had this
epiphany many years ago back when we
started using Sonos if you went through
all the trouble of setting up your sonar
system you had this magical experience
where you had all the music ever made in
your living room and and we we we made
this assumption that the home everyone
use that a CD player at home but they
never managed to get their files working
in the home having this network attached
which was too cumbersome for most
consumers so we made the assumption that
the home would skip from the CD all the
way to streaming books where where you
will get would by the steering without
all the music built-in that took longer
than we thought but with the voice
speakers that was the unlocking that
made kind of the connected speaker
happen in the home so so it really it
really exploded and we saw this
engagement that we predicted would
happen what I think is interesting
though is where it's going from now
right now you think of them as voice
speakers but I think if you look at
Google i/o for example they just added a
camera to it where you know when the
alarm goes off instead of saying hey
Google stop you can just wave your hand
so I think they're gonna think more of
it as a as an agent or as an assistant
truly an assistant and an assistant that
can see you it's gonna be much more
effective than then a blind assistant so
I think these things will morph and we
won't necessarily think of them as
quote-unquote voice speakers anymore
just as interactive access to the
Internet in the home but I still think
that the biggest use case for those will
be will be audio so for that reason
we're investing heavily in it and we've
built our own NLU stack to be able to
the the challenge here is how do you
innovate in that well it's it's it
lowers friction for consumers but it's
also much more constrained there you
have no pixels to play with in an in an
audio-only world it's really the
vocabulary that is the interface so we
started investing and playing around
quite a lot with that trying to
understand what the future will be of
you speaking and gesturing and waving at
your music and actually you're actually
nudging closer to the autonomous vehicle
space because from everything I've seen
the level of frustration people
experience upon failure of natural
language understanding is much higher
than failure in other content people get
frustrated really fast so if you screw
screw that experience up even just a
little bit they give up really quickly
yeah and I think you see that in the
data while well it's tremendously
successful
the most common interactions are play
pause
and you know next the things were if you
compare it to taking up your phone
unlocking you bringing up the app and
skipping clicking skip yeah it was it
was much lower friction but then for for
longer more complicated things like you
if I mean that song about people
stopping up the phone and search and
then play it on their speaker so we
tried again to build a fault-tolerant UI
where for the more for the more
complicated things you can still pick up
your phone have powerful full keyboard
search and then try to optimize for
whether it's actually lower friction and
try to it's it's kind of like the test
autopilot thing you have to be at the
level where you're helpful if you're too
smart and just in the way people are
gonna get frustrated and first of all
I'm not obsessed with stairway to heaven
it's just a good song but let me mention
that as a use case because it's an
interesting one I've literally told one
of I don't want to say the name of the
speaker cuz though when people are
listening to it'll make their speaker go
off but I talk to the speaker and I say
play stairway to heaven and every time
it like not every time but a large
percent of the time plays the wrong
stairway to heaven it plays like some
cover of the and that that part of the
experience I actually wonder from a
business perspective the Spotify control
that entire experience or no it seems
like the NLU the the natural language
stuff is controlled by the speaker and
then Spotify stays at a layer below that
it is a good and complicated question
some of which is dependent on the on the
partner so it's hard to comment on the
own specifics but the question is the
right one the challenges if you can't
use any other personalization I mean we
know which stairway to heaven and and
the truth is maybe for for one person it
is exactly the cover that they want and
they will be very frustrated if a place
I I think we I think we default to the
right version but but you actually want
to be able to do the cover for the
person that just played the cover 50
times or Spotify is just gonna seem
stupid so you want to be able to
leverage the personalization but you
have this stack where where you have the
the ASR and this thing called the N best
list so then that guesses
here and then the presentation comes in
at the end you actually want the
personalization to be here when you're
guessing about what they actually meant
so we're working with these partners and
it's a complicated it's a complicated
thing where you want to you want to be
able so first of all you want to be very
careful with you users data
you don't share your users data without
the permission but you want to share
some data so that their experience gets
better
so that these partners can understand
enough but not too much and so forth so
it's really the trick is that it's like
a business driven relationship where
you're doing product development across
companies together yeah which is which
is really complicated but this is
exactly why we built our own NLU so that
we actually can make personalized
guesses because this is the biggest
frustration from a user point of view
they don't understand about ASRs and n
best lists and and business deals
they're like how hard can it be
I've told this thing 50 times in this
version and still applies the wrong
thing it can't it can't be hard so we
try to take the user approach if the
user the users not going to understand
the complications of business we have to
solve it let's talk about sort of a
complicated subject that I myself I'm
quite torn about the idea of sort of
paying artists right I saw as of August
31st 2018 over 11 billion dollars were
paid to rights holders so and further
distributed to artists from Spotify so a
lot of money is being paid to artists
first of all the whole time is a
consumer for me when I look at Spotify
I'm not sure I'm remembering correctly
but I think you said exactly how I feel
which is this is too good to be true
like when I started using Spotify I
assume you guys will go bankrupt in like
a month it's like this is too good a lot
of people did like this is amazing so
one question I have is sort of the
bigger question how do you make money in
this complicated world how do you deal
with a relationship with record labels
who
are complicated these big you're
essentially and have the task of herding
cats
but like rich and powerful cats and also
have the task of paying artists enough
and paying those labels enough and still
making money in the internet space or
people are not willing to pay hundreds
of dollars a month so how do you
navigate the space how do you know
that's a beautiful description hurting
rich cats yeah that's before it is very
complicated and I think certainly
actually betting against Porter fire has
been statistically a very smart thing to
do just looking at the at the line of
roadkill in music streaming services
it's it's kind of I think if I had
understood the complexity when I joined
Spotify unfortunately fortunately I
didn't know enough about the music
industry to understand the complexities
because then I would have made a more
rational guess that it wouldn't work so
you know ignorant is bliss but I think
there have been a few distinct
challenges I think as I said one of the
things that made it work at all was that
Sweden and the Nordic was a lost market
so there was you know there was there
was no risk for labels to try this I
don't think it would have worked if if
the market was was healthy so so that
was the initial condition then then we
had this tremendous challenge with the
model itself so now most people were
pirating but for the people who bought a
download or CD the artists will get all
the revenue for all the future plays
then right so you got it all up front
whereas the streaming model was like
almost nothing they want almost nothing
they - and then at some point this curve
of incremental revenue would intersect
with your day one payment and that took
a long time to play out before before
the music labels they understood that
but on the other side it took a lot of
time to understand that actually if I
have a big hit that is gonna be played
for for many years this is a much better
model
because I get paid based on how much
people use the product not how much they
thought they would use it day one or so
forth so it was a complicated model to
get across and but time helped with that
right and now now the revenues to the
music industry actually are bigger again
then you know it's gone through this
incredible dip and now they're back up
and so we're we're very really proud of
having having been a part of that so
there have been distinct problems I
think when it comes to the to the labels
we have taken the painful approach some
of our competition at the time they kind
of they kind of look that other
companies instead if we just if we just
ignore the rights and we get really big
really fast we're gonna be too big for
the the labels too kind of too big to
fail they're not gonna kill us we didn't
take that approach we went legal from
day one and we we negotiated and
negotiated and negotiated was very slow
is very frustrating we were angry at
seeing other companies taking shortcuts
and seeming to get away with it it was
this this this game theory thing where
over many rounds of playing the game
this would be the right strategy and
even though to clearly there's a lot of
frustrations at times during
negotiations there is this there is this
weird trust where we have been honest
and fair we never screw them the inner
screwed us it's ten years but there's
this trust in like they know that if
music doesn't get really big if lots of
people do not want to listen to music I
want to pay for it
Spotify has no business model so we
actually are incredibly aligned right
other companies not to be tense but
other companies have other business
models where even if they may know music
from no money from music they still be
profitable companies but fall if I want
so and I think the industry sees that we
are actually aligned business-wise so
there is this distrust that allows us to
to do product development even if it's
scary you know taking risks the free
model itself was an incredible risk for
the music industry to take that they
should get credit for now some of it was
that they had nothing to lose in Sweden
but frankly a lot of the labels also
took risk and so I think we built up
that trust
with a I think hurting what cats sounds
a bit what's the word it sounds like I'm
dismissive of the guests dismissive no
everyday yeah you matter they're all
beautiful and and very important exactly
they've taken a lot of risks and
certainly it's been frustrating about it
yeah it's it's it's really like playing
its its game theory if you play that if
you play the game many times and then
you can have this statistical outcome
that you bet on and it feels very
painful when you're in the middle of
that thing I mean there's risk this
trust there's relationships from just
having read the biography of Steve Jobs
similar kind of relationship were
discussed in iTunes the idea of selling
a song for a dollar was very
uncomfortable for labels and exactly and
there was no it was the same kind of
thing it was trust it was game theory as
is a lot of our relationships that had
to be built and it's really a
terrifyingly difficult process that
Apple could go through a little bit
because they could afford for that
process to fail for Spotify it seems
terrifying because you can't initially I
think a lot of it comes out comes down
to honestly Daniel and his tenacity in
in negotiating which seems like an
impossible this is a fun task because he
was completely unknown and so forth but
maybe that was also the reason that that
it worked
but I think yeah I think game theory is
probably the best way to think about it
you could straight go straight for this
like Nash equilibrium that someone is
going to defect or or you played many
times you try to actually go for the top
left the corporation cell is there any
magical reason why Spotify seems to have
won this so a lot of people have tried
to do a spot if I try to do and Spotify
has come out well so the order is that
there's no magical reason because I I
don't believe in magic but I think there
are there are reasons and I think some
of them are that people have
misunderstood a lot of what we actually
do the actual the actual Spotify model
is very complicated they've looked at
the premium model and said it seems like
you can you can charge $9.99 for music
and people are gonna pay but that's not
what happened actually when we launched
the original mobile product everyone
said they would never pay what happened
was they started on the on the free
product and then their engagement grew
so much that eventually they said maybe
it is worth $9.99 right it's it's your
propensity to pay grows with your
engagement so we had this super
complicated business model we operate
two different business model advertising
and premium at the same time and I think
that is hard to replicate however I
struggled to think of other companies
that run large-scale advertising and
subscription products at the same time
so I think the business model is
actually much more complicated than
people think it is and and so some
people went after just the premium part
without the free part and ran into a
wall where no one wanted to pay some
people went after just music music
should be free just ads which doesn't
give you enough revenue and doesn't work
for the music industry so I think that
combination is kind of a pick from the
outside so maybe I shouldn't say it here
and reveal the secret but that that
turns out to be hard to replicate then
you with then you would think so there's
a lot of brilliant business strategy
here brilliance or luck
probably more luck but it doesn't really
matter it looks brilliant
in retrospect let's call it brilliant
yeah when the books are read no
brilliant you've mentioned that your
philosophy is to embrace change so how
will the music streaming and music
listening world change over the next 10
years 20 years you look out into the far
future what do you think I think that
music and for that matter
audio podcasts audiobooks I think it's
one of the few core human needs I think
it there is no good reason to me why it
shouldn't be at the scale of something
like messaging or social networking I
don't think it's a nice thing to listen
to music or news or something so I think
scale is obviously one of the things
that I really hope for I think I hope
that it's going to be billions of users
I hope eventually everyone in the world
gets access to all the world's music
ever made so obviously I think it's
going to be a much bigger business
otherwise we we wouldn't be betting this
big now if you if you look more at how
it is consumed what I'm hoping is back
to this analogy of the software tool
chain where I think I sometimes
internally I make this analogy to to
text messaging text messaging was also
based on standards in the in the area of
mobile carriers you had the SMS the 140
character 2020 carrot SMS and it was
great because everyone agreed on the
standard so as a consumer you got a lot
of distributions and interoperability
but it was a very constrained format and
and when the industry wanted to add
pictures to that format to do the MMS I
looked it up and I think it took from
the late 80s to early 2000 this is like
a 15 20 year product cycle to bring
pictures into that now once that entire
value chain of creation and consumption
got wrapped in one software stack with
in something like snapchat or whatsapp
like the first week they had a
disappearing messages like then two
weeks later they added stories like the
pace of innovation when you're on one
software stack and you can you can you
can affect both creation and consumption
I think is going to be rapid so with
these streaming services we now for the
first time in history have enough I hope
people on one of these services actually
whether it's Spotify or Amazon or Apple
or or YouTube and hopefully enough
creators that you can actually start
working with a format again and and that
excites me I think being able to change
these constraints from a hundred years
that could really that could really do
something interesting I don't I really
hope it's not just going to be their
iteration on on the same thing for the
next ten to twenty years as well
yeah changing the creation of music
equation of audio equation of podcast is
a really fascinating possibility I
myself don't understand what it is about
podcast that's so intimate it just is I
listen to a lot of podcasts I think it
touches on a human and a deep human need
for connection that people do feel like
they're connected to when they listen I
don't understand what the psychology
that is but in this world that's
becoming more and more disconnected it
feels like this is fulfilling a certain
kind of need and empowering the creator
as opposed to just the listener it's
really interesting so this is I'm really
excited that you're working on it yeah I
think one of the things that is
inspiring for our teams to work on
podcast is exactly that what do you
think like I like I probably do that
it's something biological about
perceiving to be in the middle of the
conversation that makes you listen in a
different way it doesn't really matter
people seem to perceive it differently
and there was this narrative for a long
time that you know if you look at video
everything kind of in the foreground I
got shorter and shorter and shorter
because of financial pressure is a
monetization and so forth an event at
the end there zones like 20 seconds
clipped people just screaming something
and I'm really I feel really good about
the fact that you you could have
interpreted that as people have no
attention span anymore they don't want
to listen to things they're not
interested in deeper stories like you
know people are people are getting
dumber
but then podcast came along and it's
almost like no no they need still
existed once what but maybe maybe it was
the fact that you're not prepared to
look at your phone like this for two
hours but if you can drive at the same
time it seems like people really want to
dig
and they want to hear like the more
complicated version so to me that is
very inspiring that that podcast is
actually long-form it gives me a lot of
hope for for Humanity that people seem
really interested in hearing deeper more
complicated this is a I don't understand
it it's fascinating so the majority for
this podcast listen to the whole thing
this whole conversation we've been
talking for an hour and 45 minutes and
somebody will I mean most people will be
listening to these words I'm speaking
right now they wouldn't have thought
that 10 years ago with what the world
seemed to go that's very positive I
think that's really exciting and
empowering the creator and there's
really exciting last question you also
have a passion for just mobile in
general how do you see the smartphone
world this the the digital space of of
smartphones and just everything that's
on the move whether it's Internet of
Things and so on changing over the next
10 years and so on I think that one way
to think about it is that computing
might be moving out of these
multi-purpose devices the computer we
had in the phone into specific specific
specific purpose devices and you know it
will be ambient that you know at least
in my home you just shout something at
someone and there's always like one of
these speakers close enough and so you
start behaving differently it's as if
you have the Internet
ambient ambiently around you and you can
ask it things
so I think computing or kind of get more
integrated and we won we won't
necessarily think of that as as
connected to a device and the same thing
in the same way that we do today I don't
know the path to that maybe we used to
have these desktop computers and then we
partially replace that with the with the
laptops and left you know at home and at
work and then we got these phones and we
started leaving the laptop at home for a
while and maybe the maybe for stretches
of time you're gonna start using the
watch and you can leave your your phone
at home like for a run or something and
you know we're on this progressive path
where you I think what what is happening
with the voice is that you haven't you
have an interactive interaction paradigm
that doesn't require as large physical
devices so I definitely think there's a
future where you can have your your
airports and and your watch and you can
do a lot of computing and I don't think
it's gonna be this binary thing I think
it's gonna be like many of us still have
a laptop we just use it less and so you
shift your your consumption over and I
don't know about a or glasses and so
forth um I'm excited about I spend a lot
of time in that area but I still think
it's quite far away they are VR oh yes
VR is is happening and working I think
the recent oculus quest is quite
impressive I think AR is further away at
least that type of AR I think but I do
think your phone or water glasses
understanding where you are and maybe
what you're looking at and being able to
give you all your cues about or you can
say like what is this and it tells you
what it is that I think might happen you
you know you use your your watch to your
glasses there's a as a mouse pointer on
reality I think it might be a while
before I might be wrong I hope I'm wrong
I think it might be a while before we
walk around with these big like lab
glasses then project things I agree with
you there's a it's actually really
difficult when you have to understand
the physical world enough to project
onto it well I lied about the last
question
because I just thought of audio and my
favorite topic which is the movie her
mm-hmm do you think well there's part of
Spotify or not will have I don't know if
you've seen the movie her absolutely and
their audio is the primary form of
interaction and the connection with
another entity that you can actually
have a relationship with actually fall
in love with based on voice alone audio
alone you how far do you think that's
possible first of all based on audio
alone to fall in love with somebody
somebody or well yeah let's go with
somebody just have a relationship based
on audio alone and second question to
that can we create an artificial
intelligence system that allows one to
fall in love with it and hurt him with
you so there's my personal personal
answer speaking for me as a person the
answer is quite unequivocally yes on
both I think what we just said about
podcasts and the feeling of being in the
middle of a conversation if you could
have an assistant where and we just said
that feels like a very personal settings
so if you walk around with these
headphones and this thing you're
speaking with this thing all of the time
that feels like it's in your brain I
think it's it's gonna be much easier to
fall in love with than something that
would be on your screen I think that's
entirely possible and then from that you
can programs is better than me but from
the concept of if it's going to be
possible to build a machine that they
can achieve that I think whether you
whether you think of it as if you can
fake it the philosophical zombie that it
simulates it enough or it it somehow
actually is I think there's it's only
question if you ask me about time I'd
had to be a financier but if you say a
given some half infinite time absolutely
I think it's just atoms and arrangement
of information well I personally think
that love is a lot simpler than people
think so we started with true romance
and ended in love I don't see a better
place to end beautiful Gustav thanks so
much for talking to
thank you so much there was a lot of fun
was fun
you