Kind: captions
Language: en
the following is a conversation with poe
shen lo a professor of mathematics
at carnegie mellon university national
coach of the usa
international math olympia team and
founder of
xp that does online education of basic
math and science
he's also the founder of novid an app
that takes a
really interesting approach to contact
tracing making sure you stay completely
anonymous
and it gives you statistical information
about covet cases in your
physical network of interactions so you
can maintain privacy
very important and make informed
decisions
in my opinion we desperately needed
solutions like this in
early 2020 and unfortunately i think
we will again need it for the next
pandemic
to me solutions that require large-scale
distributed coordination of human beings
need ideas that emphasize freedom and
knowledge
quick mention of our sponsors jordan
harbinger show
on it betterhelp eight sleep and
element check them out in the
description to support this podcast
as a side note let me say that poe and i
filmed a few short videos about simple
beautiful math concepts
that i will release soon it was really
fun
i really enjoyed poe sharing his passion
for math with me in those videos
i'm hoping to do a few more short videos
in the coming months
that are educational in nature on ai
robotics math science philosophy
or if all else fails just fun snippets
into my life on music
books martial arts and other random
things
if that's of interest to anyone at all
this is the lex friggman podcast and
here's my conversation
with po shenlo you know you mentioned
you really enjoy
flying and experiencing different people
in different places
there's something about flying for me i
don't know if you have the same
experience
that every time i get on an airplane
it's incredible to me
that human beings have actually been
able to achieve this
and when i look at like what's happening
now with humans traveling out into space
i see it as all the same thing it's
incredible that humans are able to get
into a box
and fly in the air and
and safely and land in the same it seems
like
and everybody's taking it for granted so
when i observe them
it's quite fascinating because i see
that cleanly mapping
to the world where we're now on uh
in rockets and traveling to the moon
traveling to mars
and at the same kind of way i can
already see the future
where we will all take it for granted
so i don't know i don't know if you have
uh you personally when you fly
have the same kind of magical experience
of like how the heck did humans actually
accomplish this
so i do especially when there's
turbulence
which is you know like on the way here
yeah there was turbulence and
the the plane jiggled even the flight
attendant had to hold on to the side
and i was just thinking to myself it's
amazing that this happens all the time
and the wings don't fall off you know
like
given how many planes are flying but
then i often think about it and i'm like
you know a long time ago i think people
didn't trust elevators
yeah in a 40-story building in new york
city and
now we just take it completely for
granted that you can step into this
shaft
which is 40 floors up and down and it
will just not
fail yeah again i'm the same way with
elevators but also buildings
when i'll stand on the 40th floor and
wonder
how the heck are we not falling right
now
like how how amazing it is with the high
winds
like structurally just the earthquakes
and the vibrations i mean
natural vibrations in the ground like
how is this how are
all of these you go like new york city
all of these buildings standing
i mean to me one of the most beautiful
things actually mathematically too is uh
bridges
i used to build bridges in high school
from like toothpicks just like out of
the pure joy
of like physics making
some structure really strong
understanding
like from a civil engineering
perspective what kind of
structure will be stronger than another
kind of structure like suspension
bridges and then you see that at scale
humans being able to span a body of
water with a giant bridge
and it's i don't know it it's so
humbling
it makes you realize how
how dependent we are on each other sort
of i talk about love a lot but
there is there's a certain element in
which we
little ants have just a small amount of
knowledge about
our particular thing and then we're
depending on a network
of knowledge that other experts hold and
then
most of our lives most of the quality of
life we have has to do with the
the richness of that network of
knowledge
of that collaboration and then sort of
the ability to build on top of it
levels of abstractions you start from
like bits in a computer
then you can have assembly and you can
have c plus so you have an operating
system that you can see plus plus and
python finally some machine learning on
top
all of these are abstractions and
eventually we have ai that runs all of
us humans but
anyway uh but speaking of abstractions
and programming in high school you wrote
some impressive games for amaz i got a
chance to
in browser somehow it's magic got a
chance to play them
alien attack one two three and four
what's the hardest part about
programming those games
and maybe can you tell the story about
about building those games
sure i actually tried to do those in
high school because i was just curious
if i could
and yeah and that's a good starting
point for anything right yeah yeah it's
like could you but
the appealing thing was also it was a
soup to nuts kind of thing
so something that has always attracted
me is i like
beautiful ideas i like seeing beautiful
ideas but i actually also like
seeing execution of an idea all the way
from beginning to end in something that
works so for example in high school i
was lucky enough to grow up
in the late 90s when even
a high school student could hope to make
something sort of comparable to the
shareware games that were out there
not i say the word sort of like still
quite far away but at least i didn't
need to hire a 3d
cg artist there weren't enough pixels to
draw anybody
even i can draw right bad art of course
but
the point is i wanted to know is it
possible for me to try to do those
things
where back in those days you didn't even
have an
easy way to draw letters on the screen
in a particular font
you couldn't just say import a font it
wasn't like python so
for example back then if you play those
games in the in the web browser which is
emulating
um the the old school computer
those even the letters you see those are
made by individual calls to draw pixels
on the screen so you built that from
scratch almost building a computer
graphics library from scratch
yes the primitive that i got to use was
some code i copied off of a book
in assembly of how to put a pixel on the
screen in a particular color
and the programming programming language
was pascal
ah yeah the first one was in pascal but
then the
other ones were in c plus plus after
that how's the emulation in the browser
work by the way is that is that true
real
because it's pretty cool you get to play
these games that have a very much
90s feeling to them ah so it's literally
making an ms-dos environment which is
literally
running the old exe file wow
that could be more amazing than the
airplane so it wasn't so much about the
video games it was more about can you
build something
really cool from scratch yes and
you did a bunch of programming
competitions
what was your interest your love for
programming
what did you learn to that experience
especially now that
as much of your work has taken a long
journey through mathematics
i think i always was amazed by how
computers
could do things fast if i wanted to make
it an abstract
analysis of why it is that i saw some
power in the computer
because if the computer can do things so
many times faster than humans
where the hard part is telling the
computer what to do and how to do it
if you can master that asking the
computer what to do
then you could conceivably achieve more
things and those contests i was in
those were the opposite in some sense of
making
a complete product like a game as a
product
those contests were effectively write a
function to do something extremely
efficiently
and if you are able to do that then you
can unlock more of the power of the
computer
but also doing it quickly there's a time
element from the human perspective to be
able to
program quickly there's something nice
so there's like almost like an athletics
component
to where you're almost like uh an
athlete
seeking optimal performance as a human
being trying to write these programs
and at the same time it's kind of art
because you're
the best way to write a program quickly
is to write a simple program
you used to have a damn good solution so
it's not necessary you have to type fast
you have to think through a really clean
beautiful
solution i mean
what do you think is the use of those
programming competitions do you think
they're ultimately something you would
recommend for students for people
interested in programming or people
interested in building stuff
yes i think so because especially with
the work that i've been doing nowadays
even
trying to control kovit something that
was very helpful from day one
was understanding that the kinds of
computations we would want to do
we could conceivably do on like a four
core cloud machine on amazon web
services
out to a population which might have
hundreds of thousands or millions of
people
the reason why that was important to
have that back of the envelope
calculation with efficient algorithms
is because if we couldn't do that then
we would bankrupt ourselves before we
could get to a big enough skill
if you think about how you grow anything
from small to big
if in order to grow it from small to big
you also already need 10
000 cloud servers you'll never get too
big
and also the nice thing about
programming competitions is that you
actually build
a thing that uh works so you
you finish it there's a completion thing
and you realize
i think there's a magic to it where you
realize that
it's not so hard to build something that
works
to have a system that uh successfully
takes in inputs and produces outputs and
solves a difficult problem
and that directly transfers to building
a startup essentially that can help
some aspect of this world as long as
it's mostly be
based on software engineering things get
really tricky when you have to
manufacture stuff
that's why people like elon musk are so
impressive that they it's not just
software
tesla autopilot is just not just
software it's it's like you have to
actually
like have factories that build uh cars
and there's like a million components
involved in in the machinery required to
assemble those cards and so on but in
software one person
can change the world which is uh uh
incredible
but on the mathematics side what uh if
you
look back or maybe today what made you
fall in love with mathematics
for me i think i've always been very
attracted
to challenge as i already indicated with
the writing the program
i guess if i see something that's hard
or supposed to be impossible it
certain sometimes i say maybe maybe i
want to see if i can pull that off
and with the mathematics the math
competitions presented problems
that were hard that i didn't know how to
start
but for which i could conceivably try to
learn how to solve them
so i mean there are other things that
are hard called like get something to
mars get people to mars
and i didn't i still don't think that
i'm
able to solve that problem on the other
hand the math problems struck me as
things which are
hard and with significant amount of
extra work i could figure it out
and maybe they would actually even be
useful like that mathematical skill is
the core
of lots of other things
that's really interesting maybe you
could speak to that because
a lot of people say that math is hard
as a kind of negative statement
it always seemed to me a little bit like
that's kind of a positive statement
that all things that are worth having in
this world are hard
i mean everything that people think
about
that they would love to do whether it's
sports
whether it's art music
and all the sciences they're going to be
hard if you want to do something special
so is there something you could say to
that idea that math is hard
should it be made easy or should it be
hard
ah so i think maybe i want to dig in a
little bit onto this hard part and say
uh i think the interesting thing about
the math is that
you can see a question that you didn't
know how to
start doing it before and over a course
of
thinking about it you can come up with a
way
to to solve it and so you can move from
a state of
not being able to do something to a
state of being able to do something
where you help to take yourself through
that instead of somebody else
spoon feeding you yes that technique so
actually here i'm already digging into
maybe part of my teaching philosophy
also which is that
i actually don't want to ever just tell
somebody
here's how you do something i actually
prefer to say here's an interesting
question
i know you don't quite know how to do it
do you have any ideas
this is i'm actually coming up with i'm
actually explaining another way that you
could try to do teaching
and i'm contrasting this to a method of
watch me do this
now practice it 20 times i'm trying to
say a lot of people
consider math to be hard because maybe
they can't remember
all of the methods that were taught but
for me
i look at the hardness and i don't think
of it as a memory hardness
i think of it as a can you invent
something
hardness and i think that if we can
teach more people
how to do that art of invention
in a pure cognitive way not as hard as
the actual hardware stuff right but like
in terms of the concepts and the
thoughts and the mathematics teaching
people how to
invent then suddenly actually they might
not even find math to be that
tiresomeness hard anymore but that
rewardingness hard
of i have the capability of looking at
something which i don't know what to do
and coming up with how to do it i
actually think we should be doing that
giving giving people that capability
so hard in the same way that invention
is hard
that is ultimately rewarding so maybe
you can dig in
that a little bit longer which is um
do you see basically the way to teach
math is to present a problem
and to give a person a chance to try to
invent a solution
without with minimal amount of
information first
is that is that basically how do you
build that muscle of invention
in a student yes so the way that i i
guess i have
two different sort of ways that i try to
teach actually one of them is
in fact this semester because all my
classes were remotely delivered i even
threw them all onto my youtube channel
so you can see
you can see how i teach at carnegie
mellon but i'd often say
hey everyone let's try to do this any
ideas
and that actually changes my role as a
professor
from a person who shows up for class
with a script of what i want to talk
through
i actually i don't have a script the way
i show up for class is
there's something that we want to learn
how to do and we're going to do it by
improv
i'm talking about the same method as
improv comedy which is where
you tell me some ideas and i'll try to
yes and them
you know what i mean and then together
we're gonna come up with a proof
of this concept where you were deeply
involved
in creating the proof actually every
time i teach the class we do every proof
slightly differently because it's based
on how the students came
up with it and that's how i do it when
i'm in person
i also have another line of courses that
we make that is delivered online
those things are where i can't do it
live but
the teaching method became also similar
it was just here's an interesting
question
i know it's out of reach why don't you
think about it and then automatic hints
we feed automatically hints
uh through you know through the internet
to go and let the person
try to invent so that's like a more
rigorous prodding of invention
but you did mention disease and
coven and you've been doing some very
interesting stuff from a mathematical
but also software engineering angle
of coming up with ideas it's back to the
i can
i see a problem i think i can help uh
so you stepped into this world can you
tell me about
your work there under the flag of novid
and uh both the the software and the
technical details of how the thing works
sure sure so first i want to make sure
that i say this is actually team effort
i happen to be the one speaking but
there's no way this would exist without
an incredible team of people who inspire
me every day to
work on this but i'll speak on behalf of
them so
the idea was indeed that we stepped
forward
in march of last year when the world
started to become our part of the world
started to become our part meaning the
united states
started to become paralyzed by coven the
shutdown started to happen
and at that time it started as a figment
of an idea which was
network theory which is the area of math
that i work in
could potentially be combined with
smartphones and some kind of
health information anonymized exactly
how
we didn't know yet we tried to
crystallize it and many months into this
work
we ended up accidentally discovering
a new way to control diseases which
is now what is the main impetus of all
of this work is to take this idea
and polish it and hopefully have it be
useful not only now
but for future pandemics the idea is
really simple to describe
um actually my main thing in the world
is i come up with obvious
observations that's that's i'll explain
it now einstein did the same thing
and he wrote a few short papers
but but so the idea is like this if we
describe
how usually people control disease
for a lot of history it was that you'd
find out
who was sick you'd find out who they've
been around
and you try to remove all of those
people from society against their will
yes now that's the problem the against
the will part
gives you the wrong kind of a feedback
loop which makes it hard to control the
disease because then the people you're
trying to control keep getting other
people sick
you can see already how i'm thinking and
talking about this feedback loops this
is actually related to something you
said earlier
about even like how skyscrapers stay in
the air uh the whole point is
control theory you actually want to or
even how an airplane stays
you need to have control loops which are
feedbacking in the right way
and what we observed was that the
feedback control loop for
controlling disease by asking people to
be removed from society against their
will
was not working it was running against
human incentives and you suddenly are
trying to control seven
billion eight billion people in ways
that they don't individually want
to necessarily do so here's the idea
and this is inspired by the fact that at
the core of our team were user
experience designers
that's actually the in fact the first
thing i knew we needed when we started
was to bring user experience
at the core okay but so um
the idea was suppose there was a penta
suppose hypothetically there was a
pandemic
what would you want you would want a way
to be able to live your life as much as
possible
and avoid getting sick can we make an
app to help
you avoid getting sick notice how i've
just articulated the problem
it is not can we make an app so that
after you are around somebody who's sick
you can be removed from society it's can
we make an
app so that you can avoid getting sick
that would
run a positive feed however i don't know
if i want to call it positive or
negative but they would run a good
feedback loop yes
okay so then how would you do this the
only problem is that
you don't know who's sick because
especially with this disease
if i see somebody who looks perfectly
healthy the disease spreads two days
before you have any symptoms
and so it's actually not possible that's
where the network theory comes in
you caught it from someone what if
we changed the paradigm and we said
whenever there's a sickness
tell everybody how many physical
relationships
separate them from the sickness that is
the trivial idea we added
the trivial idea was the distance
between you and a disease
is not measured in feet or seconds
it's measured in terms of how many close
physical relationships
separate you like these six degrees of
separation like linkedin
simple idea what if we told everyone
that it turns out that actually unlocks
some interesting behavioral feedback
loops which
for example let me let me now jump to a
non-covered example to show why this
maybe it could be useful
actually we think it could be quite
useful imagine there was ebola or some
hemorrhagic fever
imagine it spread through contact
through the air in fact pretend
pretend that's a
that's a disastrous disease it has high
fatality rate
and uh as you die you're bleeding out of
every
orifice okay so yeah
no not pleasant not pleasant so the
question is suppose that such a disease
broke
who would want to install an app that
would tell them how many relationships
away from them
this disease had struck like a lot of
people a lot of people in fact
almost i don't want to say almost
everyone that's a very strong statement
but a very large number of people
that's fascinating framing like the the
more deadly and transmissible the
disease
the the stronger the incentive to
install it
in a positive sense the in in in the
good
feedback loop sense that's a really good
example it's a really good way to frame
it because
with covet it was not as deadly
as uh as potential pandemics could have
been viruses could have been so
it's sometimes muddled with how we think
about it but yeah this is a really good
framing if the virus was a lot more
deadly
you want to create a system that has a
set of incentives that
it quickly expresses a population where
everybody is using it
and is contributing in a positive way to
the system
exactly and actually that point you just
made i don't take credit for that
observation there was another person i
talked to who pointed out that
it's very interesting that this feedback
loop is even more effective
when the disease is worse and that's
actually
not a bad characteristic to have in your
feedback loop if you're trying to help
civilization keep running yeah it's a
really
it's in this dynamic like people
figure out they dynamically figure out
how bad the disease is
the more it spreads and the deadlier it
is as the people observe
it as long as the spread of information
like uh semantic information natural
language information
is closely aligned with the reality of
the disease which is a whole nother
conversation right
we that's we might maybe we'll chat
about that how we sort of
make sure there's not misinformation
while there's accurate information but
that aside okay so this is a really nice
property
right and and just going on on that
actually just talking more about what
that could do and why we're so excited
about
it it's that not only would people want
to install it
what would they do if you start to see
that this disease is getting closer and
closer we we surveyed informally
people but they said as we saw getting
closer we would
hide we would try to not have contacts
but now you notice what this has just
achieved the whole goal
on on this whole exercise was you got
the people who might be sick
and you got everyone else set a and set
b set a is the people who might be six
that b is everyone else
and for the entirety of the past uh
contact tracing approaches
you try to get set a to do things that
might not be
to their liking or their will because
that's removing them from society
yes we found out that there's two ways
to separate set a from set b
you can also let the people at set b at
the fringe of set a
attempt to remove themselves from this
interface it's just it's the symmetry of
a and b separation
everyone was looking at a we look at b
and suddenly b
is in their incentive to do so beautiful
so there's a virus that jumps from human
to human
so there's a network sometimes called
graph
of the spread of a virus it hops from
person to person to person to person
and each one of us individuals are
sitting
or plop plopped into that network we
have
close friends and relations and so on
it's kind of fascinating to actually
think about this network and we can
maybe talk about the shapes of this kind
of network
because i was i was trying to think
exactly this like how many people do i
was i'm kind of an introvert not kind of
i'm very much an introvert
but so can i be explicit about the kind
of people i
meet in regular life say when it was
completely opened up there's no pandemic
there is a kind of network of cl and
there's maybe
um in the graph theoretic sense there's
some weights
or something about how
close that relationship is in terms of
the frequency visits
the duration of business and all those
kinds of things so you're saying
we might want to be to create
on top of that network a spread of
information
to let you know as the virus travels
through this network how close is it
getting to you
and the number of hops away it is on
that network
is really powerful information that
creates a positive uh feedback loop
where you can act essentially
anonymously
uh and on your own like
nobody's telling you what to do which is
really important is decentralized
and uh and not yeah whatever the
opposite of authoritarian is
but you get to sort of the american way
you get to choose to do it yourself you
have the freedom to do it
yourself and you're incentivized to do
it and you're most likely going to do it
to to uh
to protect yourself against um
against you getting the disease as the
the closer it gets to you based on the
information that you have
but uh can you maybe elaborate uh first
of all brilliant
uh whenever i saw the thing you're
working on so forget for covid
this is of course really relevant for
covid
but it's also probably relevant for
future diseases as well so this
uh that was the thing i'm nervous about
like if this whole
if our society shut down because of
covid
like what the heck is gonna happen
when there's a much deadlier disease
like this
this is disappointing the whole time
2020 the whole time i'm just sitting
like this
like is the incompetence of
everybody except the people developing
vaccines
uh the biologists are the only ones that
got their stuff together
but in terms of institutions and all
that kind of stuff oh it's just been
it's just been terrible
but this is exactly the power of
information
and the power of information that
doesn't limit personal freedom
so your idea is brilliant okay
mathematically can you maybe elaborate
what are we talking about like how do
you actually make that work
what's involved sure first i'm going to
reply to something you said about
the freedom inside this because actually
that was the idea the idea
is this is game theory right and
effectively what we did is analogous to
free market economy as opposed to
central planning
yeah if you just line up the set of
incentives correctly
so that people have in their purely
selfish behavior
are contributing to the optimization of
the global function
yes that's it and the the point of what
we do i guess in mathematics is we try
to explore the search space
to go and find out as many possibilities
as there are and in this case it's an
apply
in this case it's an applied search
space that's why the inputs from design
user experience design and actual people
are important
but you asked about um i guess that the
tech the mathematical or the technical
things
underpinning it so i think the first
thing i'll say is
we wanted to make this thing not require
your personal information
and so in order to do that what gave me
the confidence
to i guess lead our team to run at the
beginning is we saw that this could be
done without using gps information
so technically what's going on is if two
smartphones
it's a smartphone app if two smartphones
have this thing installed
they just communicate with each other by
bluetooth
to go and find out how far they can they
can detect nearby things by bluetooth
and then they can find out that these
two phones were approximately such and
such distance apart
and that kind of relative proximity
information is enough to construct this
big network
okay so the physical network is
constructed based on proximity
that's through bluetooth and you don't
have to specify
your exact location it's the proximity
i'm not using the pythagorean theorem
basically i mean if i just knew the gps
coordinates we could use the pythagorean
theorem too sorry that's just how i call
it
distance formula whatever you want to
call it
[Laughter]
uh yeah so we're not doing the old
pythagorean
based violation of privacy okay
[Laughter]
but so is that
is that enough to form to give you
enough
information about physical connection to
another human being
is there a time element there is there
so
okay that sounds like a really strong
like low hanging fruit like if you have
that you could probably go really really
far
my natural question is is there extra
information you can add on top of that
like
the duration of the physical proximity
uh so first of all we actually do
estimate the duration
but the way we estimate the duration is
like how a movie is filmed
in the sense that every so often every
few minutes we
check what's nearby it's it's like how a
movie is filmed you take lots of
snapshots
yes so there's no way in a battery
efficient way
to really keep track of that proximity
however fortunately we're using
probability the fact is
the paradigm that we're using is it's
not super important
if you run into that person only for 10
minutes at the grocery store
if that's a stranger that you run into
10 minutes in this grocery store
that's not going to be relevant for our
paradigm because our paradigm is not
telling you who were you around before
and might therefore
have gotten infected by already ours is
about predicting the future
we changed from i mean the standard
paradigm was what already happened
quick damage control ours predict the
future if you run into that person once
in the grocery store today and never see
them again it's irrelevant for
predicting the future
and therefore for ours what really
matters is the many hours
around the other person at which point
if you're scanning every five to eight
minutes
that's going to come out in the problem
like statistically speaking it's going
to come out as a strong relationship and
a person
in the grocery store is going to wash
out that's not an important physical
relationship
i mean this is brilliant what uh
how difficult is it to make work so you
said one there's a mathematical
component
that we just kind of talked about and
then there's the user experience
component
so how difficult is it to go just like
you built the video game
alien attack from zero to
to completion what's involved how
difficult is it
so i'm going to answer that question in
terms of building the product
but then i'm also going to acknowledge
that just having an
app doesn't make it useful because the
the that's actually maybe the easy part
if you know what i mean there's like all
of this stuff about rollout adoption and
awareness
but let's focus on the app part first so
that's again why i said
the team is incredible so we have a
bunch of people
who let's just say that the technology
that we use to make it
is not the standard way you make an app
if you think about a standard
ios app or android app those are a user
interface
that contacts a web server and sends
some information back and forth
we're doing some stuff that has to hook
into the operating system
of saying let's go use bluetooth for
something it wasn't really meant for
right so there's that part and by the
way what is the app called
oh it's called novid covered with an ad
very nice so you have to hook into
bluetooth you're saying you have to do
that
um beyond the permissions that are
like at the very surface level provided
on the phone
well i don't want to call them
permissions i just want to say that's
not what you usually do with bluetooth
gotcha
usually with bluetooth you say do i have
headphones nearby
yes okay i'm done you don't go and say
do i have headphones nearby
or do i have another phone nearby which
is doing something and then keep asking
that system keep
asking the question right so so this is
actually not easy and
i mean there were some parts of it which
actually a lot of people had tried
unsuccessfully
actually it's known that for example the
uk was trying to do something
similar and the problem they ran into
was when you program things on ios
ios is very good at making it hard to do
things in the background
and so there was quite a lot of effort
required to go and make this thing work
so the whole point this thing would run
in the background and
ios i mean most
android probably as well right but yeah
iowa certainly makes it difficult for
something to run in the background
especially when it's to eating up your
battery right
ah well we wanted to make sure we didn't
eat up the battery so that one we can we
actually are very proud of the fact that
ours uses very little battery
uh actually even if compared to apple's
own system
so beautiful so what else is required to
make this thing work
right so the the key was that you had to
do significant amount of work on the
actual mobile app development
which fortunately the team that we
brought was this kind of general
thinkers
where we would dig in deep into the
operating system documentation and the
api libraries
so we got that working but there's
another angle which is you also need the
servers to be able to compute fast
enough
which is tying back to this old school
computer programming competitions and
math olympiads
in fact our team that was working on the
algorithm and back-end side
included several people who had been in
these
competitions from before which i happen
to know because i
i do coach the team for the math yes and
so we were able to bring people
in to build servers a server
infrastructure in
c plus actually so that we could support
significant numbers of people
without needing tons of servers is there
some distributed algorithms
working here or you basically have to
keep
in in the same place the entire graph as
it builds because
especially the more and more people use
it the bigger the bigger the graph gets
i mean this is very difficult uh scaling
problem right
ah so that's actually why uh this
computer algorithm competition stuff was
handy
it's because there are only about
seven to eight giga people in the world
yeah
that's not that many so if you can make
your algorithms linear time or almost
linear time
a computer operates in gigahertz yeah i
only need to do one run
one one recalculation every hour in
terms of telling people how far away
these dangers are
yes so i suddenly have 3600 seconds
and my cpu cores are running in
gigahertz and
at most they're eight giga people well
you're skipping over the fact that uh
there's n squared potential connections
between people
so how do you get around the fact that
uh
you know that we you know the potential
set of relationship any one of us could
have
8 billion so it's 8 billion times uh
squared
that's you that's potential amount of
data you have to be storing
and computing over and constantly
updating so the way we dealt with that
is we actually expect
that the typical network is very sparse
the
the technical term sparse would mean
that the average degree or the average
number of connections that a person has
is going to be at most like 100 strong
connections that you care about
if you if you think of it almost in
terms of the heavy hitters actually
in most people's lives 100 if we just
kept track of their top 100
interactions that's probably most of the
signal
yeah yeah i i i'm saddened to think
that i might not be even in a double
digits but oh
i i was intentionally giving a crazy
number to account for college students
you call oh those are the who you're
calling the heavy hitters the people who
are like the social butterflies
yeah yeah yeah i need to uh um i'd love
to know that information about myself by
the way the
that i do do you uh expose the graph
like how many like about yourself how
many connections you have
we do expose to each person how many
direct connections they have that's
great but for privacy purposes
we don't tell anybody who their
connections like how their connections
are interconnected yes gotcha
but at the same time we do expose also
to everyone an interesting chart that
says
here's how many people you have that
you're connected to directly
here's how many at distance two meaning
via
people and then here's how many at
distance three and the reason we do that
is that actually ends up being a dynamic
that also boosts adoption it drives
another feedback loop
the reason is because we saw actually
when we deployed this in some
universities
that when people see on their app that
they are
indirectly connected to hundreds or
thousands of other people
they get excited and they tell other
people hey let's download this app yeah
but you know we also saw in those
examples especially looking at the
screenshots people gave
that is hit as soon as the typical
person
has two or three other direct
connections on the system
because that means that our app has
reached a virality
are not of two to three the key is we
were making a viral app to fight a virus
spreading on the same network that the
virus spreads out
so you're trying to out virus the virus
that's right
that's exactly right okay great what
have you learned
from this whole experience in terms of
um
let's say for covid but for future
pandemics as well
is it possible to use the power
information here of networked
information
as the virus spreads and travels in
order to basically keep the society
open is it possible for for people to
protect themselves with this information
or do you still have to have most like
in this overarching policy of everybody
should stay at home
all that kind of thing we are trying to
answer that question right now
so the answer is we don't know yet but
that's actually why we're very happy
that now the idea has started to become
become more widely known
and we're already starting to
collaborate with epidemiologists
again i'm just a mathematician right and
a mathematician should not be the person
who is telling everybody
this will definitely work but because of
the potential power
of this approach especially the
potential power of this being an
end game for kovid we have gotten the
interest
of real researchers and we're now
working together
to try to actually understand the answer
to that question because you see
there's a theory so what i can share is
the mathematics of
here's why there's some hope that this
would work yeah and that's because
i'm talking about end game now end game
means you have very few cases
but everywhere we're always thinking
once there's few cases then does that
mean we now open up
once you open up in the past then the
cases go up again
until you have to lock down again yeah
and now when we talk about the dynamic
process that makes
it's guaranteeing you always have cases
until you have the great vaccines which
is
you know we both got vaccinated this is
this is good but at the same time why
i'm thinking this is still important is
because we know that
many vaccine makers have said they're
preparing for the next dose next year
and if we have a perpetual thing where
you just always need a new vaccine every
year
it could actually be beneficial to make
sure we have as many other techniques as
possible
for parts of the world that can't afford
for example
that kind of distribution yeah so
actually no matter no matter how deadly
the virus is
no matter how many things whether you
have a vaccine or not
it's still useful to be having this
information yes because
to stay home or not depending on how
risky like
i'm a big fan just like you said of uh
having the freedom
for you to decide how risk-averse you
want to be right
and depending on your own conditions but
also on the state of like what you
just how dangerously you like to live so
i think that actually makes a lot of
sense and i also think that
um since we're when we when you think of
disease spreading
it spreads in aggregate in the sense
that
uh if there are some people who maybe
are more
risk tolerant because of other things in
their life well there might also be
other people who are
less risk tolerance and then those
people decide to
isolate but what matters is in the
aggregate that this are naught of the
infection
uh spreading drops below one and so the
key is if you if you can empower people
with that power to make that decision
you might actually still be able to
drive that are not down below one
yeah and also this is me talking i
yes people get a little bit nervous i
think
with uh information somehow mapping to
privacy violation but
i i first of all in the approach you're
describing
that's respecting anonymity
but i would love to have information
from the very beginning
from march and april of last year
almost like a map of like where
it's risky and where it's not to go
and not map based on sort of the exact
location of people but where people
usually hang out kind of thing
just maybe not necessarily about
actual location but just maybe
activities
like just to have information about what
is
what is good to do or not you know uh in
terms of like safety
is it okay to run outside and not is it
okay to go to a restaurant or not
i just feel like we're operating the
blind and then what you had
is a very imperfect signal which is like
basically politicians
desperately trying to make statements
about what is safe and not they don't
know what the heck they're doing
they have a bunch of smart scientists
telling them stuff and the scientists
themselves
also very important don't always know
what they're doing
epidemiology is not
is as much an art as a science you're
desperately trying to predict the future
which nobody can do
and then you're trying to speak with
some level of authority
i mean if i were to criticize scientists
they spoke with too much authority
it's okay to say i'm not sure but then
they think like if i say i'm not sure
then there's going to be a distrust what
they realize is when you're wrong and
you say i'm sure
it's going to lead to more distrust so
there's this imperfect like
just chaotic messy system of people
trying to figure out with very little
information
and what you're proposing is just a huge
amount of information
and information is power is there um
challenges with adoption that you see in
the future
here so there's uh maybe we could speak
to there's approaches i guess from
google
there's different people that have tried
similar kind of ideas
not in you have a quite a novel idea
actually
but speaking the umbrella idea of
contact tracing
is is there is there something you can
comment about
why their approaches haven't been fully
adopted
is there challenges there is there is
there reasons why novid might be a
better idea moving forward
in general just about adoption yeah so
first of all i want to say i always have
respect for the
methods that other people use and so
it's good to see that other people have
been trying
but what we have noticed is that the
difference between our
value proposition to the user and the
value proposition to the user delivered
by everything that was made before
is that unfortunately the action of
installing
a standard contact tracing app will then
tell you
after you have already been exposed to
the disease
so that you can protect other people
from you and what that does to your
own direct probability of getting sick
if you think about it
suppose you were making the decision
should i or should i not install one of
those apps
yeah what does that do to your own
probability of getting sick
it's close to zero this is uh the sad
thing you're
um you're speaking to not sad i suppose
it's the way the world is
the only incentive there is to just help
other people i suppose but
a much stronger incentive is is anything
that
allows you to help yourself yes so what
i'm saying is that
uh let's just say free market capitalism
was not
based on altruism i think it's based on
if you make a system of incentives so
that everybody trying to maximize their
own situation
somehow contributes to the whole that's
a game's theoretic solution to a very
hard problem
and so this is actually basically
mechanism design we've basically come up
with a different mechanism
different set of incentives which
incentivizes the adoption because
actually whenever we've been rolling it
out
usually the first question we ask people
like say in the university is
do you know what nova does and most of
them have read about the other apps and
they say oh no of it will tell you after
you've been around someone so you can
quarantine
and we have to explain to them actually
novad never wants to ask you to
quarantine
yeah that's not the principle our
principle isn't based on that at all
we just want to let you know if
something is coming close
so that you can protect yourself if you
want yeah if you want if you want
and then the the quarantine is like yes
in that case if you're quarantining
it's because you're shutting the door
from the inside if that's exciting yes
exactly exactly i mean this is brilliant
but so
what um do you think the future looks
like for future pandemics what's your
plan
with novid what's your plan with these
set of ideas i
am actually still an academic and a
researcher so the biggest work i'm
working on right now
is to try to build as many
collaborations with other public health
researchers
at other universities to actually work
on
pilot deployments together in various
places that's the goal
that's actually ongoing work right now
and so for example if anyone's watching
this and you happen to be a public
health researcher
and you want to be involved in something
like this i'm just going to say i'm
still incentive thinking
there's something in it for the
researchers too this could open up an
entire new way of controlling disease
that's my hope
i mean it might actually be true and
people who are involved
in figuring out how to make this work
well it could actually be good for their
careers too
i i always have to think like if a
researcher was getting involved what are
they getting out of it
oh so you mean like uh from a research
perspective you can um
like publications and sets of ideas
about how to from a sort of uh
uh network theory perspective understand
how we control the spread of a pandemic
yes and
what i'm doing right now is this is
basically interdisciplinary research
where maybe our side is bringing the
technology and the network theory
and the missing parts are epidemiology
and public health expertise
and if the two things start to join also
because everywhere that you deploy
let's just say that the world is
different in the philippines as it is in
the united states
and just the natures of the of the
locality would mean that
someone like me should not be trying to
figure out how to do that but if we can
work with the researchers who are based
there
now suddenly we might come up with a
solution that will help scale in parts
of the world where they aren't all
getting the modern and fizer vaccines
which cost
like 20 a pop in the u.s so if they want
to participate
who do they reach out to oh that would
just be us i mean the novid.org website
has
nova.org it has it has a feedback reach
out form
and actually we are i mean again this is
the dna of being a researcher
i am actually very excited by the idea
that this could
contribute knowledge that will outlast
all of our generations like
all of our lifetimes there you go reach
out
to novanova.org uh what about individual
people
should they install the app and try it
out or is this really geographically
restricted
oh yeah i didn't come on here to tell
everyone to install the app i did not
come to tell everyone to install the app
because it works best
if your local health authority is
working with us gotcha there's a reason
it's because this is back to the game
theory
if anyone could just say i'm positive
the high school senior prank would be to
say that
we have a massive outbreak on finals
week let's not have final exams
so the way that our system works it
actually borrows some ideas no borrowers
we came up with them independently but
this idea is similar to what
google and apple do which is that if the
local health authority is working with
this
they can for everyone who's positive
gives them a passcode that expires in a
short time
so for ours if you're on the app and
saying i'm positive you can either just
say that
and that's called unverified or you can
enter in one of these codes that you got
from the local health authority
so basically for anyone who's watching
this it's not that you should just go
and download it unless you want to go
and look at it that's cool
but if you on the other hand if you
happen to know anyone at the local
health authority which
is trying to figure out how to handle
kovid well then
i mean we'd be very happy to also work
with you
guys so the the verified there is really
important because you're
you're maintaining anonymity and because
of that you have to have some source of
verification
in order to make sure that it's not uh
possible to manipulate
because uh it's it's ultimately about
trust and information and so it could be
um verification is really important
there so basically individual people
should um
ask their local health authorities to to
just to sign up to contact you
i hope this spreads i hope this spreads
uh for future pandemics because i'm
really
i'm it's the amount the millions of
people
who are hurt by this i think our
response to the virus
economically speaking the number of
people who lost their dream
lost their jobs but also lost their
dream entrepreneurs
you know jobs often give meaning there's
people who financially and
psychologically are suffering because of
our
i'll say incompetent response to the
virus
across the world but certainly united
states that should be the beacon
of uh entrepreneurial hope for the world
so
i i hope that uh
we'll be able to respond to these kinds
of events much better in the future and
this is exactly the right kind of idea
and now is the time to do the investment
let's step back to the beauty of of uh
mathematics
maybe ask the the big silly question
first which is uh
what do you find beautiful about
mathematics
i think that being able to look at a
complicated problem which looks
unsolvable
and then to be able to change the
perspective to come from a different
angle
and suddenly see that there's a nice
solution
i don't mean that every problem in math
is supposed to be this way but i think
that these
reframings and changing of perspectives
that cause difficult things to get
simplified and crystallize
and factored in certain ways is
beautiful actually that's
related to what we were just talking
about with even this fighting pandemics
the the crystal idea was just
quantify proximity by the number of
relationships in the phys
in the physical network instead of just
by the feet and meters
right it's it's just if you change that
perspective now all of these things
follow
and so mathematics to me is beautiful
in the pure sense just for that yeah
it's quite interesting to see a human
civilization as a network as a graph
and our relationships as kind of
edges in that graph and to then
do outside of just pandemic do
interesting inferences
based on that this this is true for like
twitter social networks and so on
how we expand the kind of things we talk
about
think about sort of politically if you
have this little bubble
quote-unquote of ideas that you play
with
it's uh nice from a recommender system
perspective how do you jump out of those
bubbles
it's really fascinating uh youtube was
working on that
twitter's working on that but not always
so successfully but
there's a lot of interesting work from a
mathematical
and a psychological sociological
perspective there in
within those graphs but if we look at
the cleanest formulation of that
of looking at a problem from different
perspective
you're also involved with the
international mathematics olympiad
which takes a small clean
uh problems that are really hard but
once you look at them differently
can become easy but that little jump of
innovation is is uh is the entire trick
so maybe at the high level can you say
what is the international mathematical
olympiad
sure so this is uh the competition
for people who aren't yet in college uh
math competition which is
the most prestigious one in the entire
world it's the olympics of mathematics
but only for people who aren't yet in
college now the kinds of questions that
they ask you to do
are not computational usually you're not
supposed to find that the answer is 42
right instead you're supposed to explain
why something is true
yes and the the problem is that at the
beginning when you look at each of the
questions
first of all you have four and a half
hours to solve three questions
and this is one day and then you have a
second day which is four and a half
hours
three questions but when you look at the
questions they're all asking you explain
why the following thing is true which
you've never seen before
and by the way even though there are six
questions if you solve any one of them
you're a genius and you get an honorable
mention so this is
this is hard to explain a hard problem
what about is it one person is it a team
ah so it's each country can send six
people
and the score of the country is actually
unofficial there's not an official
country
versus country system although everyone
just adds up the point scores of the six
people
and they say well now which country uh
stacked up where
yeah so maybe as a side comment i should
say that um
there's a bunch of countries including
the former soviet union and russia
uh where i grew up where this is one of
the
most important competitions that the
country participates in like
it was a source of pride for a lot of
the country
you look at the olympic sports like uh
wrestling weightlifting there's certain
sports
and hockey that russia
and the soviet union truly took pride in
and actually the
mathematical olympiad it was one of them
for many
years it's still one of them and that's
kind of fascinating we don't
think about it this way in the united
states maybe you can correct me if i'm
wrong but
it's not nearly as popular in the united
states in terms of its integration into
the culture
into uh just basic conversation into the
pride
like you know if you win an olympic gold
medal or if you win the super bowl you
can walk around proud
i think that was the case with the
mathematical olympiad in russia
not not as much the case in the united
states i think
so i just want to give that a little
aside because beating anybody
from russia from the eastern republic or
from china is very
very difficult that like if i remember
correctly
you know there's people this was a
multi-year
training process they train hard
and this is this is everything that
they're focused on
my uh my dad was was uh was a
participant in this and
it's i mean it's uh it's as serious as
olympic sports you think about like
gymnastics like young athletes
participating gymnastics
is this as serious as that if not more
serious so i just want to give that a
little bit of context because we're
talking about serious high level math
athletics almost here yeah and actually
i also think that it
made sense from the soviet union's
perspective because
if you look at what these people do
eventually
even though let's look at the ussrs or
the
international math olympiad record even
though they i say even though they won
a lot of awards at the high school thing
many of them went on to do incredible
things
in research mathematics or research
other things and
that's showing the generalization
generalizability
of what they were working on because
ultimately
we're just playing with ideas of how to
prove things
and if you get pretty good at inventing
creative ways to
turn problems apart split them apart
observe
neat ways to turn messy things into
simple crystals
well if you're going to try to solve any
real problem in the real world
that could be a really handy tool too so
i don't think it was a bad investment
i i think it clearly worked well for
soviet union
yeah so this this is interesting people
sometimes ask me you know you go up and
and under communism you know was there
anything good about communism
and it's difficult for me to talk about
it because it's not
uh communism is one of those things
that's looked down on like without
in absolutist terms currently but you
could still
in my perspective talk about the actual
forget communism or whatever the actual
term is but you know certain
ways that the society function that we
can learn lessons from
and one of the things in the soviet
union that was highly prized
is knowledge not even now it's
wisdom and the skill of invention
of innovation at a young age
so we're not talking about a selection
process where you pick the best students
in the school to do the mathematics
or to read literature it's like
everybody
did it everybody it was almost treated
as if
anyone could be the next einstein
anybody could be the next
i don't know hemingway james joyce and
so you're
forcing an education on the populace
and a rigorous deep education like
as opposed to kind of like oh we want to
make sure we
um we teach to the weaker student in the
class
which american systems can sometimes do
because we don't want to leave anyone
behind
the the russian system was
anyone can be the strongest student and
we're going to teach you the strongest
student
and we're going to just to pretend or
force
everybody even the weakest student to be
strong and what that results in
it's obviously this is what people talk
about is a huge amount of pressure
like it's psychologically very difficult
this is why people struggle when they go
to mit this very competitive environment
it can be very psychologically difficult
but at the same time it's bringing out
the best out of people
and that mathematics was certainly one
of those things
and exactly what you're saying which
kind of clicked with me just now as
opposed to kind of a spelling bee
in the united states which i guess you
spell
i'm horrible at this but it's a
competition about spelling which
i'm not sure but you could argue doesn't
generalize well to future skills
mathematics especially this kind of
mathematics
is essentially formalized competition of
invention
of uh of creating
new ideas and that generalizes really
really well so
that's that's quite brilliantly put i
didn't really think about that so this
is not just about the competition this
is about
developing minds that uh
will come come to do some incredible
stuff in the future
yeah actually i want to respond to a
couple of things there the first one
this one
um which is this notion of whether or
not that is possible in a
non-authoritarian regime
i think it is and that's actually why i
spent some of my efforts
before the covet thing actually trying
to work towards there the reason is
because
if you think about it let's say in
america lots of people are pretty
serious about training very hard for
football
or baseball or basketball basketball
it's very very accessible but lots of
people are doing that
why well actually i think that what this
what
what was going on with the authoritarian
thing was at least
the message that was universally sent
was
being a good thinker and a creator of
ideas
is a good thing yes exactly
there's no reason why that message can't
be sent that's right everywhere
and i think it actually should be so
that's the first thing the second thing
is what you commented about
this thing about um you know the
generalizable skill and what what could
people do with olympiads afterwards so
that's actually my interest in the whole
thing
uh i don't i mean i don't
just coach students how to do problems
in fact i'm not even the best person for
that i'm not the best at solving these
problems
there are other people who are much
better at making problems and teaching
people how to solve problems
in fact when i was when the mathematical
association of america which is the
group which is in charge of
the u.s participation in these olympiads
when they were deciding whether or not
to put me in
back in 2013 as the head coach i had a
conversation with their executive
director
where i commented that we might do worse
because my position was i don't i mean i
actually didn't
want to focus on winning i said if
you're going to let me work with
60 very strong minds as picked through
this
system because the coach works with
these gets to run a camp for these
students
i said i'm actually not going to define
my success in terms of winning
this contest i said i wanted to maximize
the number of the students
that i read about in the new york times
in 20 years
oh yeah and the executive director of
the mathematical association of america
was fully in support of this because
that's also how their philosophy is
so in america the way we run this is
we're actually not
just training to win even though the
students are very good and they can win
anyway
one reason for example i went and even
did the covid thing involving quite a
few of them
is so that hopefully some of them get
ideas because in 20 30 years i won't
have the energy or the insight to solve
problems
we'll have another catastrophe and
hopefully some of these people will step
up and do it
and ultimately have that long-term
impact i wonder if this is scalable to
because that's such a great metric for
education
not how to get an a
on the test uh
but how to have how to be on the cover
of new york times
for inventing something new and i
do you think that's generalizable to
education beyond just this particular
olympia like
it's even you're saying this feels like
a rare statement
almost like a radical statement as a
goal for education
so actually the way i teach my classes
at carnegie mellon which i will admit
right away is not
equivalent to the average in the world
but it's already not
it's already not just the top 60 in the
country as picked by something
um let me just explain i have exams in
my class which are 90
of the grade so the exams are the whole
thing or most of the whole thing and
the way that i let students prepare for
the exams is i show them
all the problems i've ever given on the
previous exams and the whole the exam
that they will take is open notes they
can take all the notes they want on the
previous problems
and the guarantee is that the exam
problems this time will have no overlap
with anything you've seen me give in the
past
as well as no overlap with anything i
taught in the class
so the entire exam is invention
wow but that's how i go right my point
is
i have explained to people when i teach
you i don't want you to have remembered
a method i showed you
i want you to have learned enough about
this area that if you face a new
question which i came up with the night
before
by thinking about like what could i ask
that i have never asked before
oh that's cute i wonder what the answer
is ah that's an exam problem that's
exactly what i do before the exam
and then that's what i want them to
learn yeah and the first exam usually
people have a rough time because it's
like what what kind of crazy class is
this the professor doesn't
teach you anything for the exam but then
by the second or third and by the time
they finish the class
they have learned how to solve anything
in the area how to invent how they
intend in that area yeah
can we uh walk back to the the
mathematical olympia yes what's the
scoring and format like
and also what does it take to win so the
way it works is that
each of the six students
do the problems and there are six
problems all the problems are equally
weighted so each one's worth seven
points that means that
your maximum score is six problems times
seven points which is
the nice number of 42. and now
the way that they're scored by the way
is there's partial credit so the
question is asking you
explain why this weird fact is true
okay if you explain by you get seven
points if you make minor mistake maybe
you get six points
but if you don't succeed in explaining
why but you explain
some other true fact
which is along the way of proving it
then you get partial credit and and
actually now this is tricky because
how do you score such a thing it's not
like it was the answer was
72 and you wrote 71 and it's close right
the answer is 72
and you wrote 36 oh but that's pretty
close because you were you know you
that maybe you're just off by it by the
way they're not numerical anyway but i'm
just giving some
numerical analog to the way the scoring
might work
they're all essays and that's where i
guess i have
some role as well as some other people
who helped me in the u.s delegation for
coaches
we actually debate with the
country which is organizing it the
country which is organizing the olympiad
brings about
50 people to help judge the written
solutions
and you you schedule these half hour
appointments
where the delegation from one country
sits down at the table like this
opposite side is two or three people
from the host country
and they're just looking over these exam
papers saying
well how many points is this worth based
on some rubric that has been designed
and this is a negotiation process where
we're not
we're not trying to bargain and get the
best score we can in fact sometimes we
go to this table and we will say
we think we want less than what you gave
us this is how this is how our
these are our principles if you give us
too much we say no you gave us too much
we do that however the reason why this
is an interesting process
is because if you can imagine every
country which is participating has its
own language
and so if you're trying to grade the
mongolian scripts and they're written in
mongolia
if you don't read mongolian which most
people don't then the
the coaches are explaining to you this
is what the student has written
it's actually quite interesting process
it's almost like um
like a jury yeah you have you have uh
in the american legal system you have a
jury that where they're deliberating
but unlike a jury there's the members of
the jury
speak in different languages sometimes
that's fascinating
but uh i mean it's hard to know what to
do
because it's probably really really
competitive
but your sense is that ultimately people
like how do you prevent manipulation
here
right well we just hope that it's not
happening
so so we write in english therefore
everything that the u.s does
everyone can look at so it's a it's very
hard for me it's very hard for you to
manipulate
we don't manipulate we only hope that
other people aren't
but at the same time as you see our
philosophy was we want to use this as a
way to develop general talent
and although we do this for the six
people who go to the international math
olympiad
we really want that everyone at any
touch at any stage of this process
get some skills that can help to
contribute more later so
i don't know if you can uh say something
insightful to this
question but what do you think makes a
really hard math problem
on on this olympiad maybe in the courses
you teach
or in general what makes for a hard
problem you've seen i'm sure a lot of
really difficult problems
what makes a heart problem so i could
quantify it by the number of leaps of
insight
of changes of perspective that are along
the way and here's why
it's this is like a very theoretical
computer science way of looking at it
okay it's the is that each reframing of
the problem and using of some tool i
actually call that a leap of insight
when you say oh wow
now i see i should kind of put these
plugs into those sockets
like so yes and suddenly i get to use
that machine oh but i'm not done yet
now i need to do it again each such step
is a large
possible large fan out in the search
space the number of these
tells you the exponent the base of the
exponent is like how big
how many different possibilities you
could try and that's
that's actually why like if you have a
three inside problem
that is not three times as hard as a one
inside problem
because after you've made the one inside
it's not clear that that was the right
track necessarily well unless there's
still a branching
uh of possible yeah
[Laughter]
you're saying there's problems like on
the math olympia that requires more than
one insight yes
those are the hard ones and also i can
tell you how how you can tell
so this is how i also taught myself math
when i was in college
so in if you are taking uh not taught
myself i was taking classes of course
but i was trying to read the textbook
and i found out i was very bad at
reading math textbooks
a math textbook has a long page of stuff
that is all true
which after you read the page you have
no idea what you just read
yeah this is just a good summary of a
math textbook
okay yeah because it's it's not clear
why anything was done that way and yes
everything is true but how the heck did
anyone think of that
so the way that i taught myself math
eventually
was the way i read a math textbook is i
would
look at the theorem statement i would
look at the length
of the proof and then i would close the
book and attempt to reprove it myself
yeah now that's brilliant the length of
the proof
is telling you the number of insights
because the length of the proof
is linear in the number of insights that
each insight takes space
yeah and if i know that it's a short
proof i know that there's only one
insight
so when i'm doing my own way of solving
the problem like so i mean finding the
proof
i quit if i'm if i have to do too many
pluggings it's equivalent to a math
contest
in a math context i look is it problem
one two or three that tells me how many
insights there are
this is exactly what i did that's
brilliant linear in the number i don't
know
i i think um well it's possible that
that's true
approximately approximately
approximately yeah just i don't know uh
somebody out there is going to try to
formally prove this oh no i mean you're
right
there are cases where maybe it's not
quite linear but in general well some of
it's notation too and some of it is uh
style and all those kinds of things but
within a textbook within the same book
within
the same book yes within the same book
on the same subject
yeah that's what i was using that's
hilarious because you know if it's a
two-page proof you just know this is
going to be insane
right that's the uh that's the scary
thing about insights
uh you look like andrew wiles working on
the firm oz last theorem
is you don't know something seems like a
good idea
and you have that idea and it feels like
this is a leap
like a totally new way to see it but you
have no idea if it's
at all useful uh even if you think it's
correct you have no idea if this is like
going to go down a path that's
completely counterproductive
or not productive at all that's
that's a crappy thing about invention is
um
like i have i'm sure you do i have a lot
of really good ideas every single day
but like and then i'll i'll go inside my
head along them
along that little trajectory but it
could be just a total waste
and it's that i that you know what that
feels like it just feels like
patience is required not to get excited
any one thing
so i think this is interesting because
you raised andrew biles he spent
seven years attacking the same thing
yeah right
and and so i think that what attracts
professional researchers to this
is because even though it's very painful
that you keep fighting with something
when you finally find the right insights
and string them together
it feels really good so well there's
also like
short term it feels good to
to uh whether it's real or not
to pretend like you've solved something
in the sense like you have an insight
and there's a sense like this might be
the insight that solves it
so at least for me i just enjoy
that rush of positivity even though i
know statistically speaking is probably
going to be a dead end
i'm the same way i'm the same way in
fact that's how i know whether
i might want to keep thinking about this
general problem it's like if i still see
that i'm getting
some insights i'm not at a dead end yet
but that's also where i learned
something from my phd advisor actually
he was a
real big inspiration on my life his name
is benny sudokov in fact he grew up in
the former soviet union
he was from georgia but he he's an
incredible person
but one thing i learned was choose the
problems to work on
that might matter if you sub if you
succeed
because that's why for example we dug
into cope it was just well
suppose we succeed in finding some
interesting insight here
well it actually matters then it's
worthwhile
yeah and i think covid the the way
you're approaching
uh kovid has two interesting
possibilities
one it might help with the covet or
another pandemic
but two i mean just this whole
network theory space you might unlock
some deep understanding about the
interaction with human beings that might
have
nothing to do with the pandemic there's
a there's a space of possible impacts
that may be direct or
indirect and the same thing is like with
andrew wiles's proof
i don't understand but apparently
the pieces of it are really impactful
for
for mathematics even if the main theorem
is not
so along the way the the insights you
have
might be really powerful for
unexpected reasons so i like what you
said this is something that i learned
from another
friend of mine who's also he's a very
famous researcher all these people are
more famous than i am
his name is jacob fox he's jacob fox at
stanford also a very big inspiration for
me we were both grad students together
at the same time well most importantly
you're good at selecting good friends
ah yeah well that's that's that's that's
the key you've got to find good people
to learn things from yeah
but his thing was he often said you know
if you solve a math problem and have
this math proof
math problem for him is like a proof
right so suppose you came up with this
proof
he always asks what have we learned from
this that we could potentially use for
something else
it's not just did you solve the problem
that was supposed to be famous
it was and is there something new in the
course of solving this
that you had to invent that we could now
use as a tool elsewhere
you know there's this funny effect where
uh just looking at different fields
where people discover parallels
they'll prove something it'll be a
totally new result and then somebody
later realizes this was already done 30
years ago and another discipline
in another way and it's really
interesting
uh we did this offline in another
illustration you showed to me
it's interesting to see the different
perspectives
on a problem it kind of points like
there's just like very few novel ideas
that everything else that most of us are
just looking at different perspective on
the same idea
and it makes you wonder this this uh
this old silly question
that i have to ask you is uh do you
think
mathematics is uh discovered or invented
do you think we're creating
a new idea we're building a set of
knowledge
that's that's distinct from reality
or are we actually like it is math
almost like a shovel where we're digging
to
like this core set of truths that are oh
that
that were always there all along so i
personally feel like it's discovered
but that's also because i guess the way
that i'd like to choose what questions
to work on
are questions that maybe we'll get to
learn something about
why is this hard i mean i'm often
attracted to questions that
look simple but are hard right and what
could you possibly learn from that sort
of like probably the attraction of
fermat's last theorem
as you mentioned simple statement why is
it so hard so i'm more on the
discovered side and i also feel like if
we ever ran into an intelligent
other species in the universe
probably if we compared notes there
might be some similarities between
both of us realizing that pi is
important because
you might say why why humans do humans
like circles more than others i think
stars also like circles i think planets
like circles they're not perfect circles
but nevertheless the concept of a circle
is just point and
constant distance it doesn't get any
simpler than that it's possible that
that like an alien species will have
depending on different cognitive
capabilities and
different perception systems we'll be
able to see
things that are much different than
circles and so
if it's discovered it would still be
pointing at a lot of same geometrical
concepts
mathematical concepts but
it's interesting to think of how many
things we would have to still
align not just based on notation but
based on
understanding like just the like um
some basic mathematical concepts like
how much work is
there going to be in trying to find a
common language
i mean this is um i think stephen
wolfram and his son help with the movie
arrival
like the developing an alien language
like how would aliens communicate with
humans
it's fascinating because like math seems
to be the most promising thing but even
like math
like how do you visualize
mathematical ideas it feels like there
has to be an
interactive component just like we have
a conversation
there has to be this is something we
don't i think think about often which is
like
with somebody who doesn't know anything
about math doesn't know anything about
english or any other natural language
how would we describe we talked offline
about visual proofs
how would we through visual proofs have
a conversation
where we say something here's the
concept the way we see it
does that make sense to you and like
can you mess with that concept to make
it sense for you
and then go back and forth in this kind
of way so purely through mathematics i'm
sure it's possible to have those kind of
experiments with like tribes
on earth that don't there's no common
language through math
like draw a circle and see what they do
with it
right do some of these visual proofs uh
like the summation of the odds and the
the adds up to the squares
yes i wonder how difficult that is uh
before one one or the other species
murders
i i hope that the curiosity for
knowledge will overpower the greedy
this is back to our game theory thing
that the curiosity of like
discovering math together will overpower
the desire for resources
and uh ultimately like you know
willing to commit violence in order to
gain those resources
i think as we progress become more and
more intelligence as a species i'm
hoping
we would value more and more the
knowledge because we'll come up with
clever ways to gain more resources so we
won't be so resource starved
i don't know that's a hopeful message
from when we finally meet aliens
yeah yeah see the cool thing about
the math olympiad um i don't know if you
know
work from francois chole
from google he come up he came up with
this kind of
iq test slash it kind of has
similar aspects to it that also math
olympia does
for for ai so he came up with these
tests
where they're very simple for humans
but very difficult for ai to illustrate
exactly why we're just not good at
seeing a totally new problem
we sorry ai systems are not good at
looking at a new problem that requires
you to
detect that there's a symmetry of some
kind or
there's a pattern that's that hasn't it
hasn't seen before
the pattern is like obvious to us humans
but it's not so obvious to find that
kind of
it's you're inventing a pattern that's
there
in order to then find a solution
some i don't know if you can comment on
but from an
ai perspective and from a math problem
perspective
what do you think is intelligence what
do you think is the thing
that allows us to solve that problem and
how hard is it to build a machine
to do that asking for a friend yeah so i
guess
you see because if i just think of the
raw search space it's huge yeah
that's why you can't do it and if i
think about what makes somebody good at
doing these things they have this
heuristic sense it's almost like a good
chess player of saying
let's not keep analyzing down this way
because there's some heuristic reason
why that's a bad way to go yes
where did they get that heuristic from
now that's a good question i don't know
because that if you asked them to
explain to you
they could probably say something in
words that sounds like it makes sense
yeah but i'm guessing that's only a part
of what's really going
on in their brain of evaluating that
position you know what i mean if you ask
gary kasparov what is good or
why is this position good he will say
something yeah but it's probably not
approximating everything that's going on
inside
so there's basically a function being
computed yeah but it's hard to
articulate what that function is
now the question is could a computer get
as good at computing these kinds of
heuristic functions
maybe i i'm not enough of an expert to
understand
but one bit of me has always been a
little bit curious of whether or not the
human brain
has a particular tendency due to its
wiring to come up with certain kinds of
things
which is just natural due to the way
that the
topology of the neurons and whatever is
there for which if you tried to just
build from scratch a computer to do it
would it naturally have different
tendencies
i don't know and this is just me being
completely ignorant and just saying a
few ideas
well this is a good thing with that
mathematics shows is we don't have to be
so math and physics or mathematical
physics
operates in a world that's different
than our
descendant of a brains operate in
so it allows us to to have multiple
many many dimensions it allows us to
work on on weird surfaces
uh with like topology as a discipline
it's just weird to me
it's really complicated but it allows us
to work in that space
the differential geometry and all those
kinds of things where
it's totally outside of our natural
day-to-day
four-dimensional experience uh 3d
dimensional with time
experience so math gives us gives me
hope
that we can that
we can see we can discover the processes
of intelligence
uh outside the limited nature of our own
like human experiences
but you said that you're not an expert
it's kind of funny i i i find that um
we know so little about intelligence
that uh i think i honestly think like
almost
children are more expert at
creating artificial intelligence systems
uh
than than adults i feel like we know so
little we really need to think outside
the box
and those little i found people should
uh check out francois charley's little
exams
but even just solving math problems
i don't know if you've ever done this
for yourself but
when you solve a math problem you kind
of then
traced back and tried to figure out
where did that idea come from
like what how did like what was i
visualizing in my head
how did i start visualizing it that way
how why did i start rotating that cube
in my head in that way
like what is that if i were to try to
build a program that does that
where that come from so this is
interesting um
so i try to do this to teach middle
school students
how to learn how to create create and
think and invent and the way i do it
is there are these math competition
problems and i'm working in
collaboration with the people who run
those
and i will turn on my youtube live and
for the first time look at those
questions
and live solve them the reason i do this
is to let the middle school students and
the high school students and the adults
whoever wants to watch
just see what exactly goes on through
someone's head
as they go and attempt to invent what
they need to do to solve the question
so i've actually thought about that i
think that
first of all as a teacher i think about
that because whenever i want to explain
to a student how to do something
i want to explain how it made sense why
it's intuitive to do the following
things
and why the wrong things are wrong not
just by this one short fast way
well why this is the right way if that
makes sense so my point is i'm actually
always thinking about that
like how would you think about these
things and then i eventually decided the
easiest way to expose this
would just be to go live on youtube and
just say i've never seen any of these
questions before here we go
don't you get uh man that's that's
anxiety inducing for me
uh don't you get trapped in
a kind of like little dead ends of
confusion
even on middle school problems yes
that's what the comments are for
the live contents come in and students
say try this oh wow
it's actually pretty good i'll never get
stuck i mean i'm i'm willing to go on
camera and say
guess what potion though can't do this
that's fine but then what ends up
happening
is you will then see how maybe
somebody's saying something and i look
at the chat and i say aha
that actually looks useful now that also
shows how
not all ideas not all suggestions are
the same power
if that makes sense because if i
actually do get stuck i'll go fishing
through the chat
[Laughter]
i don't know if you can speak to this
but is there a moment
for the middle school students maybe
high school as well
where there's like a turning point for
them
where they maybe fall in love with
mathematics
or they or they get it is there um
is there something to be said about like
discovering that moment and
and trying to grab them to get to get
them to
understand that mathematics is so no
matter what they want to do in life
could be part of their life
yes i actually do think that the middle
school is exactly the right time
because that's the place where your
mathematical understanding
gets just sophisticated enough that you
can start doing interesting things
because if you're early on in counting
i'm honestly not very good at teaching
you new insights
my wife is pretty good at that but
somehow once you get to this
this part where you know what a fraction
is and when you know
um how to add and how to multiply and
what the area of a triangle is
at that point to me the whole world
opens up and you can start observing
there are really nifty coincidences
the things that made the greek
mathematicians and the ancient
mathematicians excited
actually back then it was exciting to
discover the pythagorean theorem
it wasn't just homework so is there
what which discipline do you think has
the most exciting coincidences so
is it geometry is it algebra
um is it calculus well you see you're
asking me and i'm the guy who gets the
most excited
when the combinatorics shows up in the
geometry
is it okay so it's the combinatorics
in the geometry so first of all the nice
thing about geometry
this is the same nice thing about
computer vision is it's
visual so geometry you can draw circles
and triangles and stuff so
it naturally uh presents itself to uh
to the visual proof right but also the
nice thing about geometry i think
for me is the earliest
class the earliest discipline where
there's
uh that's most amenable to the
exploration
of the invention through proofs the idea
of proofs i think is most easily
shown in in geometry because it's so
visual i guess
so that that to me is like uh if i were
to think about when i first
fell in love with math it would be
geometry and sadly enough
that's not used geometry only has a
little appears briefly
in the journey of of an um of a student
and it kind of disappears and not until
much later which
you know to maybe like differential
geometry i don't know where else it
shows up for me in computer science like
you can start to think about like
computational geometry or even graph
theory is the kind of geometry you can
start to think about it visually
although it's pretty tricky but yeah it
was always um
that that was the most beautiful one
everything else i guess calculus can be
kind of visual too that can that could
be pretty beautiful
but is there um something
you try to look for in the student
to see like how can i inspire them at
this moment
or is this like individual student to
student is there something you could say
there
so first of all i really think that
every student can pick up all of this
skill
i really do think so i don't think it's
something only for a few and so
if i'm looking for a student actually
oftentimes when i'm
if i'm looking at a particular student
the question is
how can we help you feel like you have
the power to invent also
because i think a lot of people are used
to thinking about math as something
where the teacher will show you what to
do
and then you will do it yes so i think
that the key is to
show that they have some let them see
that they have some power to invent
and at that point it's often starting by
trying to give a question that they
don't know how to do
you want to find these questions that
they don't know how to do that they
can think about and then they can solve
and then suddenly they say
my gosh i've had a situation i haven't
i've had an experience
where i didn't know what to do and after
a while i did
is there um advice you can give on how
to learn math for people whether it's
middle school whether it's somebody
as an adult kind of gave up on math
maybe early on
i actually think that these math
competition problems those school and
high school are really good
they're actually very hard so if you if
you haven't had this
kind of experience before and you grab a
middle school math competition problem
from the state level which is used to
decide who represents the state in the
country in the united states for example
those are pretty tricky and even if
you're a professional
maybe not doing mathematical things and
you're not a middle school student
you'll struggle
so i find that these things really do
teach you things but by trying to work
on these questions
is there a a googleable term that you
could use for the organization
for the state competitions ah yeah so
there are a number of different ones
that are quite popular
one of them is called math counts
m-a-t-h-c-o-u-n-t-s
and that's a big tournament which
actually has a state level there's also
a math league.org
mathleaguelegue.org also has
this kind of tiered the tournament
structure there's also the american math
competitions amc8
amc also has amc 10 that's for 10th
grade and below and amc 12.
these are all run by the mathematical
association of america
and these are all ways to find old
questions what about the daily
challenges that you run
what are those about we do that too but
i mean the difference was um ours isn't
that one is not free so so this i should
actually probably be careful the things
that i've just mentioned are also not
free
not all of those things i mentioned just
now are free either but people can
figure out what it is
but yeah this is really nice to know
what's out there but can you speak a
little bit to the daily challenges
sure sure so that's actually what we did
when um
i guess i was thinking about how would i
try
to develop that skill in people if we
had the power to architect the entire
system ourselves
so that's called the daily challenge
with potion low it's not free because
that's actually how i pay for everything
else i do so that's that was the idea
but the the concept was aha now let's
invent from scratch
so if we're going to go from scratch and
we're going to use technology
what if we made every single lesson
something where first i say hey here's
an interesting question recorded of
course it's not live but it's like i say
hey here's an interesting question why
don't we think about this
but i know i know you don't know how to
do it so now you think
and a minute later a hint pops on the
screen but you still think and a minute
later a big hint pops on the screen
you still think and then finally after
the three minutes hopefully you got some
ideas you try to answer
and then suddenly there's like this
pretty extended explanation of
oh yeah so here's like multiple
different ways that you can do the
question
and by accident you also just learn this
other concept
that's what we did so is this targeted
towards middle school students
high school students it's targeted
towards middle school students with
competitions
but there's a lot of high school
students who didn't do competitions in
middle school
where they would also learn how to think
if you can see the whole concept was
can we teach people how to think how
would you do that you need to give
people the chance to
on their own invent without that kid in
the front row answering every question
in two seconds
and people can find it i think with
daily dot
but if you go to find my website you
you'll be able to find it
beautiful can we zoom out a little bit
and uh
so day to day week to week month to
month year to year what does
the lifelong educational process look
like
do you think for for yourself
but for for me what would you recommend
in in the world of mathematics or sort
of as opposed to studying for a test but
just like
lifelong expanding
of uh knowledge and that skill for
invention i think i often articulate
this as
can you always try to do more than you
could do in the past
yeah but that comes in many ways and
i will say it's great if one wants to
build that with mathematics
but it's also great to use that
philosophy with all other things
in fact if i if i just think of myself i
just think what do i know now that i
didn't know a year ago or a month ago or
a week ago
and not just know but like what do i
have the capability of doing yes
and if you just have that attitude it
brings more
see the thing is there's also a habit
like it is a skill like i've been using
anki it's an app for helps you memorize
things
and i've actually just a
few months ago started doing this daily
of uh
setting aside time to think about an
idea
that's outside of my work like let's say
let's pick
put it it's all over the place by the
way but let's say politics like gun
control
uh is it good to have a lot of guns or
not in society
and just i've set aside time every day
i do at least 10 minutes but i try to do
30 where i think about a problem and
kind of outlining for myself from
scratch from not looking anything up
just thinking about it
using common sense and i think the
practice of that is really important
it's the daily routine of it
it's the discipline of it it's not just
that
i figured something out from that
thinking about gun control
it's more that that muscle is built
too it's that thinking muscle so i'm
kind of interested in
you know math has because especially
because
i've gotten specialized into machine
learning and because i love programming
so much
i've lost touch with math
a little bit to where i feel
quite sad about it and i want to fix
that um
even just not math like pure knowledge
math but math like these middle school
problems
the challenges right um
is that something you see a person be
able to do every single day kind of just
practice
every single day for years so i can give
an answer to that that gives a practical
way you could do it assuming you have
kids
so i
i'm just saying this because i'm just
thinking out loud right now like what
could i do
what could i do to suggest because what
i have noticed is that for example if
you do have kids who are in elementary
school or middle school
if you yourself go and look at those
middle school math problems
to think about interesting ways that you
can teach your elementary school or
middle school kid
it works that's what my wife did she
never did any of those contests before
but now she knows quite a lot about that
i didn't teach her anything i don't i
don't do that
she just was messing around with them
and taught herself
all of that stuff and that had the
automatic daily i i'm always thinking
how do you make it practical right yes
and the way to make it practical
is if the the timer on the automatically
daily is that you are going to
automatically daily do something with
your own kid
yes now it feeds back okay and that
includes the whole lesson that if you
want to learn something you should teach
it
oh i strongly believe that yes i
strongly believe
and that so i currently don't have kids
so that's uh
maybe i should just get kids to help me
with the math thing but
outside of that i i do want to do great
math into daily practice so
i'll definitely take out uh i'll
definitely check out the daily
challenges and see
because um what is it uh grant sanderson
we talked about offline
three blue one brown he he speaks to
this
as well that his videos aren't
necessarily
they don't speak to the thing that i'm
referring to which is the daily practice
they're more almost tools of inspiration
they kind of show you the beauty of a
particular
of problem in mathematics
but they're not a daily ritual and i'm
i'm in
i'm in search of that daily ritual
mathematics it's not it's not trivial to
find
um but uh i hope
i hope to find that because i think math
gives you a perspective on the world
that enriches everything else so i like
what you said about the daily also
because that's also one reason why i put
my carnegie mellon class online it's not
every day it was every it's every other
day semester is almost over
but the idea was i guess my philosophy
was if i'm already doing the class let's
just like put it there right
but i do know that there are people who
have been following it
who are not in my class at all who have
just been following it because yes it's
combinatorics
and the value of that is you could you
don't really need to know calculus to
follow it if that makes sense
so it's actually something that people
could follow so again and that one's
free so
that was just there on youtube well
speaking of combinatorics
uh what is it what do you find
interesting what do you find beautiful
about combinatorics so combinatorics to
me
is is the study of things where
they might be more finite and more
discreet
what i mean is like if i look at a
network actually a lot of times the
combinatorics will boil down to
something and the combinatorics i think
about might be something related to
graphs or networks and they're very
discreet because if you have a node
it's not that you have point seven of a
node
and point three of a node over there
it's like you got one node and then you
jump one step to go to the next node
so that that notion is different from
say calculus which is very
uh continuous where you go and say i
have this
speed which is changing over time and
now what's the distance i've traveled
that's the notion of an integral
where you have to think of subdividing
time into very very small pieces
so the kinds of things that you do when
you reason about these
finite discrete structures often might
be iterative algorithmic inductive
these are ideas where i go from one step
to the next step and so on and make
progress
i guess i actually personally like all
kinds of math
my area of research just ended up in
here because
i met a really interesting phd advisor
potential that's
actually that's honestly the reason i
went into that direction i met a really
interesting guy
he seemed like he did good stuff
interesting stuff and he looked like he
cared about students
and i said let me just go and learn
whatever you do even though
my prior practice and preparation before
my phd was not combinatorics but
analysis
the continuous stuff the the annoying
thing about
combinatorics and discrete stuff
is uh it's often uh really difficult to
solve
uh from a uh sort of uh uh running time
complexity perspective
is there could you speak to the idea of
complexity analysis of problems do you
find it useful to find it interesting
do you do you find that lens of studying
the difficulty
of how difficult the computer science
problem is
a useful lens onto the world oh very
much so
because um if you want to make something
practical which has large numbers of
people using it
the computational complexity to me is
almost question one
and that's again that's at the origin of
when we started doing this stuff with
disease control
from the very beginning the deep
questions that were running through my
mind were
would we be able to support a large
population
with only one server and if the answer
is no
we can't start because i don't have
enough money
yeah and there the question is very much
you know linear time versus
anything uh
anything slower than linear time um it's
a very specific thing you have a bunch
of really interesting papers if i could
ask
maybe we could pull out some cool
insights at the high level
can you describe the data structure of a
voting tree and
what are some interesting results on it
you have a paper that i noticed on it
yeah so this is an example of i guess
um how in math we might say here's an
interesting kind of a question
that we just can't seem to understand
enough about
maybe there's something else going on
here and the way to describe this is
you could imagine trying to hold
elections where
if you have only two candidates that's
kind of easy you just run them against
each other and see who gets more votes
but as you know once you have more
candidates it's very difficult to decide
who wins the election
and there's an entire like voting theory
around this
so a a theoretical question became
what if you made a like a system of
runoffs like a system of heads uh
head-to-head contests
which you structure like a tree almost
looking like a circuit
i'm using that way of thinking because
it's sort of like a in
electrical engineering or computer
science you might imagine having a bunch
of
leads that carry signal which are going
through and gates and or gains and
whatnot and you've managed to compute
beautiful things
this is just from a purely abstract
point of view what if the inputs are
candidates
and for every two candidates it is known
which of the candidates is more popular
than the other
now can you build some kind of a circuit
board which says
first candidate number four will play
against five and see who wins
and so on okay so now what would be a
nice
outcome right this is a general question
of could i make a big circuit board to
feed an election into
like maybe one nice outcome would be
whoever wins at least
is preferred over a lot of people yes
so for example if you ran in 1024
candidates
ideally we would like a guarantee that
says that the winner
beats a lot of people actually in any
uh system where there are 1024
candidates there's always a candidate
who beats at least 512
of the others this is a mathematical
fact that there's actually always a
person who beats at least half
of the other people i'm trying to
make sense of that mathematical fact is
this supposed to be obvious
uh no but i can explain it no no i can't
the way it works is that think of it
this way
every time i i think imagine i have all
these candidates and everyone is
competing is everyone is like compared
with everyone else at some point
well think of it this way whenever
there's a comparison
somebody gets a point that's the one who
is better than the other one my claim is
there's somebody whose
score is at least half of how many other
people there are
yeah i'm just trying to like my
intuition is very close to that being
true but it's beautiful
i didn't at first that's not an obvious
fact
no it's not and it's it feels like a
beautiful fact well let me explain it
this way
imagine that for every uh match
you didn't give one point but you gave
two points
you gave one point to each person now
that's not what we're really doing we
really want to give one point to the
winner of the match
but instead we'll just give two if you
gave two points to everyone
on every matchup actually everyone has
the same number of points
and the number of points they get is how
many other people there are
does that sort of make sense i'm just
like saying no no everything everything
is same makes perfect
okay so the point is if for every
comparison between two people
which i'm doing for every two people i
gave one point to each person
your score everyone's score is the same
it's how many other people there are yes
now we only make one change for each
match up
you give one point only to the winner so
we're awarding half the points
so now the deal is if in the original
situation
everyone's score was equal which is how
many other people there are
now there's only half the number of
points to go around
so what ends up happening is that
there's always going to be like the
average number of points per person
is going to be half of how many other
people there are and somebody is going
to be above
somebody's going to be at least average
yeah this is this notion of expected
value
that if i have a random variable which
has an expected value
there's going to be some possibility in
the probability space
where you're at least as big as the
expected yeah when you describe it like
that it's obvious
but when you're first saying in this
little circuit that there's going to be
one
candidate better than uh than half
that's not obvious yeah that's funny
math this is nice uh okay so do you have
this but
ultimately you want you're trying to
with the voting tree
i don't know if you're trying this but
uh to have a circuit that's
that's like compressed that's small well
that achieves the the achieves the
the same kind of um i mean the smaller
it is
the if we look at practically speaking
the the lower the cost of running the
election
of running through of computing the
circuit that is true
but actually at this point the the
reason the question was interesting
yes is because the there was no good
guarantee
that the winner of that circuit would
have
like have beaten a lot of people let me
give an example the best known circuit
when we started thinking about this
was the circuit called candidate one
plays against candidate two
candidate three plays against four and
then the winners play against each other
and then by the way five plays against
six seven against eight
the winners play against each other you
understand it's like a giant binary tree
where yeah the binary like a balanced
binary tree
okay it's a balanced binary tree one two
three four up two thousand twenty four
everyone going up to find the winner
beautiful well you know what
there's a system in the world where it
could just be
that there's a candidate called number
one that just beats
like 10 other people just the 10 that
they need to be
on their way up and they lose to
everyone else
but somehow they would get all the way
up yes my point is
it is possible to outsmart
that circuit in one
weird way of the world which makes that
circuit a bad one because you want to
say i will use this circuit for all
elections
and you might have a system of inputs
that go in there
where the winner only beat 10 other
people which is the people they had to
beat on the way up
so you want to have a circuit where
there's as many like the final result
is as strong as possible yes
and so what what ideas do you have for
that
so um we actually only managed to
improve it to square root of n
so if n is the number of vertices n over
two would be the ideal
we got it to and we got the square root
of n versus
log base two yeah exactly yeah
which is well that is happening it could
be a lot
yeah it could be a big improvement so
that's uh okay cool
is there something you can say with
words about what
kind of circuit what that looks like i
can give an
idea of one of the tools inside yeah but
the actual execution ends up being more
complicated
but one of the widgets inside this is
building a system where you have like a
candidate
who plays like the one part of the whole
huge huge tree is that
that same candidate let's column seven
seven plays against
somebody let's let's pick up some
numbers let's call the others like
letters so seven plays against a
seven is also going to play against b
separately and the winners of each of
those will play each other
by the way seven's also going to play c
seven's going to play d
and the winners are going to play each
other and the winners are going to play
each other we call this
seven against all well seven against
like everyone from a bunch of
got it so there's some nice overlap
between the matchups
yeah that somehow has a nice feature to
it yes and i can tell you the nice
feature because if at the base of this
giant tree
at the base of this giant circuit like
this is a widget you rebuild the things
out of widgets so i'm just describing
one widget
but in the base of this widget you have
lots of things which are seven against
someone seven against someone seven
against someone in fact
every match up at the bottom is seven
against someone
what that means is
if 7 actually beat everyone they were
matched up against well 7 would rise to
the top
so one possibility is if you see a 7
emerge from the top
you know that 7 actually beat every one
they were against
on the other hand if anyone else is on
top let's call it
f if f is on top how did f get there
well f beats seven on the way at the
beginning
so the point is the outcome of this
circuit has a certain property
if you see a seven you know that the
seven actually be the brazilian people
if you see anyone else at least you know
they beat seven
yeah then you can prove that it has a
nice property that's really interesting
is there something you can say
perhaps going completely outside of what
we're talking about is
uh how we may um
have mathematical ideas of improving
the electoral process that one no no i
can't give you that one
i mean is there like do you ever see it
as um
as as there be do you see is there being
a lot of opportunities
for improving how we vote uh
like from your i i don't know if you saw
parallels but
you know it seems like if this actually
kind of maps to your
sort of covid work which is there's a
network effect
right it seems like we should be able to
apply similar kind of effects
of how we decide other things in our
lives
and and one of the big decisions we make
is who represents us in government
do you ever think about like
mathematically about those kinds of
systems
i think a little bit about those because
where i went to college
the way we voted for student government
was based on this
is it called ranked choice where you're
eliminated you eliminate the bottom
and there's runoff elections so so i
that was the first time i ever saw that
and i thought that made sense
yeah the only problem is it doesn't seem
so easy to get something that makes
sense adopted as the new voting system
that's a whole nother there's that's not
a math solution that's uh
well it's math and sentences game
theories you have to come up with
incentive it's mechanism design you have
to figure out how to
trick us despite our basic human nature
uh to adopt uh solutions that are better
yeah that's a whole other conversation i
think um
can you just because it sounded really
cool uh talk a little bit about
stochastic coalescence
and uh you have a paper on showing that
something you describe what it is but i
guess it's a super linear
super logarithmic time and you came up
with some kind of trick to make it
faster
just can you just talk about it a little
bit yeah so this was something which
came up
when i was at microsoft research for a
summer and that i'm putting that context
because that shows that it has some
practical motivation at some point
actually i think it still it doesn't
need to yeah it doesn't need to it could
be beautiful and it's all right yeah
so the easiest way to describe this is
suppose you got like a big crowd of
people
and everybody knows how many hours of
sleep they got last night
and you want to know how many total
hours of sleep were gotten by this big
big crowd of people at the beginning you
might say that sounds like a linear time
algorithm
of saying hey you how many hours you got
how many you got how many you got add
add add yes
but there's a way to do this if you
remember that there are people and they
presumably know how to add
you could make a distributed algorithm
to make this happen
for example while we're thinking of
these trees imagine you had 1024 people
if you could just say hey person number
one and person number two you will add
your hours of sleep
person number two will go away and
person number one is going to remember
the sum
person three and four add up and person
three takes charge of remembering it
person four goes away now this like
person one knows the sum of these two
person three knows some of those two
they talk
you see what i mean it's like you're
going up this tree
same tree that we've talked about
earlier built up a tree from the bottom
up
yeah build a retreat from the bottom up
and the beautiful thing is
since everyone's doing stuff in parallel
the amount of time it takes
to get the total sum is actually just
the number of layers in the tree
which is 10. so now that's logarithmic
time to add up the number of hours
that people slept today sounds fantastic
yeah
there's only one problem how do you
decide who's person number one and
person number two
yes so if for example you just went out
into the downtown and said hey get these
thousand people
go well if you're gonna go and say and
by the way you're one and you're two and
you're three that's linear time
yes that's cheating so now the question
is how to do this in a distributed way
and there were some people who proposed
a very elegant algorithm
and they wanted to analyze it that's so
i came in onto the analyze side
but the elegant algorithm was like this
it was like well
we don't actually know what this big nut
big tree is there isn't any big tree
so what's going to happen is first
everyone is going to
decide right now oh what one important
thing
everyone is going to at the very
beginning of the whole
game uh they will have delegated
responsibility to themselves
as the one who knows the sum so far
so so the point is there's there's going
to be people are all going to have like
a pointer which says
uh you are the one who knows my you
you've taken care of my ticket my number
yeah okay they select the representative
for this
particular piece of knowledge and at the
very beginning you're your own
representative
the thing has to start simple right so
at the beginning you're pointing to
yourself you've got it
yep and and how the way this works is
that at every time step someone blares a
ding dong on the on the on the
tongue clock or whatever yes and each
person flips a coin themselves to decide
am i going to hunt for somebody to
give my number to and let them represent
me
or am i going to sit here and wait for
someone to come
okay okay well they they flip their coin
some of the people start asking other
people saying hey i would like
you to be um my representative here is
my number
but the problem is that there's limited
bandwidth of the people who are getting
asked it's like you can't
get you can't go out to prom with five
people this is not what we're doing
we're adding numbers okay
but you can only add one number yeah so
the person who has suddenly gotten asked
by all these people
well they'll have to decide who they're
going to take it from
and they randomly just choose one when
they randomly choose one all the others
are rejected and they
don't get to delegate anything in that
round but now if
this person has absorbed this one who
said okay here
you take charge of my number this person
now updates their pointer
you're you're in charge and this person
adds the two numbers
that was the first round in the next
round
when they do the coin flipping this
person doesn't flip anymore because
they're just delegating
it's that anyone who has the pointers
themselves that's a
like a person who is in charge of some
number of informations
they flip the coin to decide should i
find other people who are
agents or should i wait for people to
ask me
yes brilliant this is somebody else's
idea and and now the idea is okay if you
just keep doing this process what ends
up happening
oh yeah oh and also by the way if you
decide that you want to go reach out to
other people here's the here's the
here's the here's the catch
when you're one of these agents saying
okay i'm going to go look for someone
you have no idea who in this crowd is an
agent
or somebody who delegated it to someone
else you just pick a random person
when you pick the random person if it
lands on someone and the person says oh
i i actually delegated it to someone
then you
you you follow up you walk through the
delegation chat up delegation chain and
you can do
like path compression in the algorithm
to make it so you don't consistently do
lots of walking up
but the bottom line is that what ends up
happening is that
you end up reaching out whenever you're
one of the ones reaching out
you can think of it as each agent is
responsible for some number of people
it's almost like they're the leader of a
bunch as the process is evolving
you have these lumps each lump has an
agent and when the agent reaches out
they reach out to another
lump where the probability of them
hitting that lump
is proportional to the size of the lump
yeah that is the one funny thing about
this
process this is not that they can reach
out to a uniformly random lump
where every lump has the same chance of
going to yeah
the bigger the lump is the less like
the more likely it is that you end up
reaching that lamp which is a problem
let me explain why that's a problem
because you see you're hoping that this
has a small number of steps yeah but
here's a bad situation that could happen
imagine if you had like their end people
that you're adding up
imagine that you have exactly square
root of n
lumps left of which
almost all of them are just one person
who's still their own
boss their own their own manager except
one giant one giant one
now what's going to happen is going to
be a huge bottleneck because every round
the giant one can only absorb one of the
others
yes and now you suddenly have time which
is about square root of n
the square root of n is chosen because
that is one where the
the lumps are such that um you really
are limited by this large one
slowly sucking up the rest of them so
the the heart of the question became
well but is that just so unusual that we
don't that it doesn't usually happen
because remember you start with everyone
just being independent it's like a lot
of lumps of size well how naturally do
the big lumps emerge
yes and so what that part of the proof
was was showing that that was a joint
work with al lubetsky
that one was showing that actually
in that thing the lumps do kind of get
out of whack and so it's not
the purely logarithmic number of steps
but if you make one very slight change
which is if you are one of the agents
and you have just been propositioned
possibly relayed along
by a couple of different people if you
just say don't take a random one
but accept the smallest lump
that actually does enough to even
distribute the lump size
yeah that's i mean yeah it's fascinating
how would the distributed algorithms a
little
yes make all the difference in the world
yeah
actually by the way this does this back
to our voting conversation this makes me
think of like
these networking systems are so
fascinating to study
they immediately spring to mind ideas of
how to have
representation like i maybe
as opposed to me voting for a president
i want to vote for for like uh
for you paul to represent me maybe on a
particular issue
and then you'll delegate that further
and then we naturally construct those
kinds of networks
because that that feels like i can have
a good conversation with you and figure
out that you know what you're doing and
i can delegate it to you and
in that way construct a representative
government
or representative decision maker that
feels
that feels really nice as opposed to
like us
like a tree of height one or something
where it's like everybody's just
um it feels like there's a lot of room
for layers of representation to form
organically from the bottom up
i wonder if there are systems like that
this is the cool thing about the
internet
and the digital space where we're so
well connected just like with
the novid app to to distribute
information about the
the the spread of the disease we can the
same way
in a distributed sense form anything
like
any kind of knowledge bases that are
formed in a decentralized way and
uh in a hierarchical way as opposed to
sort of
old way where there's no mechanism for
large scale
fast uh like distributed transactional
information
this is really interesting this is where
almost like network graph theory
becomes practical yeah most of that
exciting work was done in the 20th
century but most of the application will
be in the 21st which is cool to think
about
uh let me ask the most ridiculous
question you think p equals np wow
i don't know i mean i would say
i i know there are enough people who
have very strong interest in trying to
show that it is i'm talking about
government agencies
for security purposes for security
purposes and most computer scientists
which would say believe that p equals np
uh my question almost like this is back
to our aliens discussion you want to
think outside the box
the the low probability event
what is the world what kind of
discoveries would lead us to
to to prove that p does not equal to np
like there could be giant
misunderstandings
or gaps in our knowledge about computer
science about theoretical computer
science about computation
which allow us to think like flatten all
problems
yeah so i don't know the answer to this
question i think i think it's very
interesting but i
actually i know let's put it this way by
being at carnegie mellon and being
around the theoretical computer
scientists
i know enough about what i don't know
what to say i'm wrong to be humble
i'm the only person to answer this
question yeah yeah
it's a great one well scott aaronson
who's now here ut
austin he used to be at mit puts the
probability of uh
p and p not equals to np at three
percent
he put it's i you know i always love it
when you ask
it's very rare in science in in
academics because you're
because most folks are humble
in in the face of the mystery the
uncertainty of everything around us
to have the the both the humor and the
guts to say um
like what are the chance that there's um
aliens in our galaxy intelligent alien
civilizations
as opposed to saying i don't know it
could be you know it could be zero it
could be depending on the fact you're
saying
it's 2.5 percent there's something very
pleasant about just having uh
it's the it's the number thing um
this power to the number it's just like
42 it's like why 40 i don't know but
it's a powerful number
and then everything this is the power of
human psychology is once you have the
number 42
it's not that the number has meaning but
because
it's placed in a book with humor around
it
it has the meme effect of actually
creating reality i mean you could say
that 42 has a strong contribution
of helping us colonize mars because
it created it gave the whatever
existential crisis to many of us
including elon musk when he was young
you know reading a book like that and
then like now 42 is not part of his
humor
he doesn't shut up about constantly
joking about and that humor is spreading
through our minds
and somehow this like silly number just
had an effect in that same way
after scott told me like the three
percent chance
it's stuck in my head and i think it's
been having a ripple effect in everybody
else
the believing that p is not equal to np
scott almost as a joke saying it's three
percent
is make it's actually motivating a large
number of researchers to work on it
three percent is high
it's very high because for the potential
impact that that would happen
but then three percent is not that high
because it's only you know it's
like we're not very good i feel like
humans are only able to really think
about like
one percent 50 and we kind of
i think a lot of people around three
percent up to 50 percent
like in our minds like three percent i
think
it could happen it could happen and it
could happen is like yeah
like half the time it'll probably happen
so we're not very good at that that's
that's the other thing with the pandemic
is uh we're not the exponential growth
that we also talked about offline uh is
uh something that we can't quite intuit
and that's something we probably should
if we were to predict the future to
anticipate the future and to understand
how to create technologies that'll let
us sort of control the future
can i ask you for some recommendations
uh maybe for books or movies
in your life long ago
when you were baby po or today
that uh you found insightful or you
learned
you learned a lot from what you would
recommend to others yeah
so i think i don't necessarily have an
exact name of these old things but
i was generally inspired by stories
true or fictional of
campaigns or you know like for example
like the lord of the rings that's the
content
right but yes the thing that always
inspired me was
it could be possible for somebody who's
crazy enough
to go up against adversity after
adversity after adversity
and it succeeds i mean those are false
those are
fictitious but i also spent a lot of
time i guess reading about i don't know
i was interested somehow in like
world war ii history for whatever reason
that's a campaign which is much more
brutal
but nevertheless the idea of
difficulty strategy uh fighting even
when
things that in that case was really
fighting but just pushing on even when
things are difficult
i guess these are these are the kinds of
general stories that
made me i guess want to work on things
that would be
hard and where it could be a campaign
it could be that you work on something
for a year multiple years
because that was the point yeah it
starts with a single person
that that's that's the interesting thing
i've obviously been
don't shut up about it recently about
world war ii especially on the hitler
side and the stalin side
some of that has really affected my own
family the the roots of my family very
much
but it's interesting to think um
that it was just an idea and one person
decided to do stuff
and it just builds and builds and builds
and
you can truly have an impact on the
world both uh
horrendous and and
uh like exceptionally positive
and inspiring so yeah that's uh it's
like
it's uh agency of us individuals
sometimes we think we're just reacting
to the world but we have the full power
to actually change the
world is there um advice you can give to
young folks
we talked we gave a bunch of advice on
middle school
high school mathematics is there more
general advice you would give about
how to succeed in life how to learn for
high school students for college
students
career or life in general so i think the
first one would be
to make sure that you're learning to
invent and to make sure you're not
just learning how to mimic
because a lot of times you learn how to
do x by watching somebody do x and then
repeating x many times with different
inputs
i'm just being very generic in
explaining this but i guess this is just
my own
attitude towards the world i didn't like
ever following anyone's directions
exactly even if you told me this
this is the way to do your homework is
to write in pencil i would say
but i think that is nice let's try right
so i've been that kind of a funny person
but i do
encourage that if you can learn how to
invent as your core skill then you can
do a lot but then the second piece that
comes with that is something i learned
from my phd advisor
phd advisor which was well make sure
that what you're
working on is big enough and so in that
sense i usually advise to people once
they have learned how to invent
ideally don't just try to settle for
something comfortable
try to see if you can aim for something
which is
hard which might involve a campaign
which might be important
which might make a difference and it's
more of
i guess rather than worrying what if you
didn't achieve that
there's also the regret of what if i
didn't try
see that's how i operate i don't operate
based on did i succeed or fail it was
hard anyway
if i did this novid thing and the whole
thing failed would i feel terrible no
it's a very hard problem but would i
have had the regret of
not jumping in yes so it's that
different mentality of don't worry about
the failing part as much of the
make sure you give yourself the shot at
those potentially unbounded
opportunities you almost make it sound
like there's a meaning to it all uh let
me ask the big ridiculous question what
do you think is the meaning of life
or maybe the easier version of that is
what brings your life joy
so i'll just answer that one personally
for me i'm a little bit weird i sort of
i guess you can tell that but i see the
pen and pencil discussion from earlier
yes yeah yeah
so i mean my thing is i guess i
personally
just wanted to maximize a certain score
which was for how many person years
after i'm no longer
here anymore did what i do mattered
yeah and it didn't matter if it's
necessarily attributed to me it's just
like
did it did it matter and so that's what
i that's what i
wanted um i guess that is very inspired
by
how scientists work it's like why do we
keep talking about newton it's because
newton discovered some interesting
things and so newton's score is pretty
high
it's going to be infinity right well
let's hope it's infinity but pretty high
ah yes yes so you're going for so person
person years you're going for like
triple digits you're going for
it's like newton is like four digits
probably like a thousand years
yeah yeah or personal lifetimes like how
do you like to think well what are we
sorry i've met people times years people
sometimes so so then it's like
actually this is huge this is like going
to be billions or trillions yeah
right the trillions but um i guess
for me i actually changed the metric
after a while and the reason is because
you may have seen i found some simple
way to solve quadratic equations
that is easier than every textbook yes
so
my score might already be not bad which
is why i decided then let's change it
into
the number of hours in the lifetimes as
well so the way i was doing it
before is that
if a person was sort of remembering
or using or or or appreciating what i
had done
for for like 10 years if
of their life oh that would count as 10
i see so if there was one person who for
10 years
yes remembered or appreciated something
i did that counts as a score of 10 and
we add up over all people
yes and then uh and that was with the
hypothesis that the score
would be very finite in the sense that
if i didn't come up with anything that
might potentially help a lot of
generations
in a forever way then your score will be
finite because at some point
it's not people don't remember that you
made like nice bottles or something yeah
right but then after the quadratic
equation thing
it was that there's some chance that
that actually might make it into
textbooks
yes and if it makes it in textbooks the
chance that there will be an easier way
discovered is actually quite small
yes so in that case then the score might
get bigger
i was just saying the score might
actually already have been achieved in
non-trivial way so see because it's fun
to think about
because it could be different you can
achieve a high score
by a small number of people using it for
most of their lifetime and then
generations and generations
or you can have if we do dissipate if we
do spread colonize
become multi-planetary species you could
have that little
um a clever way to solve differential
equations
um uh spread
through like trillions of people as they
yes spread throughout the galaxy
and they it would they would only use it
each one
a few hours in their lifetime but yes
their kids would use it the kids the
kids would use it it'll spread and
you'll have that
impact in that kind of way yes so that's
why i renormalized it because i was like
well that's kind of dumb because what's
the importance of that that will save
people 15 minutes
um but so so what i meant is i didn't
want to count that as the main score
well i'm gonna have to try to come up
with uh some kind of device that
everyone would want to use maybe to make
coffee because
coffee seems to be the prevalent uh
performance performance-enhancing
chemical that everyone uses so
i'll have to think about in those kinds
of metrics yeah but
but you see that that's just giving an
idea of i guess what i found meaningful
in general like whether or not it's like
whether or not that quadratic thing is
important or not
the general idea was i wanted to do
things that would outlast me
yes and that was what inspired me and
that's just how i choose what problems
to work on
and that's the kind of immortalities
ideas that you've
invented living on long after you
in the minds of others and humans are
ultimately not
are like meat vehicles
that carry ideas for brief for just a
few years
may not be the important thing it might
be the ideas that we carry with us
and invent new ones like we get a bunch
of baby ideas in our head
we borrow them from others and then
maybe we invent a new one
and that new one might have a life of
its own and
it's fun it's fun to think about that
idea living from many
centuries to come unless we destroy
ourselves but maybe ai will borrow it
and we'll remember poe as like that one
human that
that that helped us out before we of
course killed him and the rest of human
civilization
on that note well this is a huge honor
you're one of the great educators
i've uh ever gotten a chance to interact
with so it's it's truly an honor that
you would talk with me today
it means especially a lot that you would
travel out to austin to talk to me
it really means a lot so thank you so
much keep on inspiring
and i'm one of your many many students
thank you so much for talking today
thank you thank you it's actually a real
honor for me to talk to you and and to
get this chance to have this
really intellectual conversation through
all of these topics thanks bo
thanks for listening to this
conversation with paul shanlow and thank
you to
jordan harbor the show on it better help
eight sleep and element check them out
in the description to support this
podcast
and now let me leave you with some words
from isaac newton
i can calculate the motion of heavenly
bodies but not the madness of people
thank you for listening and hope to see
you next time