Kind: captions
Language: en
[Music]
so much fast they had students and
researchers and the things you teach are
pragmatically minded
I practically minded freaking figuring
out ways how to solve real problems and
fast right so from your experience
what's the difference between theory and
practice of deep learning well most of
the research in the deep mining world is
a total waste of time
all right that I was getting it yeah
it's it's a problem in science in
general scientists need to be published
which means they need to work on things
that their peers are extremely familiar
with and can recognize in advance in
that area so that means that they all
need to work on the same thing and so it
really Inc and and the thing they work
on there's nothing to encourage them to
work on things that are practically
useful so you get just a whole lot of
research which is minor advances and
stuff that's been very highly studied
and has no significant practical impact
whereas the things that really make a
difference like I mentioned transfer
learning like if we can do better at
transfer learning then it's this like
world-changing thing we're suddenly like
lots more people can do world-class work
with less resources and less data and
but almost nobody works on that or
another example active learning which is
the study of like how do we get more out
of the human beings in the loop where's
my favorite homage yeah so active
learning is great but it's almost nobody
working on it because it's just not a
trendy thing right now you know what
somebody's suicide interrupt you're
saying that nobody is publishing an
active learning but there's people
inside companies anybody who actually
has to solve a problem they're going to
innovate an active learning yeah
everybody kind of reinvents active
learning when they actually have to work
in practice because they start labeling
things and they think gosh this is
taking a long time and it's very
expensive and then they start thinking
well why am i labeling everything I'm
own
the machines only making mistakes on
those two classes they're the hard ones
maybe you ought to start labeling those
two classes and then you start thinking
well why did I do that manually why
can't I just get the system to tell me
which things are going to be hardest
it's an obvious thing to do but yeah
it's it's just like like transplant
learning it's it's under studied and the
academic world just has no reason to
care about practical results the funny
thing is like I've only really ever
written one paper I hate writing papers
and I didn't even write it it was my
colleague sebastian ruder who actually
wrote it I just did the research for it
but it was basically introducing
transfer learning successful transfer
learning to NLP for the first time the
algorithm is called GLM fit and it
actually I actually wrote it for the
course for the first day of course I
wanted to teach people in LP and I
thought I only want to teach people
practical stuff and I think the only
practical stuff is transfer learning and
I couldn't find any examples of transfer
learning and NLP so I just did it and I
was shocked to find that as soon as I
did it was you know the basic prototype
took a couple of days smashed the
state-of-the-art on one of the most
important data sets in a field that I
knew nothing about and I just thought
well this is ridiculous
and so I spoke to the best unit and he
kindly offered to write it up the
results and so it ended up being
published in a CL which is the top link
with computational linguistics
conference so like people do actually
care once you do it but I guess it's
difficult for maybe like junior
researchers or like like I don't care
whether I get citations or papers
whatever I was right there's nothing in
my life that makes that important which
is why I've never actually bothered to
write a pic of myself now for people who
do I guess they have to pick the kind of
safe option which is like yeah make a
slight improvement on something that
everybody is already working on
you