Gradient descent, how neural networks learn | Deep Learning Chapter 2
3Blue1Brown4 HN recommenders · 2017–2024
9.7M views · explainer · evergreen · Uploaded 2017-10-16
3Blue1Brown's second neural network chapter explains cost functions and gradient descent, showing how a network learns by minimizing error during training.
HN recommends the series as a very accessible way to understand the math behind machine learning and LLMs.
2:25:52The spelled-out intro to neural networks and backpropagation: building micrograd
Andrej Karpathy
18:40But what is a neural network? | Deep learning chapter 1
3Blue1Brown
1:30:28AlphaGo - The Movie | Full award-winning documentary
Google DeepMind
27:14Transformers, the tech behind LLMs | Deep Learning Chapter 5
3Blue1Brown
51:22USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?
USENIX
1:08:40Lecture 1 | Machine Learning (Stanford)
Stanford
5:58MarI/O - Machine Learning for Video Games
SethBling
3:31:24Deep Dive into LLMs like ChatGPT
Andrej Karpathy
5 comments from Hacker News
This was very timely for me and for anyone else learning, here are the first few videos of the series: [LINK] [VIDEO] [LINK] (Original video) [LINK]
A bit of a side-note, but I think it is an interesting piece of marketing that Amplify Partners decided to sponsor[1] the previous video in this series. I wonder (and hope) we'll see more VCs sponsoring open educational content relevant to their focus. [1] [VIDEO]
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must. A few months ago, I stumbled upon very amazing YouTube Channel 3Blue1Brown which explains math in very accessible way and at the same time I got feeling that I finally started understanding core ideas behind linear algebra and calculus. Just recently he published 4 videos about deep neural networks: [LINK] [VIDEO] [LINK] [LINK] So my fear of ML was gone away and I'm very excited to explore whole new world for neural networks and other things like support vector machines etc
If you want an extensive answer, I recommend the neural network playlist on 3Blue1Brown: [LINK] For a quick answer, watch this segment: [VIDEO]
> when an LLM tells us that the sun rises in the east, it's not because it truly understands astronomy or the solar system. Rather, it has seen the phrase "sun rises in the east" repeated so many times in its training data that this becomes the highest probability answer. It's less about understanding and more about pattern recognition. This is not too different from most human person's experience with basic education ( [LINK] ). Though it is obvious that humans can do more than that too. And while "voting" that may be a close-enough metaphor for the multi-layer perceptron layers, it is not what is happening in the attention layers. I suggest checking out 3blue1brown's videos on gradient descent [VIDEO] and "How might LLMs store facts" [LINK] versus how attention works [LINK]