Google's DeepMind AI Just Taught Itself To Walk
Insider Tech0 HN recommenders · 2017–2022
11.5M views · news · dated · Uploaded 2017-07-12
A two-minute Insider Tech clip on DeepMind's simulated agents that learned to walk, run, jump and climb without guidance, with the goofy resulting gaits.
HN uses it as a quick example of reinforcement learning agents acquiring locomotion in simulation, rather than recommending it as a deep resource.
2:25:52The spelled-out intro to neural networks and backpropagation: building micrograd
Andrej Karpathy
18:40But what is a neural network? | Deep learning chapter 1
3Blue1Brown
15:01What Happened to Horses Is Happening to Us
CGP Grey
1:30:28AlphaGo - The Movie | Full award-winning documentary
Google DeepMind
27:14Transformers, the tech behind LLMs | Deep Learning Chapter 5
3Blue1Brown
51:22USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?
USENIX
1:08:40Lecture 1 | Machine Learning (Stanford)
Stanford
5:58MarI/O - Machine Learning for Video Games
SethBling
4 comments from Hacker News
It didn't annoy me, but it definitely gave off a Disney Trailer vibe. I can just hear the deep announcer voice over the video coming in: "Coming Winter 2017, A movie about adventure, friendship, AND simulated self-play! Join Burp, Jenny and Bop on a fantastic adventure .... blah blah ..." etc.. hah Heres an example of good, unobtrusive but fun music for showing AI bots: [VIDEO]
If you watch videos of AIs learning to run [1] or playing Mario [2], you see that there's definitely "try random shit until your reward function gives you positive feedback" as a method for training AIs. The difficult part is developing the reward function. There's a lot of intelligence in our hormone-based incentive system - that we feel pain, that we feel sad, etc. Many of those emotions are pre-programmed. If we find a way to design reinforcement systems with the right goals, we can do it like mother nature did it for several billion years. But it's still a ton of computational power required. If we look at nature as a massively parallelized computational system that defined "just live" as the reward (not because it "wants" it, but because it just emerged), it shows us how much power we would need if we would try to build a completely random process that gains intelligence. I think your proposed method works ("just assemble random solutions and test them") because we've already seen it in nature. But it takes a lot of time and energy, regardless of our computational power. I think we're faster when we find the right boundaries and reward functions instead of randomly trying stuff. [1]: [VIDEO] [2]: [LINK]
I'm not sure about doing that with physical robots, but I've seen a couple interesting iterations using simulated robots [VIDEO]
Hardware is hard. Making an AI that learns to control a purely virtual body appears to be a fairly standard project. 1) [LINK] 2) [LINK] 3) [LINK] 4) [VIDEO]