MarI/O - Machine Learning for Video Games

I believe it's been posted before, but sethbling (of vanilla-minecraft-hacks fame) pulled off the impossible and did the credit warp on an actual SNES, live[1]. As this video very-nicely explains, this required incredible amounts of pixel-perfect accuracy. Lining up the line of koopa shells on exact x-offsets and very narrow heights mid jump (!) to write the landing-sled - using a controller, realtime - is utterly insane. As a side note sethbling's latest project is teaching[2] (using neuro-evolution) his machine learning bot to play SMW[2], SMB1[3], and Mario Kart (original)[4]. While he machine learning is simple by today's standards, his brief videos give a really nice introduction to the topic that should be accessible to most people. [1] [LINK] [2] [VIDEO] [3] [LINK] [4] [LINK]

Jason Weston (Facebook) etc's work on Memory Nets[1] Anything to do with transfer learning (the 2013 Zero-Shot Learning Through Cross-Modal Transfer [2] paper is a good place to start) The increasing amount of demos around using NNs to generate "things" that look kinda-almost "intelligent". I can't point at a paper, but Andrej Karpathy demo of generating Shakespere-like writing, "Wikipedia" pages and "C" code in The Unreasonable Effectiveness of Recurrent Neural Networks [3] is the kind of thing I'm talking about. The beginnings of work around goal-seeking. The (now Google) DeepMind Atari demo[4] and Marl/O[5] Finally, the work being done on making this stuff usable by programmers (Torch etc). [1] [LINK] [2] [LINK] [3] [LINK] [4] [LINK] [5] [VIDEO]

There is an objective, but it may not be exactly as you reason about it. There's a great video that made the rounds last year about building a neural net that plays Super Mario World, that may help visualize what's going on - [VIDEO] There's also a great snippet in the currently-ongoing AlphaGo videos that explains that when AlphaGo plays in ways that you may not expect, it's because it's strictly worried about _winning_ (even by the slimmest margin) with the greatest probability, and not necessarily by winning handily, like a human might.

If someone did this in the US, can they get in trouble? When exactly does "hacking" a game become illegal? What about people doing it for research like the machine learning mario guy: [VIDEO] . Or tool-assisted speedruns? Or the starcraft broodwar ai? Is it only when you breach the TOS or copyrights? Just curious since it never even occurred to me that such things are punishable.

You mean this script? Or machine learning in general? If the later then it has already been done: [VIDEO] (great educational video)

It's true that game AI is surprisingly under-developped compared to what you see in academia on other topics... but it is not always that basic. What you describe in your football example is often called "Behavior Tree", and there are ways to learn those using machine learning (usually by reinforcement learning, using neural networks and/or genetic programming). There was a popular video showing this applied on Mario [1] ... and a paper published 7 years before that video doing the same thing [2] (more context about this in [3]). I remember seeing something on Gamasutra saying that similar methods are used in some AAA games. [1] [VIDEO] [2] [LINK] [3] [LINK]

This video of a Super Mario playing AI was pretty neat: [VIDEO] It may, or may not be on a useful path to solving harder AI problems. I don't know. But it's a good way of demonstrating value (and publishing papers).

This is from the same guy who made the MarI/O video ( [VIDEO] - using Neuroevolution). In the video he also compares MariFlow vs MarI/O.

connectionism is an approach to the philosophy of mind, the most successful approach as evidenced by recent deep learning hype the only book in the OP I thought was even mildly relevant was the one by Dennet, and it's generally acknowledged that it's pretty bad. A more interesting but flawed approach, that resonates as someone who loves Dawkins and Hitchens (atheists and advocates of evolution), is Gerald Edelman's Neural Darwinism (1978). There is even modern research (that is real, working code) at the University of Texas that could be considered a successor to Edelman: NeuroEvolution of Augmenting Topologies (NEAT) [VIDEO]

Also the video discusses previous work starting at 4:40, so I'm confused what OP is complaining about unless the MarI/O guy re-uploaded with this attribution. [VIDEO]

OP, Togelius, is quite wrong in 1 point at least. tl;dr Seth Bling directly references NEAT and his links[Show More [1]] link to 2 Togelius papers[13][14] among many others ! To wit, in OP, Togelius mentions Seth Bling's MarI/O[1] as lacking scholarship because he didn't mention Togelius: " seen as research, it is atrocious because of the complete lack of scholarship. The guy didn't even know he was reinventing the wheel, and didn't care to look it up. " Rather disdaignfully (sadly/ ironically/ poigniantly) Togelius refers to Seth not by name but only as " some guy ". Yet Seth Bling[2] in his short video, mentions and explains NEAT and directly links Ken Stanley's and Risto Miikkulainen's foundational NEAT paper[10] at U.texas which Togelius builds on [hint: click SHOW MORE under Seth's video] And much more, i.e.: ---- the innovative toolchain Seth uses: Bizhawk to script SNES emulators in Lua [12]! ---- Seth links the wikipedia page on Neuroevolution[11] which directly references 2 of Togelius's own papers[13][14] Seth's [Show More] Links on the MarI/O video[1] are very very comprehensive and (IMHO) indeed a great example of scholarship ! Togelius is a top flight scholar, and publishes in the field of AI game players and level generators[15] so I hope he will welcome some random tinkerer pointing out his error. [full disclosure: I am a Togelius fan] To give Togelius his due he does at least link to Seth's MarI/O video. Perhaps Togelius failed to click / read the Show More li

Thanks for the commentary togelius [1, 2] and deepnet [above]. Anyone interested in adding a soupçon of scholarship to Seth's project? Words do mean different things to different people, in different contexts.. There is a recent body of literature that explores the modern "maker" [3] movement. However, "maker" as a term may not have been a good fit for the OP's argument [1], which contrasted (academic) researchers with so-called "tinkerers". An alternative term for "tinkerer" might be "bricoleur", a loanword from French. (Roughly, it still means one who tinkers: [LINK] but has other meanings depending on the academic lens.) Given that we are discussing AIs that play, in the context of education, we can also go back to Seymour Papert's work on [LINK] . Originally known for work on _Perceptrons_ with Marvin Minsky, AI researcher Papert later adapted theories from education towards the vision of "learning-by-making" and the (young) bricoleur [4]. This can approach can be seen in the evolution from 1960s graphical [turtle] [LINK] to Lego [LINK] to modern day efforts to encourage coding-for-kids [5,6,7]. One of Papert's later collaborators, Sherry Turkel, discusses bricolage as it applies to programming -- [LINK] . When it comes to early education, Turkel argues for epistemological pluralism [8] and cites anthropologist Levi-Strauss in comparing analytic science with a "science of the concrete". We can appreciate both Seth Bling's concreteness [9] and Togelius's original papers fo

If you watch videos of AIs learning to run [1] or playing Mario [2], you see that there's definitely "try random shit until your reward function gives you positive feedback" as a method for training AIs. The difficult part is developing the reward function. There's a lot of intelligence in our hormone-based incentive system - that we feel pain, that we feel sad, etc. Many of those emotions are pre-programmed. If we find a way to design reinforcement systems with the right goals, we can do it like mother nature did it for several billion years. But it's still a ton of computational power required. If we look at nature as a massively parallelized computational system that defined "just live" as the reward (not because it "wants" it, but because it just emerged), it shows us how much power we would need if we would try to build a completely random process that gains intelligence. I think your proposed method works ("just assemble random solutions and test them") because we've already seen it in nature. But it takes a lot of time and energy, regardless of our computational power. I think we're faster when we find the right boundaries and reward functions instead of randomly trying stuff. [1]: [LINK] [2]: [VIDEO]

This is very similar to Picbreeder [1], which allows you to evolve art in a very similar manner, with evolutionary algorithms. It runs on the same evolutionary algorithm behind MarI/O [2], called NEAT [3]. [1] [LINK] [2] [VIDEO] [3] [LINK]

Neuroevolution of Augmenting Topologies is a classic genetic algorithm, as a teenager it was my first genetic algorithm I implemented after watching a Youtube video of an implementation of it - [VIDEO]

If a human brain doesn’t use maths, what does it use? It’s Magic? > "training", more accurately described as references Uh, no, that's not at all how the training process works. The training process very roughly mirrors evolution/natural selection except more more direct and faster. This is a good introduction to the concept of how a computer could learn: [VIDEO] This is a more technical look at how exactly it works in more modern AIs: [LINK] (it's a series) > If it encounters a problem it hasn't been programmed for, it cannot reason a new solution. But an LLM can? It's not as good at it as a human but it can Relevant: [LINK] (if you don’t want to click it it says “referring to AI models as "just math" or "matrix multiplication" is as uselessly reductive as referring to tigers as "just biology" or "biochemical reactions"”) > By your logic, video game characters must use reasoning, when no, they don't. It's maths. “By your logic, if tigers are dangerous, then bananas must be too, when no, they aren’t. It’s biology.”

The genetic programming scene kind of evolved into NEAT, HyperNEAT and ES-HyperNEAT as a meta learning concept. Connections between layers/nodes are serialized as genes of agents with phenotypes and dominant/recessive markers, and an observing CPPN learns to categorize agents into different traits to find more efficient breeding mechanisms. It's a strong concept, and AFAIK it's still used a lot in the robotics world where you have to guarantee behaviors and have to be able to reproduce behaviors due to safety regulations. There was a nice intro video into the underlying base concept which is called NEAT by a youtuber named SethBling [2] In my case I'm using this as a decentralized learning approach, because peer to peer networks can benefit heavily from genetic AIs that can communicate, adapt and learn from themselves. [1] [LINK] [2] [VIDEO]

I'm a novice at machine learning, but Open AI made a python library for reinforcement learning in video games, and a fork of it is still actively maintained [1]. It's been a few years, but I remember being able to get it up and running in a day or two (maybe a weekend). It used the Retroarch emulator, which is compatible with a huge number of emulators and consoles. [LINK] There also SethBling's excellent video on YouTube about machine learning specifically with Super Mario World: [VIDEO] I encourage you to give it a try! I feel that video games are a bit underrated by current AI buzz, I think there's lots of potential for a machine to learn a skill through playing a game. And lots of potential for a game being selected or even created with the goal of teaching a specific skill. However at that point maybe it's better to forego the audio and visuals and speak to the machine in text or with pure data. On the other hand, I have seen a video about convolutional neural networks that feed on each pixel of an image. So perhaps training with sound data, or with the pixels of a spectrogram, could have some positive results. And certainly I would be amused to see a game played in time with music, or possibly even dancing with a songs melody, harmony, and story as well. Anything that's ever been created by humans, existed first in the imagination of a human brain. And you've got one of those. A mental vision pursued and brought from the mind into physical reality is a beautiful thing,

Here is my favorite introduction to NEAT: SethBling's MarI/O - Machine Learning for Video Games [VIDEO]

If your goal is to get the network as small as possible, you could try using NEAT[1]. A cool example of a small network for a hard task using NEAT is MarI/O[2]. [1]: [LINK] [2]: [VIDEO]