HackerTube

Lecture 1: Introduction to Information Theory

Jakob Foerster3 HN recommenders · 2018–2024
411K views · lecture · evergreen · Uploaded 2014-04-26

The first lecture of David MacKay's Cambridge course on information theory, pattern recognition and neural networks. It introduces the core ideas of his book Information Theory, Inference, and Learning Algorithms.

Recommended as an enjoyable, clear introduction to information theory that pairs well with MacKay's free textbook.

Information theory & signal processingUniversity lectures & coursesMachine learning & deep learning

Watch on YouTube

6 comments from Hacker News

@swframe22018 · in “Probabilistic Programming and Bayesian Methods for Hackers (2013)”

I recommend David MacKay's Intro to Info Theory class: [VIDEO] There is a free book for it at: [LINK] Daphne Koller's 3 coursara classes on Probabilistic Graphical Models are also really good.

RecommendsView on HN
@maxov2020 · in “Ask HN: What, in your opinion, are the greatest and most useful textbooks?”

Had . Unfortunately Professor MacKay passed away from cancer in 2016. We lost a great soul. I concur that his textbook and sense of humor were fantastic. It was very enjoyable for me to learn the material from his lectures. They are a great supplement to the textbook (or maybe vice versa?) and you can find them here: [VIDEO]

RecommendsView on HN
@samch932021 · in “Sustainable Energy without the Hot Air (Revised, Community Edition)”

It‘s such a tragedy that he died so young. Prof Mackay was a role model who impacted our world in many positive ways (e.g. his work in information theory, Bayesian statistics, environmental science). I recommend his lecture on information theory which is available on youtube [1]. He also documented the progress of his cancer very detailed on his blog [2], which is a very interesting (and sad) read. [1] [VIDEO] [2] [LINK]

RecommendsView on HN
@noud2023 · in “David MacKay: Information Theory, Inference, and Learning Algorithms: The Book”

"Your browser does not support playback of available video formats. Please install Adobe Flash player or upgrade to a more modern browser." ...Okay? I found the videos on youtube as well: [VIDEO]

CitesView on HN
@scythmic_waves2023 · in “David MacKay: Information Theory, Inference, and Learning Algorithms: The Book”

From lecture 1 [1]: > Ok, so, we have an answer of 1000 plus or minus 30 thanks to "Mr. Binomial", the inventor of the distribution that bears his name. Lol, this is great. [1]: [VIDEO]

CitesView on HN
@teleforce2024 · in “Reading list to join AI field from Hugging Face cofounder”

Thanks for the list. Interesting to see that the book by the late David MacKay made it to the list, it is available to download in several formats from the book's website. Information Theory, Inference, and Learning Algorithms: [LINK] The lectures based on the book by David MacKay himself: [VIDEO] Probably it's not a surprise that the father of information theory Claude Shannon is the earliest one to propose stochastic LLM based on Markov chain.

CitesView on HN