Survivor Bias
2veritasium1 HN recommenders · 2016–2022
1M views · explainer · evergreen · Uploaded 2016-05-10
A Veritasium video on survivor bias, explaining how an observed sample can be incomplete and skewed. It uses examples such as bombers and famous success stories.
HN uses it as a handy reference whenever a thread needs to point at survivorship bias.
57:151. Introduction to Human Behavioral Biology
Stanford
2:51:44Essential Ideas for Parents
Jason Salotti
19:09A Better Default Colormap for Matplotlib | SciPy 2015 | Nathaniel Smith and Stéfan van der Walt
Enthought
3:54:44The Century of the Self (Full Adam Curtis Documentary)
David Lessig
58:51DON'T PANIC — Hans Rosling showing the facts about population
Gapminder Foundation
1:16:27Randy Pausch's Last Lecture: Achieving Your Childhood Dreams
Carnegie Mellon University
2:23:07Alain de Botton: Status Anxiety
Alain de Botton
2:16:32Controlling Your Dopamine For Motivation, Focus & Satisfaction
Andrew Huberman
7 comments from Hacker News
Makes me think of the survivor bias episode of Veritaseum [VIDEO] or the french caster ScienceEtonnante on the Simpson Paradox [LINK] [LINK]
> my primary source of linus news comes from articles It's easy to fall into survivorship bias[1] when "news" is your only exposure to a topic. You are only seeing the extreme events that were "newsworthy". > why should i care to seek out more about one who i find is well known to throw e-mail tantrums Do you want to have an accurate understanding of reality or the distorted view limited to whatever was sensational/scandalous enough to be "newsworthy"? When knowledge is derived only from the extreme cases that make headlines, people start to believe inaccurate generalization like "black people are criminals", "Muslims are terrorists", or "gamers are antisocial nerds that might shoot up a school" even though all of these examples refer only to the most extreme members of the group[2]. > but those articles have been my portal to his world If you're going to make accusations about someone, you should probably check if those articles (aka indirect knowledge) are both accurate and representative. [1] [LINK] [2] [VIDEO]
I currently live on SSDI due to unfortunate medical problems that made driving impossible. Three doctors independently looked at a particular lab result and said, "How are you still alive?". (Previously I had a decent engineering salary writing embedded firmware) SSDI pays very close to the federal poverty level. Even with the benefit of receiving Medicare, the monthly cost of very-necessary medication plus very frugal living expenses usually leaves me with up to ~$(40 +/- 40) each month in "discretionary spending" that I can use to buy anything beyond rent/medication/food. (e.g. clothes, household cleaning/repair/etc, very rarely travel expenses) I'm lucky that my interest has always involved computers, which I can use for free for many different purposes. Well, as long as nothing happens to my old desktop (Core 2 Quad Q9650, nForce 790i, RAID-1 of ancient SATA disks). An extra $300/yr would would mean buying new boots and some new shirts. Paying that would mean dropping on of my medications (maybe fatal in the short term, defiantly fatal within a year or two), or skipping a few months of food . For the record, I'm fine living frugally. I simply suggest that it's important to avoid falling victim to a filter bubble or survivor bias[1], because there are a lot of people - even engineers in Silicon Valley - that have an even worse[2] situation that mine. [1] [VIDEO] [2] I'm fortunate to have the privilege of being male and white . My friends without those privileges have only
Happy to see Veritasium getting some attention on HN. You may have seen his previous video from 2014, Facebook Fraud: [LINK] One of my favorite videos is on his alternate channel, 2veritasum. Survivor Bias: [VIDEO]
> Allowed to be bald?! Allowed to be bald and keep their job . > What are they supposed to do? Straighten it? YES . Again, from the article: >> But dismissing certain styles as “distracting” can also amount to discrimination, especially when it comes to women of color in the industry. “You’ll do better with straight hair,” says Brittany Noble Jones, a digital and broadcast journalist who is black and who relaxed her natural hair for years. For many like her, the expectation isn't just that you'll conform to a certain anchor bob — it includes replacing your hair's natural texture with something else. Also, the business doesn't pay for the chemical relaxers. That gets expensive on a local news anchor salary (2017 avg starting pay: $29,500). > The same standards exist for men. All newscasters look the same. They obviously don't, but I suspect it's difficult for you to see this because comments like this suggests a limited, biased experience possibly due to Survivorship Bias[2]. Yes, all newscasters look very similar! That's exactly the point of the article! You don't see many women , people of color , LGBT people with the hairstyles they consider natural (or, in the some cases, are natural due to genetic differences). Men are allowed some additional variance, but a lot of people - especially women, and doubly so for women of color - are being forced to damage their hair at personal expense to maintain a look that “is stereotypically heteronormative, not overly sexy, and predicta
is that actually the case? [are model weights being published less frequently?] It sure feels like it. But it could be survivorship bias: [VIDEO] or regression to the mean: [LINK] . It's possible that StyleGAN and GPT-2 just happened to coincide, and both companies just happened to decide to release their models rather than hoard them, and that the usual case is for companies to rarely (if ever) release weights with their research. But the more interesting question is, why wouldn't they? So onward to your second one: are ethical concerns actually the reasons some models/weights are not released? Or is it about hoarding a potentially valuable commodity? Or some other reason, or a combination of several? As with most things, the answer is likely the mundane, dull truth that big companies tend to have lots of inertia and friction preventing such things from happening. Sources of friction seem to include (but are not limited to): - are we legally responsible for what other people do with the model? (no.) - should we restrict people from using the model commercially? (why bother? but sure, nVidia did that for stylegan. It didn't stop Artbreeder from blatantly ignoring and monetizing it anyway. And while that sounds negative, Artbreeder is one of the coolest ML projects ever, as far as I'm concerned, so the world is better off for ignoring your stupid policy.) - What if people start misusing the model? (So what? I feel like I should just get in the face of whoever is asking this qu
IMHO, this was not cheating. This is a phenomenon called success. If you are fortunate (?) to be around in the entrepreneurial world long enough, you will realize that intelligence, hard work, good time management etc are not adequate. They just qualify as deserving. There are serendipities that separate those who succeed from everybody else who was deserving. Survivor bias [1]. Calling Eli was your serendipity. The opposite is also true: killer asteroids hit the planets of otherwise deserving individuals and initiatives. I was co-founder of a municipal fiber-to-the-home initiative in 1996. We had both a winning technical recipe and also a financial engineering recipe. (Extremely rare even today) We were then fortunate enough to acquire a customer that was a newly minted public company, representing a whopping 65% of our business. Banker's dream, but in actuality a big mistake. Then the double-whammy of fraud by that public company (wiping it out completely before our contract was realized) AND the dot-com crash. Killer asteroids are real. [1] [VIDEO]