Bringing the Unix Philosophy to Big Data by Bryan Cantrill: FutureStack13
New Relic2 HN recommenders · 2014–2022
18K views · talk · dated · Uploaded 2013-11-12
Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.
HN recommends it as an elegant argument for simple Unix tools replacing heavyweight Hadoop-style pipelines.
19:09A Better Default Colormap for Matplotlib | SciPy 2015 | Nathaniel Smith and Stéfan van der Walt
Enthought
47:18The Tragedy of systemd
linux.conf.au
40:20"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
31:44The Value of Values with Rich Hickey
InfoQ
58:51DON'T PANIC — Hans Rosling showing the facts about population
Gapminder Foundation
1:48:55The Thirty Million Line Problem
Molly Rocket
1:03:51VCF East: Ken Thompson interviewed by Brian Kernighan
Vintage Computer Federation
59:56AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401)
Amazon Web Services
3 comments from Hacker News
They're good points, but from the philosophical point The Unix Way , systemd leaves some things to desire. Is there a modern Unix way? I don't see a lot of exploring in this area/side, but I see a lot of this is the way it's gonna be from now on on the other side. I'm not on either side, but I appreciate the way some complexity is dealt with simple little things. So elegantly solving problems: [VIDEO]
This article echoes a talk Bryan Cantrill gave two years ago: [VIDEO] It's about how Joyent took the concept of a UNIX pipeline as a true powertool and built a distributed version atop an object filesystem with some little map/reduce syntactic sugar to replace Hadoop jobs with pipelines. The Bryan Cantrill talk is definitely worth your time, but you can get an understanding of Manta with their 3m screencast: [LINK]
1.) Be curious. Stay curious. Experiment, play with things. Your time during university is likely the best time for this and it will likely get harder later on when you're on the job and there's much more emphasis on finished work than on learning experiences. Also, I'm quite sure that it will be beneficial for you if you familiarize yourself with different ways of thinking (obviously programming languages, environments and paradigms, but that will likely apply to anything non-technical, as well). I gained this perspective from dealing with people who didn't really do that, but also from knowing that I personally also did this way too late (and from the benefits it brought me nonetheless). If you're only familiar with Windows, try Linux. Try BSD. Try OS X, if you want to. Try working in the terminal. Familiarize yourself with different editors (especially Vim and Emacs, maybe also newer ones like Kakoune). Learn bash, have a look at AWK and sed. Maybe learn Perl. Try out functional programming, logic programming (like Prolog) and so on and so forth. If you're feeling adventurous, play with niche systems like Haiku[1] or TempleOS[2], just to gain new insights on how things could work. Maybe try the ACME editor[3]. [1]: [LINK] [2]: [LINK] [3]: [LINK] Of course, there might be courses that will nudge towards some of those tools. If you visit some of those: Cool! But if there are no such courses (I didn't encounter many of them myself), try to explore these things nonetheless. Ev