AWS re:Invent 2016: Tuesday Night Live with James Hamilton
Amazon Web Services5 HN recommenders · 2016–2023
133K views · talk · dated · Uploaded 2016-11-30
James Hamilton's Tuesday Night Live keynote at AWS re:Invent 2016, touring AWS infrastructure at scale including networking, power and custom silicon. It also features NASA JPL's Tom Soderstrom.
HN treats Hamilton's annual talks as the best window into how AWS regions and hardware are actually built.
1:01:20Indistinguishable From Magic: Manufacturing Modern Computer Chips
Andor Gafotas
42:03The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019
GOTO Conferences
40:20"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
19:10Velocity NY 2013: Richard Cook, "Resilience In Complex Adaptive Systems"
O'Reilly
52:35Debugging Under Fire: Keep your Head when Systems have Lost their Mind • Bryan Cantrill • GOTO 2017
GOTO Conferences
59:56AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401)
Amazon Web Services
49:15Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp
Erlang Solutions
47:43"Turning the database inside out with Apache Samza" by Martin Kleppmann
Strange Loop Conference
7 comments from Hacker News
The keynote by James Hamilton[1] is absolutely awesome. You can genuinely tell that James finds all this technology fascinating. He talks about fiber, networking, power, redundancy, scale all with utter enthusiasm. [1] [VIDEO]
Amazon announced they were doing network processing in hardware at Re:work last week [1]. Custom silicon, rather than the FPGA or ASICS they were using when they started the project in 2012. They tout not only the freed CPU resources, but lower latencies and increased security of isolating the networking from the CPUs running hosted VMs. [VIDEO]
Others have answered, but I think the general principle for a Region is that between the AZs you have less than 1ms latency, physical separation between the data centers, but they may still be in the same floodplain, be able to be hit by the same hurricane, etc. More info here if you want to see how regions are structured at a high level of detail: [VIDEO]
James Hamilton talked about their commitment to 25GbE hardware at Reinvent in 2016. Fast forward to ~23m. [VIDEO]
James Hamilton [0][1] and others [2][3][4] have given pretty good overview over the years of the AWS infrastructure. [0] AWS Innovation at Scale (2014), [LINK] [1] Tuesday Night Live (2016), [VIDEO] [2] Operation of the AWS Global Network (2019), [LINK] [3] Foundations of AWS Infrastructure (2018), [LINK] [4] Exploring the AWS Global Network (2018), [LINK]
Watch this video from 2016 by James Hamilton at AWS Re:Invent [VIDEO] It'll tell you lots about how AWS is built
Mark Russinovich gives a good talk most years on the internals of Azure and the systems that run it. [1] is an example. Look for talks from other years as well. Meta also publishes a number of papers/blogs/OSS projects on their engineering site [2] James Hamilton of AWS gives a talk most years on their infrastructure. Worth watching multiple years [3]. [1] [LINK] [2] [LINK] [3] [VIDEO]