ElixirConf 2018 - Docker and OTP Friends or Foes - Daniel Azuma
ElixirConf4 HN recommenders · 2018–2023
16K views · talk · dated · Uploaded 2018-09-07
Daniel Azuma's ElixirConf 2018 talk on whether Docker and OTP conflict. He shows they can coexist and uses Horde and DeltaCrdt, with a live tank-game demo, to keep state alive across container kills.
Recommended for its practical strategy for preserving state on ephemeral containers, and for the live demo.
53:22Ruby Conf 12 - Y Not- Adventures in Functional Programming by Jim Weirich
Confreaks
42:03The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019
GOTO Conferences
1:01:39"Simple Made Easy" - Rich Hickey (2011)
Strange Loop Conference
25:06"Making Impossible States Impossible" by Richard Feldman
elm-conf
1:31:06William Byrd on "The Most Beautiful Program Ever Written" [PWL NYC]
PapersWeLove
40:20"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
31:44The Value of Values with Rich Hickey
InfoQ
19:10Velocity NY 2013: Richard Cook, "Resilience In Complex Adaptive Systems"
O'Reilly
6 comments from Hacker News
You can use Docker + containers with OTP, they are not mutually exclusive. Daniel Azuma gave a talk at the most recent Elixirconf about this subject.[0] [0]: [VIDEO]
Horde is indeed a really interesting Elixir library. It's based on the CRDT library, DeltaCrdt [0], also by Derek Kraan. He discusses DeltaCrdt here [1] and Horde here [2]. BTW, Horde is a distributed supervisor, i.e. analogous to what regular Elixir OTP supervisors do, except that it can restart processes on different cluster nodes. It also provides a distributed process registry. Daniel Azuma of Google gave a great 39 minute talk at ElixirConf 2018, "Docker and OTP: Friends or Foes?" [3], where he shows Horde (and DeltaCrdt) in action to keep a multi-player tank video game going while the Docker container it's running in gets killed and and the Elixir processes get respawned on another container. The state is persisted across Elixir nodes in other containers in an instance of DeltaCrdt. (Again, the talk is really good -- if the above sounds interesting, you'll want to check it out [3]). [0] [LINK] [1] [LINK] [2] [LINK] [3] [VIDEO]
Honestly, in most scenarios, you kill the vm and let it restart with new code (just like you would in a kubernetes pod, and in fact using kubernetes pods with BEAM languages is also a valid strategy). If you must keep some state preserved, there's even ways to do it without erlang's hot code reloading strategies, I would recommend watching Dan Azuma's fantastic video (stick around for the live demo): [VIDEO]
In the elixir community CRDTs are well known for various HA; AP distributed state strategies, Phoenix Presence uses CRDTs to track processes and broadcast them over pubsub channels. This makes it trivial to represent "user state" in chat rooms, Sophie DiBenedetto describes using it for writing a trivial chat app: [LINK] For something a bit wilder, you can track your state machines across a distributed cluster and be able to pass and preserve state across the cluster using CRDTs, as is done in Dan Azuma's (live) demo of a tank game: [VIDEO] (worth it for the demo at 29m)
> you should store the state in the external system It's an old talk, but this one has a strategy for robustly persisting state across ephemeral containers. The example here is running a multiplayer game server while doing code updates etc. using Horde and CRDTs [VIDEO]
Can a LiveView process migrate to a different machine/VM? Liveview makes no assumptions about the client connection or even stickiness of client to nodes in a cluster. If you store critical data in the liveview you could be in for some trouble. Always store important stuff into a recoverable, network aware datastore. The nice thing is that liveview is designed to be tolerant to client or client connection failures, you should design to be able to repopulate state sanely in the on_mount call. If you want some sort of ephermal network state that lives on one node and not the database only (e.g. a game), the liveview should connect into that. If you want to store that in a genserver, a strategy like this is doable: [VIDEO]