By using Kubernetes, you get a scalable infrastructure.
By using OTP/Akka, you get a scalable application.
While there is common problems to both domains, they are still 2 different domains.
For example, using only Kubernetes, you won't have the ability to react to a Pod restart within your application (unless your application is aware of Kubernetes).
Using only OTP/Akka, you still need a workflow for deployment and infrastructure management, and you still need to implement node discover for clustering.
NB: For Elixir, you have libcluster[1] that can use different strategies for node discovery, including using a Kubernetes Service to discover the nodes (as Pods).
EDIT: Using Kubernetes, libcluster, and Horde[2], you get the best of both worlds IMHO.
Thanks! I was just going to point out that Elixir/Phoenix + Libcluster + K8s is like a match made in heaven. I haven't tried Horde yet but I'm quite intrigued now.
Horde uses a CRDT[1] (Conflict-Free Replicated Data Type) to provide a distributed Supervisor/Registry, this allows you to run an OTP supervision tree only once in your cluster, with automatic takeover. Basically, you run your supervisor on each nodes, but only one will run it (thanks to the CRDT).
I find it very useful because "Distributed OTP Application" are a pain IMHO (they must be started during the boot of the BEAM).
wow thankyou! this is actually handles a usecase we're trying to solve in an upcoming sprint. (each tenant needs to maintain a persistent websocket connection to a third party api but though us) I was trying to use registry to do it but was was wondering how I'd scale it once it got big enough.
Well, yes and no.
By using Kubernetes, you get a scalable infrastructure. By using OTP/Akka, you get a scalable application.
While there is common problems to both domains, they are still 2 different domains.
For example, using only Kubernetes, you won't have the ability to react to a Pod restart within your application (unless your application is aware of Kubernetes).
Using only OTP/Akka, you still need a workflow for deployment and infrastructure management, and you still need to implement node discover for clustering.
NB: For Elixir, you have libcluster[1] that can use different strategies for node discovery, including using a Kubernetes Service to discover the nodes (as Pods).
EDIT: Using Kubernetes, libcluster, and Horde[2], you get the best of both worlds IMHO.
[1] - https://hexdocs.pm/libcluster/2.5.0/readme.html
[2] - https://hexdocs.pm/horde/0.1.4/api-reference.html