Yeah, it's basically this. I'm running this as an initContainer for my K8s-based deployments. Took me a bit to get everything going, but my stack is pretty much similar to OP's article, although not quite as advanced in the automated-deployment of containers and monitoring. I'm not at a position where usage needs heavy monitoring because I'm still in the pre-launch phase of things and I'm using this side project to learn stuff I've yet to get experience with at multiple companies.
In the authors case, terraform will create the EKS (kubernetes) cluster, which then is responsible for creating the EC2 instances. The actual application containers are then created by EKS.
Thanks for a great post! It was super detailed and I loved reading it. I had a quick question about your pg setup. You mentioned that you use EBS for your persistence storage, which is locked by zone . You can't have an EC2 instance in Zone 1 mount a storage in Zone 3. Does this cause issues with your db? Especially as you have HPA and ClusterAutoscaler, your k8s nodes could be spun up in Zone 1 for pg autoscaling but your data is in Zone 3.
- Disable autoscaling if appropriate during outage. For example if the web server is degraded, it's probably best to make sure that the backends don't autoscale down.
- Panic mode in Envoy is amazing!
- Ability to quickly scale your services is important, but that metric should also take into account how quickly the underlying infrastructure can scale. Your pods could spin up in 15 seconds but k8s nodes will not!
OpenAI ignore.
OpenAI train.