Agent to the build the right controls helped automate the entire change

October, San Diego police announced that they had arrested a 75-year-old man on suspicion of Mary’s murder. John Jeffrey Sipos from Schnecksville, Pennsylvania had just left the Navy and was living in San Diego when he attacked Mary.
This was the most obvious one. Our infrastructure today has far less compute, memory and storage provisioned than we had before. Apart from better capacity utilisation due to better packing of containers/processes, we were able to better utilise our shared services such as processes for observability (metrics, logs) than before.

https://soe.rutgers.edu/sites/default/files/webform/harutaka-valley-tv04.pdf
https://soe.rutgers.edu/sites/default/files/webform/harutaka-valley-tv.pdf
https://soe.rutgers.edu/sites/default/files/webform/harutaka-valley-tv02.pdf
https://soe.rutgers.edu/sites/default/files/webform/harutaka-valley-tv03.pdf
https://soe.rutgers.edu/sites/default/files/webform/harutaka-valley-tv05.pdf
https://www.umass.edu/studentlife/sites/default/files/webform/harutaka-valley-tv.pdf
https://www.umass.edu/studentlife/sites/default/files/webform/harutaka-valley-tv05.pdf
https://www.umass.edu/studentlife/sites/default/files/webform/harutaka-valley-tv03.pdf
 

Using spot instances with Kubernetes is a lot easier than using spot instances with vanilla VMs. With VMs, you can manage spot instances yourself which might have some complexity of ensuring a proper uptime for your applications or use a service like SpotInst. The same applies to Kubernetes as well but the resource efficiency brought in by Kubernetes can leave you enough room for keeping some buffer so that even if a few instances in your cluster get interrupted, the containers scheduled on them can be quickly rescheduled elsewhere. There are a few options for efficiently managing spot interruptions.

We have also built a few CRDs. One of them is widely used today to generate monitoring dashboards on Grafana by declaratively specifying what monitoring dashboards should be constructed with. This makes it possible for developers to check-in their monitoring dashboards next to their application code base and deploy everything using the same workflow — kubectl apply -f . .

Spot instances helped us get massive savings. Today, our entire stage Kubernetes cluster runs on spot instances and 99% of our production Kubernetes cluster is covered by reserved instances, savings plan and spot instances.

Migration to Nginx ingress was relatively simple for us and didn’t require a lot of changes because of our controller approach. More savings can come if we use ingress in production as well. It’s not a simple change. Several considerations have to go in configuring ingress for production the right way and needs to be looked at from the perspective of security and API management as well. This is an area we intend to work in the near future.

In our two years of journey with Kubernetes, we learned that Kubernetes is great but it’s better when you are using its features such as controllers, operators and CRDs to simplify daily operations and provide a more integrated experience to your developers.

Pods can be provisioned on any node. Even if you control how pods are spread in your cluster, there is no easy way to control how services discover each other in a way that a pod of one service talks to the pod of another service in the same AZ to reduce cross-AZ data transfer.

After a lot of research and conversations with peers in other companies, we learned that something like this can be achieved by introducing a service mesh to control how traffic from a pod is routed to the destination pod. We were not ready to take the complexity of operating a service mesh ourselves just for the benefit of saving the cost of cross-AZ data transfer.

We have started investing in a bunch of controllers and CRDs. For instance, LoadBalancer service type to ingress conversion is a controller operation. Similarly, we use controllers to automatically create CNAME records in our DNS provider whenever a new service is deployed. These are a few examples. We have 5 other separate use-cases where we are relying on our internal controller to simplify daily operations and reduce toil.

We used Ingress to consolidate ELBs in our stage environment and reduce the fixed costs of ELBs drastically. To avoid this from becoming a cause of dev/prod disparity in code, we decided to implement a controller that would mutate LoadBalancer type services to NodePort type services along with an ingress object in our stage cluster.