After a lot of research and conversations with peers in other companies, we learned that something like this can be achi

This does not apply in case of non-production environments (such as development, staging and CI). These environments don’t get any spike in traffic. Theoretically you can run infinite containers if you set CPU requests to zero and set a high enough CPU limit for your containers. If your containers start utilizing a lot of CPU, they will get throttled. You can do the same with memory requests and limits as well. However, the behaviour of reaching memory limits is different than that of CPU. If you utilize more than the set memory limit, your containers get OOM killed and they restart. If your memory limit is abnormally high (let’s say higher than the node’s capacity), you can keep using memory but eventually the scheduler will start evicting pods when the node runs out of available memory.

Pods can be provisioned on any node. Even if you control how pods are spread in your cluster, there is no easy way to control how services discover each other in a way that a pod of one service talks to the pod of another service in the same AZ to reduce cross-AZ data transfer.

In non-production environments, we safely over commit resources as much as possible by keeping resource requests extremely low and limits extremely high. The limiting factor in this case is memory i.e. no matter how low the memory request is and how high the memory limit is, pod eviction is a function of sum of memory utilized by all containers scheduled on a node.

http://team.vidrio.org/myp/video-Odense-Bulldogs-Frederikshavn-White-Hawks-v-en-gb-1dbm30122020-10.php

http://main.ruicasa.com/tgq/video-gmac-bologna-v-pinar-karsiyaka-v-it-it-1mcq2-12.php

http://molos.bodasturias.com/jph/Video-bili-tygri-liberec-v-vitkovice-ridera-v-cs-cs-1zpa-11.php

http://team.vidrio.org/myp/videos-Odense-Bulldogs-Frederikshavn-White-Hawks-v-en-gb-1ipa30122020-13.php

http://startup.munich.es/mlt/videos-KH-Sanok-GKS-Katowice-v-en-gb-1hou-10.php

http://molos.bodasturias.com/jph/v-ideos-bili-tygri-liberec-v-vitkovice-ridera-v-cs-cs-1whx-9.php

http://main.ruicasa.com/tgq/videos-gmac-bologna-v-pinar-karsiyaka-v-it-it-1anm2-21.php

http://elta.actiup.com/cqn/Video-Spain-Croatia-v-en-gb-mff30122020-.php

http://team.vidrio.org/myp/Video-Herning-Blue-Fox-Herlev-Eagles-v-en-gb-1ith30122020-.php

http://elta.actiup.com/cqn/videos-Spain-Croatia-v-en-gb-hih-.php

http://elta.actiup.com/cqn/video-Spain-Croatia-v-en-gb-nvx-.php

http://startup.munich.es/mlt/videos-KH-Sanok-GKS-Katowice-v-en-gb-1hgq-16.php

http://molos.bodasturias.com/jph/Video-bili-tygri-liberec-v-vitkovice-ridera-v-cs-cs-1wgq-5.php

http://team.vidrio.org/myp/Video-Herning-Blue-Fox-Herlev-Eagles-v-en-gb-1rcu30122020-26.php

http://main.ruicasa.com/tgq/videos-motor-ceske-budejovice-v-olomouc-v-cs-cs-1nea-16.php

http://team.vidrio.org/myp/videos-Herning-Blue-Fox-Herlev-Eagles-v-en-gb-1vhz-17.php

http://main.ruicasa.com/tgq/v-ideos-motor-ceske-budejovice-v-olomouc-v-cs-cs-1vhy-1.php

http://startup.munich.es/mlt/videos-sts-sanok-v-katowice-v-pl-pl-1ydp-5.php

http://elta.actiup.com/cqn/video-spanjolska-v-hrvatska-v-hr-hr-1ohp-28.php

http://www.ectp.org/kzz/videos-TPS-Turku-JYP-Jyvaskyla-v-en-gb-xom30122020-.php

e don’t know. But we can speculate that other intelligent beings will be able to figure out math and extrapolate the laws of physics. Even if they interpret them differently, their technology has to follow a path similar to ours. It’s inevitable they’ll undergo a stone age until discovering metals, followed by increasingly efficient ways to put energy to work — leading to what we recognize as advanced technology.

Deploying Open Policy Agent to build the right controls helped automate the entire change management process and build the right safety nets for our developers. With Open Policy Agent, we can restrict scenarios like one just mentioned before — it is possible to restrict service objects from getting created unless the right annotation is present so that developers don’t accidentally create public ELBs.

Spot instances helped us get massive savings. Today, our entire stage Kubernetes cluster runs on spot instances and 99% of our production Kubernetes cluster is covered by reserved instances, savings plan and spot instances.

High infrastructure cost due to large resource buffers was a big problem. We were not really able to realise any benefits of capacity utilisation due to Kubernetes that we should have. It was after migrating to EKS and observing the stability it brought helped us become more confident, which helped us take the necessary steps to correct resource requests and bring down resource wastage drastically.

This was the most obvious one. Our infrastructure today has far less compute, memory and storage provisioned than we had before. Apart from better capacity utilisation due to better packing of containers/processes, we were able to better utilise our shared services such as processes for observability (metrics, logs) than before.

We used Ingress to consolidate ELBs in our stage environment and reduce the fixed costs of ELBs drastically. To avoid this from becoming a cause of dev/prod disparity in code, we decided to implement a controller that would mutate LoadBalancer type services to NodePort type services along with an ingress object in our stage cluster.

Our learning was to keep resource requests high enough but not too high so that during low traffic hours we are wasting resources and keep resource limits relatively close to resource requests to allow for some breathing room for spiky traffic without pod evictions due to memory pressure on nodes. How close must the limits be to requests depends on your traffic patterns.

Migration to Nginx ingress was relatively simple for us and didn’t require a lot of changes because of our controller approach. More savings can come if we use ingress in production as well. It’s not a simple change. Several considerations have to go in configuring ingress for production the right way and needs to be looked at from the perspective of security and API management as well. This is an area we intend to work in the near future.

Sometimes this independence could pose severe risks. For example, using the LoadBalancer type service in EKS provisions a public-network facing ELB by default. Adding a certain annotation would ensure that an internal ELB is provisioned.We made some of these mistakes early on.

Kubernetes is meant to unlock the cloud platform for developers, make them more independent and push the DevOps culture. Opening up the platform to developers, reducing intervention by cloud engineering teams (or sysadmins) and making development teams independent should be one of the important goals.

Using spot instances with Kubernetes is a lot easier than using spot instances with vanilla VMs. With VMs, you can manage spot instances yourself which might have some complexity of ensuring a proper uptime for your applications or use a service like SpotInst. The same applies to Kubernetes as well but the resource efficiency brought in by Kubernetes can leave you enough room for keeping some buffer so that even if a few instances in your cluster get interrupted, the containers scheduled on them can be quickly rescheduled elsewhere. There are a few options for efficiently managing spot interruptions.

However, initially we had an enormous amount of wastage of resources while we were migrating. Owing to our inability to tune our self-managed Kubernetes cluster the right way which led to a ton of performance issues, we ended up requesting a lot of resources in our pods as buffer and more like insurance to reduce chances of outages or performance issues due to lack of compute or memory.