Kubernetes or K8s is a system built to do container orchestration, which is running and managing containers on top of an existing computing resources.
K8s is a system of multiple components that interact with each other intricatedly. Understanding and Operating on K8s are not trivial tasks- there are thousands of pitfalls and the project itself is constantly evolving.
This page tracks my notes on working with K8s.
K8S components
Control Plane components are:
- kube-apiserver: HTTP API, stateless and horizontally scalable. Every other component interacts with this, the Kubernetes API, to keep track and update the state of things. It runs on a database,
- etcd: The data layer of K8s. It is highly-available and is a key-value store. Obviously stateful but can be distributed. TODO: Why is etcd picked?
- kube-scheduler: a daemon, stateless, it queries kube-apiserver and try to schedule (find a node and run the containers) pods. There can be multiple instance of this but they uses leader election process to determine a leader. That leader is the only active instance.
- kube-controller-manager: Runs the control loop, which decide what to do to reach a desired state i.e the classic thermostat examples. TODO: Know more about this. Same with scheduler, there can be one controller manager although you can install additional controller. (TODO) One such controller is (TODO)...
- cloud-control-manager: Cloud/Platform specific logic for certain operations. A common theme with Kubernetes is that a significant of a running Kubernetes system is up to implementation. For Example, when you specify a service with type
LoadBalanceron AWS, the cloud control manager will creates NLB to satisfy this. (NLB is an AWS service).
The de-facto rate that Cloud charges you to run these components are ten cents($0.1) per hour. Similar to other system sith a "control plane" (Like routing/switching), the load on these components are small, espeially comparing to the amount of workloads a K8s cluster usually hosts. The services are EKS(AWS), GKE(Google Cloud), AKS(Azure), among others.
You pay for:
- Cloud providers to automate (You are not paying for people to manually operate and maintain them) running the components for you, as unless you are doing supercomputing, these components can run just fine on a very small computer.
- The ease (with a HUGE asterisk) and an opinionated way to set up these components. Since there are multiple components there are many, many ways (TODO add links) to get them configured, up and running.
- Maybe the compute resources need to run these components, the main point being that you don't have to care about resources (CPU/Memory/Disk) requirement of the components - although you tend to be overpaying (re-read point 1, and Cloud providers does not do bulk discount (TODO Commitment Options) with K8s)
A "managed" K8s cluster on the Cloud will cost you a bit more than $73 a month (Cloud Providers, like AWS EKS does not recommend you to spin up and destroy cluster at a wimp, and there is no obvious way to extract just the data part in etcd out).
You do NOT need multiple clusters for whatever reason. AWS EKS also now start charging for cluster that you do not upgrade because of reasons (TODO reasons).
Here is my short rule of thumb: If it costs more than $73 a month (e.g if you are setting for a year this is $876) for you to maintain these components, then use the K8s services of the Cloud, otherwise, it is a waste of money.
Do not get into the habit of spinning up multiple clusters for different purposes. Try to use K8s constructs like namespaces for things like environment and workload segregation. Doing things this way could mean that you facilitate people to spin up cluster whenever they want, like with Rancher. This is expensive for YOU and pretty much is just free money for the Cloud provider.