Docker Swarm or Kubernetes — Help me decide

By Lee Sylvester

High Availability

Both Docker Swarm and Kubernetes provide High Availability and both support the inclusion of multiple Manager nodes, required to ensure high availability should the Leader Manager node fail.

Docker Swarm recommends an odd number of Manager nodes (1 of 3, 3 of 5 etc.) with no more than seven in a given swarm. Each Manager node can be configured to run application services, though this should be applied with caution. Kubernetes Manager nodes run additional containers, known as Pods, that facilitate core Kubernetes functionality. As such, it is considered bad practice to run application services on Kubernetes Manager nodes. Kubernetes Manager nodes are marked with a NoShedule taint to prevent accidental deployment of services to these nodes.

Both Docker Swarm and Kubernetes utilise Ingress networking, which allow the routing of external traffic to internal services. However, while Kubernetes Ingress is standardised, flexible and extensive, the Docker Swarm implementation aims for simplicity and ease of use.

Kubernetes can be configured to use a local Ingress load balancer, such as Ingress-Nginx, a simple exposed public port using NodePort, or a third-party load balancer, such as Amazon’s ELB.

Docker Swarm uses Raft Consensus to manage cluster state. Each Manager node is kept up-to-date with state information so that, should the Leader Manager fail, another Manager can be delegated as Leader and continue the role without significant changes to application stability. This is the reason for a maximum number of Managers. Maintaining state consistency between large numbers of nodes without a third-party store is problematic.

Kubernetes uses Etcd to manage cluster state. An Etcd Pod is deployed to each Manager node as replicated units. Data stored within one Pod instance is distributed to each of the other instances. As Etcd is an open-source data store, it can be utilised by the deployed application to store custom application configuration data.

Kubernetes provides greater configuration for management of service deployment to nodes when scaling or when nodes fail. As such, the decision to choose Kubernetes over Docker Swarm might be relevant when building applications that require fine tuning in order to ensure optimal use of available resources.

Service Discovery

Both Docker Swarm and Kubernetes provide DNS based service discovery. Docker Swarm utilises a DNS service integrated within the Docker Engine. Each service host domain name is equivalent to the provided service name. There is no notion of namespaces. In order to separate connectivity between services, each service can belong to one or more overlay networks. Direct calls to containers can be achieved by using the container alias as the domain or by using the containers assigned IP address.

Kubernetes service domains are maintained by the KubeDNS Pod. Each Pod and service receive an IP address relative to its parent. Thus, Pods within a service can communicate by IP address, too. Kubernetes provides numerous additional means to reference Pods and services through metadata, such as labels and annotations. Like Docker Swarm, Kubernetes services may be referenced by its simple service name. However, Kubernetes also provides for namespacing and use of Fully Qualified Domain Names (FQDN).

The Kubernetes platform also provides a number of environment variables for use in referencing Pods and services in a cluster.

Communicating between services in Docker Swarm is relatively easy when you are aware of the service relationships during orchestration. However, when dealing with runtime service discovery and cross-cluster service discovery, Kubernetes provides more options.

Deployment

Docker Swarm deployments are managed using the Docker utility via the Stack commands. A Stack is a collection of services that form a deployment unit in Docker Swarm.

A Stack deployment is described using the docker-compose specification in YAML files. These files define the services to be deployed within a stack, their source repositories and their relationships to one another. The YAML files also detail the overlay network configurations and which services to assign to them, allowing for compartmentalization and security. A collection of services can be deployed using a single docker-compose.yaml file and additional files are often created to modify values for other deployments, such as testing, staging and production.

Kubernetes deployments are also described using YAML, but can be described using JSON if preferred. The deployment specification is extensive, but minimally requires definitions for Pods and Services. Deployments in Kubernetes are typically much more verbose than those of Docker Swarm, due to the increased configurability.

Both Docker Swarm and Kubernetes provide a means to apply rolling updates and for rolling back those updates as required. If an update deployment fails in Docker Swarm, the update is automatically rolled back to the previous version. Docker Swarm provides no endpoints to query an update status. A failed Kubernetes update will result in both the failed Pods and the original Pods existing side by side. A status endpoint does exist in Kubernetes. Once identified, rollbacks need to be requested explicitly.

As Kubernetes provides a means to select Pods, Services and other assets in a deployment using labels and annotations, running staging containers alongside production ones in the wild is easier in Kubernetes. This way, DevOps can roll out a single unit and test how it works in the production environment before committing to a cluster wide update. Performing the same feat in Docker Swarm is not so easy and is not typically done for this reason.

Docker Swarm deployments are considered much more user-friendly than Kubernetes. A misconception of Docker Swarm is that it cannot support large volumes of nodes and containers. It has been proven to support one thousand nodes and tens of thousands of containers in a single cluster. However, it is important to note that Docker Swarm does have a task limit. Kubernetes, on the other hand, has been documented as supporting ten times this number. With a recommended maximum of seven Managers in a cluster, Docker Swarm might not be sufficient for everyone’s needs, but this should only matter in the very largest of projects.

Performance

Comparing Docker Swarm and Kubernetes for performance is interesting, as both platforms utilise Docker containers and, for the most part, only manage the orchestration aspect of applications. Therefore, applications deployed with Docker Swarm and Kubernetes will operate with similar speed and efficiency.

When analysing orchestration performance, it has been documented that Docker Swarm will deploy and start containers as much as five times more quickly than Kubernetes when under heavy load, mainly due to the fewer moving parts within Docker Swarm and because of its tight integration with the Docker Engine.

This same investigation also highlights Docker Swarms ability to respond to API calls much more quickly. While the responsiveness differences are negligible, the investigation does show that Docker Swarm performs without degradation at up to around 90% load, while Kubernetes degrades once it reaches only around 50% load.

In a minimal Kubernetes installation, several containers are deployed to each Manager node, including:

- Etcd manager Pod
- API server Pod
- Kube Controller Pod
- KubeDNS Pod
- A network Pod, such as [Flannel](https://github.com/coreos/flannel) (optional)
- Kube Proxy Pod
- Scheduler Pod

On the worker nodes, Kubernetes will install the Kube Proxy and Kubelet Pods.

Each of these Pods use resources from the machine they are deployed to. In comparison, Docker Swarm doesn’t require the deployment of any containers as the Swarm functionality is built into the Docker service itself.

For most web applications, this should not pose a problem. The resources used by the default Kubernetes Pods is minimal and doesn’t perform much of an impact on the running application. However, this may indeed pose a problem for resource restricted environments, or environments that run on low power, such as Internet-of-Things (IoT) projects.

By Lee Sylvester