2014-11-07

Docker Networking Ecosystem

One of the things that Clocker is designed to accomplish is to make multi-host Docker networking as easy as using single-host Docker. The way we do this currently is by using the Weave project. Weave is very interesting, because it is an entirely user-space software defined network, and is incredibly simple. Because Clocker is trying to make Docker Cloud applications easy to use and orchestrate, we found Weave a great match for us, with just the right feature-set to allow applications with more complex networking requirements to run in a Clocker provisioned environment without modification. Weave fulfils the essential requirements of a sort of Minimum Viable Network for Docker, and hence Clocker, but still leaving room for other projects to provide segmentation, external endpoints and gateways, and deeper integration.

For example Riak uses Erlang and requires epmd (the Erlang port mapper daemon) for clustering, and this uses up to a thousand TCP ports which are controlled by epmd and used simply for inter-node communication. It makes no sense to use the Docker port mapping facilities to expose these ports externally, and there is no reason for any application outside the Riak cluster to have to access them. All a Riak cluster needs to expose is a single port for web client access, and for admin console access. Weave allows Clocker to set up the Riak nodes in containers, each of which is attached to a private LAN, and exposes the Docker port forwarded web client and admin ports only. This was demonstrated at RICON recently, the Running Riak in a Docker Cloud using Apache Brooklyn talk showed a demonstration of a multi-node Riak cluster running in Clocker.

Weave is, in essence, a software ethernet switch. As mentioned earlier, it is entirely user-space, which is very important for ease of use in the cloud. Clocker, and the underlying Brooklyn control plane, is cloud-agnostic, due to our use of the Apache jclouds library. This means we try to design application blueprints in such a way that they will run anywhere. More complex SDN solutions will require drivers loaded into the kernel, and specific images used for particular operating system versions, which requires customising the configuration of each virtual machine on each cloud provider that a blueprint is required to run on. Weave allows us to ignore this and simply start the weave router in a container on each Docker host, Clocker then assigns IP addresses from the link-local address space range to each container it provisions, attaching each one to the same LAN. This is a flat network architecture, and in many ways is not suitable for production use, particularly with multi-tenant deployments where separation of traffic between applications is a concern. A simple type of access control is possible with Clocker and Weave, in the form of iptables firewall rules, since Clocker controls the IP address space used by each application, it can erect firewall rules preventing traffic from crossing application boundaries. So, this simplicity and portability has allowed Clocker to quickly demonstrate the feasibility of multi-host Docker deployments in the cloud.

But, in the future, Clocker users will want to deploy more complex applications with networking requirements that are not achievable with the Weave network model. To make this transition as seamless as possible, we need a way of supporting more traditional SDN services, such as OVS. There has been a lot of discussion in the Docker community about ways of achieving this, and how to ensure that the chosen solution fits The Docker Way and maintains the flexibility and openness that has been a hallmark of Docker from the start. The proposals in issues #8951 and #8997 discuss adding a new networking API which will allow plugins for various different SDN solutions to be integrated with Docker. In particular, issue #8997 seems to offer the flexibility and choice that Clocker would need, allowing a simple out-of-box experience with Weave, leading to more complex production deployments with OVS or other SDN solutions, when the environment is under the complete control of the blueprint.

The Docker ecosystem is an amazing collection of projects, Clocker and Weave are just a small part of this. One of the features of this ecosystem is the ability to experiment and innovate quickly - many projects, mine included, are permanently in beta, and new features are constantly being added. I hope that the networking and orchestration APIs being added to Docker will maintain this policy and continue to allow small projects to fill their specialised niches in this space.

Further information on Clocker can be found at the main Clocker site or on GitHub.