Why we created an orchestration tool for you

One question I have had to answer a few times as the tech lead of ConductR, and I think it is a healthy question, is on why did Lightbend create ConductR? This post is my personal attempt to describe the rationale for it two years ago, and why I think it is more relevant than ever.

Back then we wanted a tool that was focused on making the deployment and management of applications and services built on Java, Scala, Akka and Play as easy as it could be. We wanted ConductR to be to operations what Play is to web application developers; a "batteries included" approach to deploying and managing reactive applications and services.

Two years ago, there really wasn't anything else out there that we felt offered such a packaged approach to solving these new use-cases for operations people. The sentiment was that we had done a reasonable job with the Reactive Manifesto at that point, and that we'd definitely engaged developers, but we were quickly going to arrive at a situation where operational people were going to find it a challenge to manage these new distributed applications and services. We also wanted something that had the reactive DNA.

That's how it all started. So, what's changed, and why is ConductR relevant now?

There are a number of players emerging in the orchestration space presently. This certainly validates our being a player in this space from a needs perspective. If you're happy to roll your own orchestration (which actually remains what we're up against in terms of competition, and this hasn't changed much in two years), then be prepared to have two people spend at least year tackling a problem that is harder than you think, and then realise that you have an operational cost in maintaining it. Atop of this, there's the risk to your company regarding those individuals leaving... is it sufficiently documented in terms of others taking over? Nobody has won in the orchestration space, but there's enough to choose from that will trump the business risk to your company of rolling your own. My advice here having been involved in designing and writing an orchestration tool (twice) is to not roll your own and focus on your core-business.

While I personally think that the operational productivity culture that permeates through our design is still the single most important reason to consider ConductR, here are some other reasons:

• a means to manage configuration distinctly from your packaged artifact;
• consolidated logging across many nodes;
• a supervisory system whereby if your service(s) terminate unexpectedly then they are automatically restarted;
• the ability to scale up and down with ease and with speed;
• handling of network failures, in particular those that can lead to a split brain scenario;
• automated seed node discovery when requiring more than one instance of your service so that they may share a cluster;
• the ability to perform rolling updates of your services;
• support for your services being monitored across a cluster; and
• the ability to test your services locally prior to them being deployed.

Furthermore ConductR is the complete manifestation of the entire stack of technologies that we at Lightbend both contribute to and support. It is a great example of an Akka based distributed application that uses in particular, akka-cluster, akka-distributed-data and akka-streams/http. It is also tightly integrated with our Akka monitoring based instrumentation, and the monitoring story around events, tracing and metrics is going to get stronger. If you like our stack, you should feel good about the way ConductR has been put together.

We have programmed ConductR in the spirit of the Reactive Manifesto, with resiliency and elasticity being a particular focus. There is no single point of failure and our ability to scale out is holding up.

One last point: we use ConductR for our own production environment at Lightbend hosting our websites and sales/marketing services. With any product out there, you should always look for this trait. If a supplier is not dependent on their own technologies in terms of running their core business then beware; they can lose enthusiasm very quickly.

ConductR is as relevant as it ever was, and with its batteries-included approach for operations, I'm sure it'll become even more relevant as the industry moves toward deploying and managing microservices.

One last tidbit: ConductR is becoming a framework for Mesos/DCOS. Exciting times!

Thanks for reading this far!