Containers are the word of the day in enterprise computing right now, promising a new method of deploying applications and attracting the support of cloud providers and software vendors such as Google, Microsoft, Red Hat, IBM and Rackspace. Docker is the open source technology, and eponymous software company, leading the container charge. Docker Founder and CTO Solomon Hykes came on this week’s Structure Show podcast to talk about his company’s newfound success and the challenges it still faces.
Here are some highlights from the interview, but if you’re into the past, present and future of managing distributed applications, you’ll want to listen to the whole thing. Hykes does a good job explaining how Docker works, how it came to be and why it has made the design and product decisions it has.
Back to the future
Cloud computing watchers might remember dotCloud, the platform-as-a-service startup Hykes launched in 2010 that attracted more than $10 million in venture capital, as well as a lot of good press and both the judges’ and people’s choice awards at our Structure conference in 2011. Well, that wasn’t always the plan.
“The irony in all this is, our big break, our big focus was 'Hey, there's this really awesome technology, it's called containers. It's very young and immature' — this was in 2008 — 'and so we're going to use it to build this new automation layer and it's going to be open source,’” Hykes explained. “This was the original dotCloud, and it basically bombed.”
Solomon Hykes at the Structure Launchpad in 2011.
For a variety of reasons, PaaS turned out to not be such a great business for dotCloud, but it did buy the company enough time for the world to come back around on the company’s original plan, which has since been renamed Docker. Looking back, Hykes has some perspective on how the PaaS market has evolved and where Docker fits into it:
“I think the meaning of PaaS is definitely being redefined — for the better, I think. I think it draws a clearer line between very different types of companies. Either you're providing some sort of core fundamental piece of infrastructure — you're at the 'operating system' layer … of this new architecture — or you're in a vertical and you have a unique perspective and value-add for a specific group of customers or developers. … There's a thousand ways to slice the market.”
Don’t rush for ubiquity
Hykes doesn’t buy into the notion that containers are only good for certain types of applications, instead suggesting that containers really can take over the world in due time:
“Depending on who you ask, not everyone agrees on what's suitable for a container or not. For example, a lot of people actually run very stateful, long-lived payloads that could definitely qualify as traditional. At the same time … there are very obvious examples of applications you cannot run in a Linux container, starting with applications that don't run on Linux. …
“Then there's an area where technically it could run in Docker, but it's also a question of perception. Containers are a relatively new technology, and there's always a long tail of adoption. So there are organizations out there that could deploy to containers today but will not for a while because they need to make sure it's safe. … I think that's fine.”
Hykes today. Source: Docker
Mesos is great, but …
Gigaom has covered Apache Mesos on many occasions — including on a recent podcast — and explained its potential as a resource-management layer that can help applications scale with ease and move relatively seamlessly among different environments and clouds. Hykes agrees Mesos is a cool technology — and a good framework on top of which to run Docker applications — but he also thinks a focus on scale is putting the cart before the horse:
“One of the key design philosophies that drove Docker was looking at Mesos and other tools like that, and saying, 'That's a good thing to aim for.' But it always seemed to me that these mega-orchestration tools were just kind of trying to solve the next problem before solving the immediate problem. … Those tools have been around for a while, several years, and they are not mainstream. Pick a random developer; very low probability that they're using a tool like that. …
“Mesos is the most popular one, and it didn't really take off because your typical developer does not need orchestration yet — because there is nothing to orchestrate. There's no, kind of, reliable Lego brick. It's like I send you the set for the Death Star in Lego bricks before the Lego brick was invented. … And by the way, we made the exact same mistake with dotCloud."
The Mesosphere approach to running atop Mesos.
Hykes thinks it’s best to start with the Lego brick and build out from there.
“With Docker, we changed focus radically and we said, 'Let's start with the Lego brick,'” he explained. “I think that led to a lot of misunderstandings, because a Lego brick is something that is very small. And if you look at it you say, 'That's just a piece of plastic, that's very small, that doesn't scale,' but actually it's the exact opposite. It's because it's so small and precise that actually it lets you build structures that scale a thousand times better than what you could scale with an end-to-end, fully monolithic stack.”
A diagram and explanation of Docker from the company’s website.
Google: Inspiration, debtor and potential undoing
Hykes acknowledged that every developer currently developing web applications that run across multiple servers owes a debt of gratitude to companies such as Yahoo and Google, which helped bridge the gap between scientific and mainstream computing. However, he added, some aspects of Google’s approach have never caught on with the mainstream, and it’s only the bottom-up adoption of Docker that’s now giving Google an opportunity to capitalize on them.
“I think Google is excited that Docker has made containers popular, because internally they use the same low-level technology for their own stuff, but that model has never really been adopted by the rest of the world,” Hykes said. “Putting myself … in the shoes of a large enterprise platform player, the more the rest of the world's developers think and reason about their architecture in a way that's compatible with how Google does things internally, the better it is for the future of Google.”
Google is now a force to be reckoned with in cloud computing. (c) Janko Roettgers/Gigaom
But despite a close relationship between Docker and Google — including around the Kubernetes project that has also attracted other Docker backers such as Microsoft, IBM, Red Hat and Mesosphere — Hykes, who calls himself the “benevolent dictator” of Docker, is careful to ensure no one company gets too much power in the community or too upset with how things are being run.
“It's all about finding that balance where there's just enough standardization for that competition to be constructive, and keeping incentives aligned so that at any given time — even the largest guy in the room, even the Google or Amazon or IBM — has more to gain by not forking than by forking,” Hykes said.
He added, “It's like we've booby trapped our position in the market, we've booby trapped our own headquarters, so that if we end up not living up to the principles, not delivering on the promises to the community, there is no way we can survive that, basically. … We'll be forked in a heartbeat if we start pulling shenanigans.”
Related research and analysis from Gigaom Research:
Subscriber content. Sign up for a free trial.
