Celebrating A Decade of Docker Containers

A transformative software developer tool that was created through years of work by researchers here, and is now used by organisations from NASA to Netflix, is being celebrated this month by the Association for Computing Machinery (ACM).

In March the ACM, the world's largest educational and scientific computing society, is putting the story of Docker containers – a robust solution for building, shipping, and sharing applications which is used by three in every four software developers – on the front cover of its journal, Communications of the ACM.

In the same way that shipping containers have standardised how goods are transported, Docker packages applications with everything they need to run (code, data, configuration) into portable 'containers' that work anywhere.

An article by three of the lead researchers involved, A Decade of Docker Containers, traces Docker's development from the original underlying technique they developed of 'unikernels': small, potentially transient, computer modules that increase the security, speed and scale of computer operating systems.

After spinning this research out from this Department as a business called Unikernel Systems Ltd, it was bought by Docker and has been jointly co-developed since then. The article traces the many complex technical challenges it has had to overcome. But doing so has made it exceptionally popular.

Docker is consistently at the top of Stack Overflow's community rankings as the "most desired" and "most used" developer tool. The Docker Hub, just one of several registries where images can be shared, hosts more than 14 million application images and delivers more than 11 billion image pulls per month.

Docker's popularity stems from the way it tackles a longstanding problem many developers face: how to develop and deploy microservices that are increasingly written in diverse languages.

"If we were writing software 20 years ago," explains Anil Madhavapeddy (right), Professor of Planetary Computing here and one of the lead researchers involved, "we'd have to spend a lot of time figuring out where the software comes from, how to get it all compiling, and the finally figure out how to share it and publish it for other people. Docker unified this entire workflow, so I could write a single Docker file that lets me build software from lots of different sources, run it on my computer or on the cloud, and finally publish it socially as well for other people.

Writing software to solve real problems
"By pulling all of these things into one workflow mechanism," he adds, "we unlock the ability for people to share open source software across lots of different languages and to really get on with the productive business of writing software to solve real problems." 

Little wonder, then, that since its first release in 2013, Docker has seen rapid adoption in diverse sectors. It's estimated that almost all the companies in the Fortune 500 index use Docker. One such is Netflix, which uses Docker containers in its streaming services.

The tool is also used by NASA. Former PhD student here David J Scott, who is now Principal Software Engineer at Docker Inc, heard a NASA speaker at a conference talking about the agency's DART project. "This was designed to nudge asteroids out of position if they were heading towards the Earth. Docker helped them package up their simulation software so they could easily share it with other developers on the project." 

The impact of AI on software development
But the article doesn't just dwell on the past. It looks to current and future challenges too as the way software is written undergoes fundamental change.

"Security is crucial," says Anil. "So Docker is adding support for confidential computing, for hardware enclaves, such that it can seamlessly partition the bits that we want to keep private away from the public data."

It's also adapting to the impact of AI on the way that software is developed. "Docker is adding support for seamless local workflows so that you can use AI coding agents. They can generate huge amounts of code, but you can keep it secure, sandboxed, and under control, so you can decide what to share with the wider world."

And as Anil's personal passion is environmental science, biodiversity and climate change. "So I'm also adding in support for planetary computing to Docker," he says. "We'd like to make it easier to build a planetary Wikipedia for everyone to know what's going on in the world around them."

• A Decade of Docker Containers is by Anil Madhavapeddy, David J. Scott, and Justin Cormack and appears in the March issue of Communications of the ACM.