Vagrant, Docker, Puppet, Chef

Not exactly.

Chef/Puppet are the "same", they are configuration management. While you can use them to manage virtual machines or public/private clouds, most people don't tend to use them that way. They are configuration management. They typically come into play after a virtual machine is fired up to get them in a desired state. That is to say, what software is needed on the virtual machine, what users need to be added, what configuration is needed, etc. Thus, it tends to be used for scaling infrastructure.

Vagrant, while also can be used to manage virtual machines and public/private clouds, is usually only used for one off environments. It provides a cohesive file for creating a virtual machine. It is similar to chef/puppet that way, but doesn't tend to be used at scale.

Docker is a separate beast. It has several components, but primarily it is used for "bundling" (Note: it does much more than that, but that is an ELI5 answer) software and requires a host system (or infrastructure) to run on. It adds a little security to applications but mostly provides a consistent "OS" for an application to run on.

In practice, all of these can be utilized in an environment. Here is an example:

Say you have application FunTime. You have eight developers who contribute to this, and FunTime is designed to be ran on a scale-able infrastructure on AWS. It is designed to have a front-end (FunTime-Front) and a back-end (FunTime-API), and requires postgres. 4 developers work on the front end, four developers work on the backend.

I would do the following (there are many ways to skin this cat, but this is one example):

I would use Docker for FunTime-Front and FunTime-API. I would use Vagrant to set up a dev environment for the developers (so that they can tweak various components). Vagrant would: start up the VM locally (or on a cloud if needed), Install docker, pull down the docker images for FunTime-Front and FunTime-API, install postgres, and populate postgres with dummy data, configure network ports to the various components.

Now the developer has the full FunTime stack on their local machine and doesn't have to screw around with configuring anything themselves: they can just type "vagrant up".

On the infrastructure side, I would use chef (or puppet) to configure the Environments: Production, Stage, and Development (or whatever is needed), then chef would install docker on the "application" servers, "postgres" on the postgres servers, apply security settings, etc. In this way all the related servers are the same. If I needed to update a server or add a patch, it would be trivial with configuration management.

In all cases Docker would be used so that there is no application difference between environments, including the developers work station.

This would make sure that you don't hear the excuse "Well, it works on my local machine!" very often. In addition, if there is a bungled deployment, rolling back the application would be VERY easy with Docker.

I hope that provides a little more insight into how they could be used.

I will attempt to pitch as in daily talk,

Vagrant - For making development environments spin up on new developer's machine in same project, ideally within few minutes. Usually used on top of virtualbox but can used with different machine providers.

Docker - Nothing virtual here, simply imagine a real process and required libraries are sandboxed, then shipped to the server as an archive. Resource usage is decided upon run command.

Chef/Puppet - Can be used to automate anything you type in bash to make your project setup (except application keys etc). You can use them to build docker images or vagrant environments, so they don't necessarily have to exist on physical production server. See Packer.

AWS/Rackspace/Azure - are IaaS providers. In plain english, they spin up servers as you demand and give you the ssh access. Of course, they provide more than that.

Guess what, they can be used within one another.

Those tools appeared to make deployment easier, but as a rule of thumb, don't use them all together unless it will save time for you and your team. Sometimes, it is way easier to spin up an EC2 by hand and scp your project with manual configuration. You can automate later.

Premature optimization is the root of all evil - Sir Tony Hoare

Chef : Chef is an automation platform to transform your infrastructure into code. It is generally called, Configuration Management Software. You can define the state with different parameters as your config files, s/w, tools, access types and resource types, etc. Also, you can configure different machines with different functionality based on your need.

Puppet : Puppet is a tool that allows you to abstract specific concepts of the target machine and make the configuration process more operating system agnostic. It allows to install a package or run a background service on startup independently. Define a command, packages those need to installed, step dependencies, file content, and other things you would need to get machine up and running without problems.

Vagrant : Vagrant is a project that helps the spawning of virtual machines. It started as an command line of VirtualBox, something similar to Gemfile for VM's. You can choose the base image to start with, network, IP, share folders and put it all in a file that anyone can reuse to spawn the same configured machine. Vagrant has different extensions, provisioning options and VM providers. You can run a VirtualBox, VMware and it is extensible enough to be able to create instances on EC2.

Docker : Docker, allows to package an application with all of its dependencies into a standardized unit of software development. So, it reduces a friction between developer, QA and testing. It dynamically change your application, adding new capabilities every single day, scaling out services to quickly changing the problem areas. Docker is putting itself in an excited place as the interface to PaaS be it networking, discovery and service discovery with applications not having to care about underlying infrastructure. Yes, their are still issues with docker in production(thought all major companies already migrated their infrastructure to docker), but, hopefully, we'll see the solutions to those problems, as docker team and contributors working hard on those issues. As Docker Volume driver allows third-party container data management solutions to provide data volumes for containers which operate on data, such as database, key-value stores, and other stateful applications. As you can see one, rexray, as volume plugin and provides advanced storage functionality. emccode/rexray We're finally starting to agree on more than just images and runtime.