Skip to main content

The missing guide to deploying docker containers on Apache Mesos and Marathon

What is Apache Mesos?

It is an open source cluster management software. In plain english, it lets you connect a bunch of machines (called Slaves) and allows transparent resource allocation on them.

What is Marathon?

Mesos gets real shit done through its frameworks. Marathon is one such framework. This guide uses Marathon to run docker containers on Mesos Slaves.

What will this guide achieve?

By the end of this guide, you will be able to run, start, stop, scale docker containers on top of mesos slaves through a Restful API (that marathon provides). This guide does not (yet) abstract the containers behind static address bindings, so once new containers come online, the services needing them will have to be manually updated. In the next guides I will talk about how you can automate that using ha-proxy, consul, consul-template and registrator.

For simplicity, we will use a single machine to deploy everything. However that is not a production-grade setup. A production grade setup should have atleast three masters (with zookeeper running) and atleast 3 slaves (or more depending on your workload). But it is only a minor change running slaves on multiple boxes and connecting them to the masters, so can be taken up later.

Let's get started.

Step 1 : Install Mesos and Marathon

$ sudo apt-key adv --keyserver --recv E56151BF
$ DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
$ CODENAME=$(lsb_release -cs)
$ echo "deb${DISTRO} ${CODENAME} main" | \
sudo tee /etc/apt/sources.list.d/mesosphere.list
$ sudo apt-get update
$ sudo apt-get install mesos marathon

After this, point your browser to to review mesos UI and for marathon UI. 
Verify service status : 
$ systemctl status mesos-master

Step 2 : Configure docker and marathon

$ sudo -s
$ apt-get -y install
$ echo 'docker,mesos' > /etc/mesos-slave/containerizers
$ echo '10mins' > /etc/mesos-slave/executor_registration_timeout
$ service mesos-slave restart

Step 3 : Run containers

At this point, our slaves are ready to run docker container.  Lets use the marathon rest API to start new containers.
Save the following contents in a file app.json

"id": "bridged-webapp", "cmd": "python3 -m http.server 8080", "cpus": 0.5, "mem": 64.0, "instances": 2, "container": { "type": "DOCKER", "docker": { "image": "python:3", "network": "BRIDGE", "portMappings": [ { "containerPort": 8080, "hostPort": 0, "servicePort": 9000, "protocol": "tcp" }, { "containerPort": 161, "hostPort": 0, "protocol": "udp"} ] } }, "healthChecks": [ { "protocol": "HTTP", "portIndex": 0, "path": "/", "gracePeriodSeconds": 5, "intervalSeconds": 20, "maxConsecutiveFailures": 3 } ] }

$ curl -X POST -d @app.json -H "Content-type: application/json"

This will download the image since it does not exist on the slave yet. Give it some time and then visit Marathon UI at 
You can also do $ docker ps and check the port assigned to this new container. In my case, it was>8080/tcp so pointing the browser to should load the directory browser. 

That's all folks. 


Popular posts from this blog

How the Python import system works

How the Python import system works From: If you ask me to name the most misunderstood aspect of Python, I will answer without a second thought: the Python import system. Just remember how many times you used relative imports and got something like  ImportError: attempted relative import with no known parent package ; or tried to figure out how to structure a project so that all the imports work correctly; or hacked  sys.path  when you couldn't find a better solution. Every Python programmer experienced something like this, and popular StackOverflow questions, such us  Importing files from different folder  (1822 votes),  Relative imports in Python 3  (1064 votes) and  Relative imports for the billionth time  (993 votes), are a good indicator of that. The Python import system doesn't just seem complicated – it is complicated. So even though the  documentation  is really good, it d

On working remote

The last company I worked for, did have an office space, but the code was all on Github, infra on AWS, we tracked issues over Asana and more or less each person had at least one project they could call "their own" (I had a bunch of them ;-)). This worked pretty well. And it gave me a feeling that working remote would not be very different from this. So when we started working on our own startup, we started with working from our homes. It looked great at first. I could now spend more time with Mom and could work at leisure. However, it is not as good as it looks like. At times it just feels you are busy without business, that you had been working, yet didn't achieve much. If you are evaluating working from home and are not sure of how to start, or you already do (then please review and add your views in comments) and feel like you were better off in the office, do read on. Remote work is great. But a physical office is better. So if you can, find yourself a co-working s

Todo lists are overrated

My tasks come from a variety of sources: 1) Tasks from emails  2) Meeting notes with details of people who participated  3) Project related tasks that can have a long format and can be tagged/ delegated  4) Scratchpad for unrefined ideas  5) Detailed documentation for completed technical tasks / ideas  6) FIFO list of high priority small daily tasks No one app has been able to map all the requirements above, and I have tried a lot of them! In my lifetime I’ve tried a dozen todo apps. In the beginning they all seem different, novel and special. Slick UI, shortcuts, tags, subtasks, the list goes on and on. But all our stories were the same: I start using the new app, then after awhile I stop using it. Up until the last week I thought the problem was in myself (you probably think so too). After all, David Allen seems to have figured this shit out. Also there are people leaving long 5 star reviews on every major todo list app, they discuss them on forums, recommend them to friends. But the