Skip to main content

The missing guide to deploying docker containers on Apache Mesos and Marathon

What is Apache Mesos?

It is an open source cluster management software. In plain english, it lets you connect a bunch of machines (called Slaves) and allows transparent resource allocation on them.

What is Marathon?

Mesos gets real shit done through its frameworks. Marathon is one such framework. This guide uses Marathon to run docker containers on Mesos Slaves.

What will this guide achieve?

By the end of this guide, you will be able to run, start, stop, scale docker containers on top of mesos slaves through a Restful API (that marathon provides). This guide does not (yet) abstract the containers behind static address bindings, so once new containers come online, the services needing them will have to be manually updated. In the next guides I will talk about how you can automate that using ha-proxy, consul, consul-template and registrator.

For simplicity, we will use a single machine to deploy everything. However that is not a production-grade setup. A production grade setup should have atleast three masters (with zookeeper running) and atleast 3 slaves (or more depending on your workload). But it is only a minor change running slaves on multiple boxes and connecting them to the masters, so can be taken up later.

Let's get started.

Step 1 : Install Mesos and Marathon

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
$ DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
$ CODENAME=$(lsb_release -cs)
$ echo "deb http://repos.mesosphere.io/${DISTRO} ${CODENAME} main" | \
sudo tee /etc/apt/sources.list.d/mesosphere.list
$ sudo apt-get update
$ sudo apt-get install mesos marathon

After this, point your browser to http://127.0.0.1:5050 to review mesos UI and http://127.0.0.1:8080 for marathon UI. 
Verify service status : 
$ systemctl status mesos-master

Step 2 : Configure docker and marathon

$ sudo -s
$ apt-get -y install docker.io
$ echo 'docker,mesos' > /etc/mesos-slave/containerizers
$ echo '10mins' > /etc/mesos-slave/executor_registration_timeout
$ service mesos-slave restart

Step 3 : Run containers

At this point, our slaves are ready to run docker container.  Lets use the marathon rest API to start new containers.
Save the following contents in a file app.json

{
"id": "bridged-webapp", "cmd": "python3 -m http.server 8080", "cpus": 0.5, "mem": 64.0, "instances": 2, "container": { "type": "DOCKER", "docker": { "image": "python:3", "network": "BRIDGE", "portMappings": [ { "containerPort": 8080, "hostPort": 0, "servicePort": 9000, "protocol": "tcp" }, { "containerPort": 161, "hostPort": 0, "protocol": "udp"} ] } }, "healthChecks": [ { "protocol": "HTTP", "portIndex": 0, "path": "/", "gracePeriodSeconds": 5, "intervalSeconds": 20, "maxConsecutiveFailures": 3 } ] }

Then
$ curl -X POST http://127.0.0.1:8080/v2/apps -d @app.json -H "Content-type: application/json"

This will download the image since it does not exist on the slave yet. Give it some time and then visit Marathon UI at http://127.0.0.1:8080. 
You can also do $ docker ps and check the port assigned to this new container. In my case, it was 0.0.0.0:31515->8080/tcp so pointing the browser to http://127.0.0.1:31515 should load the directory browser. 

That's all folks. 



Comments

Popular posts from this blog

Learn frontend to get good at backend : lessons for my 20 year old self

Only when you are the consumers of your own products, can you design the best ones! Same philosophies go for API design and programming in general.

This advice may seem counter-intuitive at first, but trust me, the best way to get better at backend programming is to spend some time doing frontend! It will teach you a lot about how your APIs are being used in the "real world". A first-hand experience of API consumption will tell you a lot about your design philosophies!

The economics of crypto investing

If you believe in the greater fool theory, there is no other market as speculative and volatile as the crypto market today. We are perhaps living in the biggest bubble of our times. I am not bullish on this market in particular. I am bullish on the mania. 90% of the cryptos we see today will crash. They are just tokens with no tangible value generation capability. However, I believe that the mania and euphoria will stay.

Having said that, should one consider investing in this market? Certainly!
The risk/reward is lovely, potential upsides and margins are huge and with 3-5% of your net worth, the bet on the mania is worth it.

How does one choose where to invest?

If you follow the stock markets, you are expected to do thorough Fundamental Analysis before investing. Expect the same for the crypto market. I invest in large caps. I invest in index funds. And I invest over and over again. Markets rise, always. Extrapolating the same strategy - invest in indices - the top 10 tokens by perfo…

Capture and compare stdout in python unit tests

A recent fan of TDD, I set out to write tests for whatever comes my way. And there was one feature where the code would print messages to the console. Now - I had tests written for the API but I could not get my head around ways to capture these messages in my unittests.
After some searching and some stroke of genius, here's how I accomplished capturing stdout.