Home

Awesome

mesos-module-flocker

This is a proof of concept. It is not recommended for production systems.

Motivation

Enable storage managed by Flocker to be consumed by tasks launched on mesos slave nodes.

Architecture

Architecture

Design Decisions

Integration with Flocker Control Service

Flocker Isolator integrates with Flocker Control Service (as opposed to Docker Flocker Plugin, diverging from DVDI design ) for the following reasons:

Building

We have provided a dockerfile that helps you build the project. Note that the dockerfile uses ubuntu 14.04. We have only tested on an ubuntu 14.04 server. Other linux distributions will differ.

To build and test the project, follow the steps below:

Mac (assumes your docker-machine is called dev and you have eval'ed for docker)

git clone https://github.com/ClusterHQ/mesos-module-flocker.git
cd mesos-module-flocker/docker/mesos-modules-dev
docker build -t containersol/mesos-modules-dev:14.04 .
cd ../..
rm -rf ./build ./bin 
docker-machine ssh dev 'rm -rf /tmp/build/*' 
docker-machine scp -r /Volumes/source/clusterhq/mesos-module-flocker/. dev:/tmp/build 
docker run -it --env MESOS_ROOT=/mesos -v /tmp/build:/build containersol/mesos-modules-dev:14.04 sh -c 'cd /build ; cmake . ; make ; ./build/test_flocker_isolator' 
docker-machine scp -r dev:/tmp/build/bin /Volumes/source/clusterhq/mesos-module-flocker

Linux

git clone https://github.com/ClusterHQ/mesos-module-flocker.git
cd mesos-module-flocker/docker/mesos-modules-dev
sudo docker build -t containersol/mesos-modules-dev:14.04 .
cd ../..
rm -rf ./build ./bin 
mkdir build
docker run -it --env MESOS_ROOT=/mesos -v ./build:/build containersol/mesos-modules-dev:14.04 sh -c 'cd /build ; cmake . ; make ; ./build/test_flocker_isolator'

Testing

To run the tests, first build the project then run the application ./build/test_flocker_isolator. This will run through the gtests in the test folder.

Installing on Mesos

We have also provided a set of terraform/bash scripts to install a mesos cluster with a working flocker configuration. If you need help creating a cluster, definitely check this out: https://github.com/philwinder/mesos-flocker-terraform

Copy the built bin/libisolator.so file to all slaves in your mesos cluster. Next, create a file named modules.json which has the settings for your cluster:

 {
   "libraries": [
     {
       "file": "/home/ubuntu/libisolator.so",
       "modules": [
         {
           "name": "com_clusterhq_flocker_FlockerIsolator",
           "parameters": [
             {
               "key": "ipaddress",
               "value": "$(YOUR_MASTER_HOSTNAME_OR_IP)"
             },
             {
               "key": "port",
               "value": "4523"
             }
           ]
         }
       ]
     }
   ]
 }

Now enable the module on the slave with:

printf /home/ubuntu/modules.json | sudo tee /etc/mesos-slave/modules
printf com_clusterhq_flocker_FlockerIsolator | sudo tee /etc/mesos-slave/isolation
sudo service mesos-slave restart

Running your stateful application

To use flocker as a backend, you must pass several environmental variables along with your application. Note that this does not work with the Mesos Docker containerizer, since it is not implemented. See: https://issues.apache.org/jira/browse/MESOS-2840. If you want to run a docker container, run it from the cmd (see example).

    "FLOCKER_CONTAINER_VOLUME_PATH": The path that the stateful application will write to
    "FLOCKER_ID": A unique id representing the dataset of this webapp

For example:

{
  "id":"flocker-webapp-demo",
  "cpus": 0.5,
  "mem": 128,
  "instances": 1,
  "cmd": "sudo docker run --net=bridge -p 8500:80 -v /tmp/data:/data binocarlos/moby-counter:localfile",
  "env": {
    "FLOCKER_CONTAINER_VOLUME_PATH": "/tmp/data",
    "FLOCKER_ID": "webapp-data"
  },
  "ports":[
      8500
    ]
}

Known issues

Whats Next

Flocker Resource Provider

Flocker Mesos Isolator allows Flocker volume lifecycle management in a reactive fashion to task start/stop workflows.

In addition to creating/deleting volumes through Mesos Slave, Flocker can also advertize storage capabilities to Mesos Master, to be presented to Frameworks. Storage capabilities include:

Available resources:

Used resources:

Presenting these rich cluster-wide storage data to Mesos Master (and eventually Frameworks) will enable frameworks to make intelligent decisions regarding initial placement of applications.

screen shot

ETA on Resource Provider availability is January 2016.

Acknowledgements

Many thanks to Adam Bordelon and Michael Park at Mesosphere for productive discussions around the design!

This project was developed by Container Solutions and sponsored by ClusterHQ.