Home

Awesome

Simple REST service for Apache Aurora

Note: work in progress

RESTful server exposing API to interact with the Apache Aurora scheduler.

You may also want to look at this Review Request #23741 which enables REST calls to the service points of the Aurora scheduler that were available only through the Thrift protocol.

Purpose

Aurora includes a powerful command-line tool that is used to send all sorts of query and management commands to the scheduler. Users can start, stop and monitor jobs that are controlled by the Aurora Scheduler.

However this command-line tool requires users to build and install the Aurora client libraries and binaries on their machine, or to login to server that has everything already set up.

The REST service provides remote access to the Aurora Scheduler without prerequisites. Any REST client, or the curl command, can be used to call the REST API.

Implementation

This is a brief description, for more detailed presentation look at the design document.

This REST service is "simple" because it is just a wrapper around the functionality provided by the Aurora client library (the same one that is used by the Aurora client tool):

Build Instructions

Ideally this repo would be a standalone source tree that can be downloaded, built and installed. That is not possible because the Aurora client libraries are not packaged and published to a public repository. For that reason the build requires the presense of the Apache Aurora source code.

The workaround that is applied here is to downlad the that code, then stick in the code of this server inside at some location, and then build it as if it was part of the Apache Aurora source code tree.

It is a sort of hack, but it is an experiment and project to learn. Git submodules may be useful here to implement the same "hack", it is in the TODO list.

Follow the steps below to build the RESTful server.

Step 1: Apache Aurora source code

Use git-clone or curl to bring in the Apache Aurora source code.

$ git clone git@github.com:apache/incubator-aurora.git
$ cd incubator-aurora
$ ./pants
Bootstrapping pants @ 0.0.24
+ VIRTUALENV_VERSION=1.11.6
+ which python2.7
...
Cleaning up...
Pants 0.0.24 https://pypi.python.org/pypi/pantsbuild.pants/0.0.24

Usage:
  ./pants goal [option ...] [goal ...] [target...]  Attempt the specified goals.
  ./pants goal help                                 Get help.
  ./pants goal help [goal]                          Get help for the specified goal.
  ./pants goal goals                                List all installed goals.
...

Make sure the pants build tool managed to bootstrap itself and finished with success.

Step 2: Simple RESTful server source code

Do the same for this repository.

$ pushd src/main/python/apache/aurora/
$ git clone git@github.com:misho-kr/mesos-aurora-restful.git
$ mv mesos-aurora-restful rest
$ popd

Step 3: Execute the build

Execute the following command which will produce executable file dist/aurora_rest.pex

$ ./pants build src/main/python/apache/aurora/rest/bin:
Build operating on top level addresses: ...
...
Wrote dist/aurora_rest.pex
$ dist/aurora_rest.pex --help 
Usage: dist/aurora_rest.pex [OPTIONS]
...

Start the REST server

The command below will start the Aurora REST service. The paramters that are passed to the executable will have the following effect:

$ dist/aurora_rest.pex --port=8888 --executor=internal --application=thread --parallel=4

These parameters and their effects are explained in the design document.

REST API

GET /alpha/jobs/{cluster}/{role}

HTTP/1.1 200 OK
Content-Type: application/json
Server: TornadoServer/3.2.1

{
    "count": 3,
    "jobs": {
        "1": "paas-aurora/mkrastev/devel/rhel59_world2",
        "2": "paas-aurora/mkrastev/devel/kraken_app",
        "3": "paas-aurora/mkrastev/devel/rhel59_world"
    },
    "key": "paas-aurora/mkrastev",
    "status": "success"
}

PUT /alpha/job/{cluster}/{role}/{environment}/{jobname}

$ curl -s -X PUT --data-binary @rhel59_world2.aurora \
  "http://localhost:8888/alpha/jobs/paas-aurora/mkrastev/devel/rhel59_world2" | \
  python -m json.tool

Response:

HTTP/1.1 201 Created
Content-Type: application/json
Server: TornadoServer/3.2.1

{
    "count": 1,
    "job": "paas-aurora/mkrastev/devel/rhel59_world2",
    "key": "paas-aurora/mkrastev/devel/rhel59_world2",
    "status": "success"
}

PUT /alpha/job/{cluster}/{role}/{environment}/{jobname}/update?shards={X}

$ curl -s -X PUT --data-binary @rhel59_world2.aurora \
  "http://localhost:8888/alpha/jobs/paas-aurora/mkrastev/devel/rhel59_world2?shards=1" | \
  python -m json.tool

Response:

HTTP/1.1 202 Accepted
Content-Type: application/json
Server: TornadoServer/3.2.1

{
    "count": 1,
    "job": "paas-aurora/mkrastev/devel/rhel59_world2",
    "key": "paas-aurora/mkrastev/devel/rhel59_world2",
    "status": "success"
}

DELETE /alpha/job/{cluster}/{role}/{environment}/{jobname}/update

HTTP/1.1 202 Accepted
Content-Type: application/json
Server: TornadoServer/3.2.1

{
    "count": 1,
    "job": "paas-aurora/mkrastev/devel/rhel59_world2",
    "key": "paas-aurora/mkrastev/devel/rhel59_world2",
    "status": "success"
}

PUT /alpha/job/{cluster}/{role}/{environment}/{jobname}/restart?shards={X}

HTTP/1.1 202 Accepted
Content-Type: application/json
Server: TornadoServer/3.2.1

{
    "count": 1,
    "job": "paas-aurora/mkrastev/devel/rhel59_world2",
    "key": "paas-aurora/mkrastev/devel/rhel59_world2",
    "status": "success"
}

DELETE /alpha/job/{cluster}/{role}/{environment}/{jobname}?shards={X}

$ curl -s -X \
  DELETE "http://localhost:8888/alpha/jobs/paas-aurora/mkrastev/devel/rhel59_world2?shards=2" | \
  python -m json.tool

Response:

HTTP/1.1 200 OK
Content-Type: application/json
Server: TornadoServer/3.2.1

{
    "count": 1,
    "job": "paas-aurora/mkrastev/devel/rhel59_world2",
    "key": "paas-aurora/mkrastev/devel/rhel59_world2",
    "status": "success"
}
```

#### `GET` /alpha/version

```
HTTP/1.1 200 OK
Content-Type: application/json
Server: TornadoServer/3.2.1
```
```json
{
    "status": "success",
    "version": "0.1"
}
```