Home

Awesome

SuttaCentral server and client repository

SuttaCentral is a Python Flask server which serves a Progressive Web App (client) and its associated JSON API (server).

The API pulls its data in real time from an ArangoDB instance populated periodically with data from the sc-data repository.

Deploying for production

  1. $ git clone git@github.com:suttacentral/suttacentral.git
  2. $ cd suttacentral
  3. $ git checkout production
  4. $ make prepare-host
  5. $ make run-production-env -> Supply needed env variables, if you chose random you will be prompted back with generated values. Remember them! You will use some of them to access admin services.
  6. Done

Or updating in individual steps

  1. cd /opt/suttacentral
  2. git pull
  3. make generate-env-variables -> Supply needed env variables (only if env has been changed)
  4. make run-prod-no-logs -> run docker containers
  5. make delete-database -> OPTIONAL: Skip this if data hasn't seriously changed (do it, if texts have been deleted or renamed)
  6. make migrate -> only needed for delete-database, but harmless to run anyway
  7. make load-data -> load data into arangodb
  8. make index-arangosearch -> index arangosearch
  9. make reload-uwsgi -> make sure flask server is not serving cached stale data

Minimally disruptive update

If no containers need to be rebuilt then this is all that needs to be run:

  1. cd /opt/suttacentral
  2. git pull
  3. make load-data
  4. make reload-uwsgi
  5. make index-arangosearch
  6. make rebuild-frontend

Changing the branch(s) the server, or staging server, uses

  1. cd /opt/suttacentral
  2. git checkout <code-branch>
  3. cd server/sc-data
  4. git checkout <data-branch>

Then run the commands for updating, probably including the make delete-database step.

Development

1. Server

1.0 Setting up the project and first run

  1. Install docker and docker-compose.
  2. Clone the repo git clone git@github.com:suttacentral/suttacentral.git.
  3. Cd into the repo cd suttacentral.
  4. run make prepare-host in order to make some small adjustment on the host machine.
  5. run make run-preview-env - Build images, load data, index-arangosearch and more. This will run the project for the first time.

1.1 Running the project

  1. run make run-dev.

1.2 Loading the data

When changes are made on bilara-data and sc-data, they do not automatically get updated in suttacentral. During initial setup in step 1.0.4 above, the raw data from those repositories is brought into suttacentral and added to the database. So if you make changes in bilara-data and sc-data you must run the steps below to see them in the build.

  1. ensure server is up and run make load-data.
  2. To index arangosearch run make index-arangosearch.

1.3 Docs

API documentation is available at suttacentral.net/api/docs.

Swagger documentation is generated from doc strings in api methods. The docstring should use OpenAPI specification 2.0 yaml format. This yaml docstring will be interpreted as OpenAPI's Operation Object.

Development

In this mode server, nignx, client dirs are mounted in Docker's containers so that any local changes take place in the container as well.

In addition Uwsgi+Flask expose port 5001 on local host, arangodb port 8529.

1.4 Makefile

There is a Makefile with following commands:

In addition, the following rebuilds the front end.

1.5 Working with ArangoDB

Our project is using ArangoDB on the back-end. In the development mode it exposes port 8529 on the localhost. You can access its web interface on http://127.0.0.1:8529.

In the code that is running in the docker containers you can access the database on the address sc-arangodb on the same port.

In the development mode:

Login: root

password: test

In order to change password you have to change ARANGO_ROOT_PASSWORD in env's .env fiel eg. If you want to change it in development env you have to edit .dev.env file.

1.6 Nginx proxy

Our project is using nginx as a HTTP reverse proxy. It is responsible for serving static files and passing /api/* endpoints to the uwsgi+flask server.

1.7 Flask + uWSGI

Flask is hidden behind uWSGI. uWsgi communicate with nignx with unix socket. The socket file (uwsgi.sock) is in socket-volume shared between nginx and flask+uwsgi

Creating db migrations

In order to create database migration in out app you have to follow those simple steps:

  1. in server/server/migrations/migrations folder create file with name <migration_name>_<id of the last migration + 1>.py.
  2. Add this line at the top of the file: from ._base import Migration.
  3. Create class that inherits from Migration class.
  4. Set migration_id class attribute to match the file name.
  5. create some tasks. Each task should be separate method accepting only self as a parameter.
  6. Set tasks = ['first_task', 'second_task', ...] in class attributes.
  7. You are good to go just remember to never change the 'migration_id'. otherwise your migrations might fail.

For example:

from common.arangodb import get_db
from migrations.base import Migration


class InitialMigration(Migration):
    migration_id = 'initial_migration_001'
    tasks = ['create_collections']

    def create_collections(self):
        """
        Creates collections of suttas and collection of edges between them.
        """
        db = get_db()
        graph = db.create_graph('suttas_graph')
        suttas = graph.create_vertex_collection('suttas')
        parallels = graph.create_edge_definition(
            name='parallels',
            from_collections=['suttas'],
            to_collections=['suttas']
        )

Flask manage tasks

  1. python manage.py migrate - Run migrations.
  2. python manage.py list_routes - Lists all available routes/URLs.

1.10 Style guidelines

2. Client

2.1 Style guidelines

General considerations: