Home

Awesome

Docker & ZeroMQ

Update 2019-08-17

Simplified sample code:

Overview

Example repository to demonstrate how you can turn Python scripts into micro-services in Docker containers, which can communicate over ZeroMQ. The examples here can be run as just Python-Python, Docker-Docker (docker-compose) or Docker-Python (docker run).

Examples are using a Publisher-Subscriber pattern to communicate. This means that the publisher micro-service just send messages out to a port, without knowing who is listening and a subscriber micro-service receiving data, without knowing where the data comes from.

With ZeroMQ, only 1 micro-service can socket.bind(url) to 1 address. However, you can have unlimited micro-services socket.connect(url) to an address. This means that you can either have many-pub to 1-sub (examples in this Git repo) or 1-pub to many-sub on 1 ip:port combination.

Install Docker

1. Python-Python

  1. Open a terminal and navigate to folder pyzmq-docker/sub
  2. Execute python main.py
  3. Open a terminal and navigate to folder pyzmq-docker/pub
  4. Execute python main.py
  5. See subscriber receiving messages from the publisher!

Notes:

2. Docker-Docker with docker-compose

  1. Open a terminal and navigate to folder pyzmq-docker
  2. Executedocker-compose up --build
  3. See a Dockerized subscriber receiving messages from a Dockerized publisher! (That's really everything? 0.o)

Notes:

3. Docker-Python with docker run

Notes:

3a. pub-Docker, sub-Python

  1. Open a terminal and navigate to folder pyzmq-docker/sub
  2. Execute python main.py
  3. Open file pub/Dockerfile and change "yo.ur.i.p" to your machine IP (something similar to: "192.168.99.1")
  4. Open a terminal and navigate to folder pyzmq-docker/pub
  5. Execute docker build . -t foo/pub
  6. Execute docker run -it foo/pub
  7. See that your subscriber receives messages from your Dockerized publisher.

Notes:

3b. pub-Python, sub-Docker

  1. Open a terminal and navigate to folder pyzmq-docker/sub
  2. Execute docker build . -t foo/sub
  3. Execute docker run -p 5551:5551 -it foo/sub (maps port of Docker container to localhost)
  4. Open a terminal and navigate to folder pyzmq-docker/pub
  5. Execute python main.py
  6. See that your Dockerized subscriber receives messages from your publisher.

Notes:

Other

Inspiration

Stackoverflow question: https://stackoverflow.com/questions/53802691/pyzmq-dockerized-pub-sub-sub-wont-receive-messages

Useful Docker commands

sudo usermod -a -G docker $USER  # add current user to group docker on Linux systems (Ubuntu)

docker build . -t foo/sub  # build docker image
docker run -it foo/sub  # run build docker image and enter interactive mode
docker run -p 5551:5551 -it foo/sub  # same as above with mapping Docker port to host
docker run -p 5551:5551 --name foo-sub -it foo/sub  # same as above with naming container
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' pyzmq-docker_sub_1  # get ip of container
docker rm foo-sub  # remove container by name

docker-compose up  # run docker-compose.yml
docker-compose build / docker-compose up --build  # rebuild images in docker-compose.yml

docker image ls  # show docker images
docker container ls  # show docker containers
docker exec -it pyzmq-docker_pub_1 bash  # enter bash in container
docker attach pyzmq-docker_sub_1  # get

To detach the tty without exiting the shell, use the escape sequence Ctrl+p + Ctrl+q

docker rm $(docker ps -a -q)  # Delete all containers
docker rmi $(docker images -q)  # Delete all images

Debug docker-machine IP not found (probably not necessary)

Docker machine working check:

Debug attempts: