Home

Awesome

docker Scrutinizer Code Quality Upload Frontend pychat.org Refresh backend pychat.org

Live demo: pychat.org, video

Table of contents

About

Pychat is an opensource absolutely free communication tool targeted for a company use. It's created as alternative to Slack/Discord. See the table below to understand its key features.

When should I use pychat

PychatSlackSkypeTelegramViberDiscord
Open Source+-----
Free++/-+/-++/-+/-
Screen sharing++-+-+
Stream drawing+-----
Syntax highlight+----+
Only company users++---+
Audio/Video conference++++-+
Can run on your server+-----
Audio/Video messages+--++-
P2P file sharing+-----
P2P messaging+--+--
Message read status+-+++-
Tagging user++-+++
Message threads++----
PWA (works w/o lan)+-----
Desktop client+/-++++/-+
Mobile client+/-+++++
3rd-party plugins-+---+

I would personally use discord or slack as a company chat. They are built and maintained by thousands of people rather than a single person. BUT wait!!! There're some key factors of picking pychat over others:

  1. Being opensource. If you need to add some custom tool or feature, you will never able to do this with any other messanger. Slack and discord provides plugins but they are still limited.
  2. Being absolutely free. You don't need to pay anything to use or setup pychat at all. You can host pychat on low-end hardware like Raspberry Pi which costs under 50$ and will easily handle thousands of active users. Slack and Discord will charge you for the set of features you need now or WILL need in the future. Telegram/Skype/Viber and etc are not corporate chats and they lack a lot of features and there're people all over the world which could accidentaly be invited to your group.
  3. Security. All of the chats above are SAAS solutions, but not Pychat! Remember wHen you chose any messangers, all of your communication is stored on external hard drive which is always less secure. Some messangers like viber or whatsapp backup do not store messages but rather backup history to your google driver. But that often leads to holes in history and broken search. Also only pychat features p2p file sharing. Do you still use messangers to echange ssh keys or any other secure files? Never store them on the server! Only with pychat you can send file directly to another person ommiting persisting it on the server.
  4. You just feel enthusiastic for bleeding-edge opensource projects.

How to host pychat

Notice: pychat is migrating from vue2 to vue3 and this change has been released to master. The older code that supports some feature (electron/cordova) is still not migrated and located at branch vue2-webpack

Run test docker image

Please don't use this build for production, as it uses debug ssl certificate, lacks a few features and all files are located inside of container, meaning you will lose all data on container destroy.

docker run -tp 443:443 deathangel908/pychat-test

Run prod docker image

Please run each step very carefully. Do not skip editing files, reading comments or any instructions. This may lead to bugs in the future.

openssl req -nodes -new -x509 -keyout server.key -out certificate.crt -days 3650
wget https://raw.githubusercontent.com/akoidan/pychat/master/backend/chat/settings_example.py
wget https://raw.githubusercontent.com/akoidan/pychat/master/docker/pychat.org/production.json
wget https://raw.githubusercontent.com/akoidan/pychat/master/docker/pychat.org/turnserver.conf

docker volume create pychat_data
containerid=`docker container create --name dummy -v pychat_data:/data hello-world`
docker cp settings_example.py dummy:/data/settings.py
docker cp production.json dummy:/data/production.json
docker cp turnserver.conf dummy:/data/turnserver.conf
docker cp certificate.crt dummy:/data/certificate.crt
docker cp server.key dummy:/data/server.key
docker rm dummy

This volume will contain all production data: config, mysql data, redis and etc. If you need to edit files inside container you can use:

docker run -i -t -v pychat_data:/tmp -it alpine /bin/sh

docker run -t -v pychat_data:/data -p 443:443 -p 3478:3478 deathangel908/pychat

Native setup

If you don't or unable to run docker you can alway do the setup w/o it. You definitely spend more time, so I would recommend to use docker if possible. But if you're still sure, here's the setup for cent-os/archlinux based system:

  1. For production I would recommend to clone repository to /srv/http/pychat. If you want to close the project into a different directory, replace all absolute paths in config files. You can use download_content.sh rename_root_directory to do that.
  2. Install packages:
    • For archlinux follow Install OS packages, add add these ones: pacman -S postfix gcc jansson.
    • For centos use add alias yum="python2 $(which yum)" to /etc/bashrc if you use python3. And then install that packages yum install python34u, python34u-pip, redis, mysql-server, mysql-devel, postfix, mailx
    • If you use another OS, try to figure out from Install OS packages guide which things you need
  3. If you want to use native file-uploader (nginx_upload_module written in C) instead of python uploader (which is a lot slower) you should build nginx yourself. For archlinux setup requires pacman -S python-lxml gd make geoip. To build nginx with this module run from the root user: bash download_content.sh build_nginx 1.15.3 2.3.0. And create dir + user useradd nginx; install -d -m 0500 -o http -g http /var/cache/nginx/. If you don't, just install nginx with your package manager: e.g. pacman -S nginx or yum install nginx on centos
  4. Follow Bootstrap files flow.
  5. I preconfigued native setup for domain pychat.org, you want to replace all occurrences of pychat.org in rootfs directory for your domain. To simplify replacing use my script: ./download_content.sh rename_domain your.new.domain.com. Also check rootfs/etc/nginx/sites-enabled/pychat.conf if server_name section is correct after renaming.
  6. HTTPS is required for webrtc calls so you need to enable ssl:
  1. Open /etc/nginx/sites-enabled/pychat.conf and modify it by:
  1. Change to parent directory (which contains frontend and backend) and Copy config files to rootfs with from root user sh download_content.sh copy_root_fs.
  2. Create a directory mkdir backend/downloading_photos in the backend directory and give it access chmod 777 downloading_photos cDon't forget to change the owner of current (project) directory to http user: chown -R http:http. And reload systemd config systemctl daemon-reload. Also you
  3. Follow the Frontend steps
  4. Generate postfix files: install -d -m 0555 -o postfix -g postfix /etc/postfix/virtual; postmap /etc/postfix/virtual; newaliases; touch /etc/postfix/virtual-regexp; echo 'root postmaster' > /etc/aliases
  5. Start services:
  1. You can also enable autostart (after reboot)
  1. Open in browser https://your.domain.com. Note that by default nginx accepts request by domain.name rather than ip.
  2. If something doesn't work you want to check logs:

Frontend

Desktop app

Pychat uses websql and built the way so it renders everything possible w/o network. You have 3 options:

PWA

This is the simplest one. Just open settings page from you user and click "Add to home screen". Note that PWA is only available from chrome and chrome android. No support for IOS and other browsers. But PWA is the most stable from ones below.

Natifier

Use nativifier to create a client (replace pychat.org for your server): npx run nativifier pychat.org

Electron

Android app

You can use PWA as it's described in desktop app section which I recommend. Other way is cordova which is a lot harder. If you're not familiar with android SDK I would recommend doing the steps below from AndroidStudio:

Example for mac:

  1. Download oracle jdk-8
  2. Install android studio
  3. Run android studio that will install Android sdk for you
  4. Accept licence with ~/Library/Android/sdk/tools/bin/sdkmanager --licenses
  5. Install gradle. brew install gradle
  6. Open frontend/platforms/android with androidStudio
  7. Start android emulator / connect device
  8. put index.html into www
  9. Run dev server with yarn start; bash download_content.sh android
  10. TO debug java files you can run it directory from android studio. Debug button should be available out of the box after openning a project
  11. To debug js you can open chrome://inspect/#devices in chrome
  12. For any question check cordova docs

Development setup

The flow is the following

Install OS packages

This section depends on the OS you use. I tested full install on Windows/Ubuntu/CentOs/MacOS/Archlinux/Archlinux(rpi3 armv7). pychat.org currently runs on Archlinux Raspberry Pi 3.

Windows:

  1. Install python with pip. only Python 3.6+ is required
  2. Add pip and python to PATH variable.
  3. Install redis. Get the newest version or at least 2.8.
  4. Install mysql. You basically need mysql server and python connector.
  5. You also need to install python's mysqlclient. If you want to compile one yourself you need to vs2015 tools. You can download visual-studio and install Common Tools for Visual C++ 2015. You need to run setup as administrator. The only connector can be found here. The wheel (already compiled) connectors can be also found here Mysqlclient. Use pip to install them.
  6. Add bash commands to PATH variable. Cygwin or git's will do find.(for example if you use only git PATH=C:\Program Files\Git\usr\bin;C:\Program Files\Git\bin).
  7. Install nvm.
  8. From backend dir (cd backend). Create virtualEnv python3 -m venv .venv. Activate python virtual environment: source .venv/bin/activate

Ubuntu:

  1. Install required packages: apt-get install python pip mysql-server libmysqlclient-dev (python should be 3.6-3.8) If pip is missing check python-pip. For old versions of Ubuntu you can use this ppa: sudo add-apt-repository ppa:deadsnakes/ppa; sudo apt-get update; sudo apt-get install python3.8 python3.8-dev python3.8-venv python3.8-apt; curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py; python3.8 get-pip.py
  2. Install redis database: add-apt-repository -y ppa:rwky/redis; apt-get install -y redis-server
  3. Install nvm
  4. From backend dir (cd backend). Create virtualEnv python3 -m venv .venv. Activate python virtual environment: source .venv/bin/activate

Archlinux:

  1. Install system packages: pacman -S unzip python python-pip redis yarn mariadb python-mysqlclient. nvm and pyenv is located in aur so yay -S nvm pyenv-virtualenv (or use another aur package)
  2. If you just installed mariadb you need to initialize it: mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql.
  3. cd backend; pyenv virtualenv 3.8-dev pychat. Activate it source ~/.pyenv/versions/pychat/bin/activate

MacOS

  1. Install packages: brew install mysql redis python3
  2. Start services brew services run mysql redis
  3. From backend dir (cd backend). Create virtualEnv python3 -m venv .venv. Activate it: source .venv/bin/activate

Ssl

Since we're using self singed certificate your OS doesn't know about for development. We need to do some tricks for browser to make it work. If you have valid certificates for your domain you can skip this step.

  1. I used the following commands to generate a new self signed certificate. You can use mine located in frontend/build/certs directory. So you can skip this text
cd frontend/build/certs
openssl genrsa -out private.key.pem 4096
openssl req -new -sha256 -out root.ca.pem -key private.key.pem -subj '/CN=localhost' -extensions EXT -config <( printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")
openssl x509 -req -days 3650 -in root.ca.pem -signkey private.key.pem -out server.crt.pem -extfile ./v3.ext

Useful links:

  1. You have multiple options:

Bootstrap files:

  1. I use 2 git repos in 2 project directory. So you probably need to rename excludeMAINfile to .gitignoreor create link to exclude. ln -rsf .excludeMAIN .git/info/exclude
  2. Rename backend/chat/settings_example.py to backend/chat/settings.py. Modify file according to the comments in it.
  3. Install python packages with pip install -r requirements.txt. (Remember you're still in backend dir)
  4. From root (sudo) user create the database (from shell environment): echo "create database pychat CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci; CREATE USER 'pychat'@'localhost' identified by 'pypass'; GRANT ALL ON pychat.* TO 'pychat'@'localhost';" | mysql -u root. You will need mysql running for that (e.g. systemctl start mysql on archlinux) If you also need remote access do the same with '192.168.1.0/255.255.255.0';
  5. Fill database with tables: bash ../download_content.sh create_django_tables. (Remember you're still in backend dir)

Build frontend

Change to frontend directory cd frontend I would recommend to use node version specified in nvm, so nvm install; nvm use.

Configure IDEs if you use it:

Pycharm

  1. I recommend open backend as root directory for pycharm.
  2. Django support should be enabled by default when you open this project. If it doesn't happen go to Settings -> Languages and Framework -> Django -> Enable django support.
  1. If pycharm didn't configure virtualenv itself. Go to Settings -> Project backend -> Project Interpreter -> Cogs in right top -> 'Add' -> Virtual Environment -> Existing environment -> Interpereter = pychatdir/.venv/bin/python (or on Archlinux .pyenv/versions/pychat/bin/python). Click ok. In previous menu on top 'Project interpreter` select the interpriter you just added.
  2. Settings -> Project backend -> Project structure
  1. Add tornado script: Run -> Edit configuration -> Django server -> Checkbox Custom run command start_tornado. Remove port value.

Linting

Current linting supports:

Webstorm

Set template

  1. New
  2. Edit files templates...
  3. Vue single file component
<template>
    <div>#[[$END$]]#</div>
</template>

<script lang="ts">
  import {State} from '@/utils/store';
  import {Component, Prop, Vue, Watch, Ref} from 'vue-property-decorator';

  @Component
  export default class ${COMPONENT_NAME} extends Vue {
   
  }
</script>
<style lang="sass" scoped>

</style>

Change linting settings

Disable tslint, since it's not used, and enable eslint:

  1. Settings

  2. Typescript

  3. Tslint

  4. Disable tslint

Start services and run:

Contribution guide

Description

Pychat is written in Python and typescript. For handling realtime messages WebSockets are used: browser support on client part and asynchronous framework Tornado on server part. For ORM django was used with MySql backend. Messages are being broadcast by means of redis pub/sub feature using tornado-redis backend. Redis is also used as django session backend and for storing current users online. For video call WebRTC technology was used with stun server to make a connection, which means you will always get the lowest ping and the best possible connection channel. Client part is written with progressive js framework VueJs which means that pychat is SPA, so even if user navigates across different pages websocket connection doesn't break. Pychat also supports OAuth2 login standard via FaceBook/Google. Css is compiled from sass. Server side can be run on any platform Windows, Linux, Mac. Client (users) can use Pychat from any browser with websocket support: IE11, Edge, Chrome, Firefox, Android, Opera, Safari...

Shell helper

Execute bash download_content.sh it will show you help.

Frontend logging

By default each user has turned off browser (console) logs. You can turn them on in /#/profile page (logs checkbox). All logs are logged with window.logger object, for ex: window.logger('message')(). Note that logger returns a function which is binded to params, that kind of binding shows corrent lines in browser, especially it's handy when all source comes w/o libraries or other things that transpiles or overhead it. You can also inspect ws messages here for chromium. You can play with window.wsHandler.handleMessage(object) and window.wsHandler.handle(string) methods in debug with messages from log to see what's going on

Icons

Chat uses fontello and its api for icons. The decision is based on requirements for different icons that come from different fonts and ability to add custom assets. Thus the fonts should be generated (.wolf etc). W/o this chat would need to download a lot of different fonts which would slow down the loading process. You can easily edit fonts via your browser, just execute bash download_content.sh post_fontello_conf. Make your changes and hit "Save session". Then execute bash download_content.sh download_fontello. If you did everything right new icons should appear under frontend/src/assets/demo.html

Sustaining online protocol

Server pings clients every PING_INTERVAL miliseconds. If client doesn't respond with pong in PING_CLOSE_JS_DELAY, server closes the connection. If ther're multiple tornado processes if can specify port for main process with MAIN_TORNADO_PROCESS_PORT. In turn the client expects to be pinged by the server, if client doesn't receive ping event it will close the connection as well. As well page has window listens for focus and sends ping event when it receives it, this is handy for situation when pc suspends from ram.

Database migrations

Pychat uses standard django migrations tools. So if you updated your branch from my repository and database has changed you need to ./manage.py makemigration and ./manage.py migrate. If automatic migration didn't work I also store migrations in migration. So you might take a look if required migration is there before executing commands. If you found required migration in my repo don't forget to change Migration.dependencies[] and rename the file.

Screen sharing for Chrome v71 or less

ScreenShare available for Chrome starting from v71. For chrome v31+ you should install an extension. It uses chrome.desktopCapture feature that is available only via extension. The extension folder is located under screen_cast_extension`. If you want to locally test it:

Tp publish extension:

WebRTC connection establishment

The successful connection produces logs below in console

Sender:

ws:in {"action": "offerCall", "content": {"browser": "Chrome 86"}, "userId": 2, "handler": "webrtc", "connId": "YZnbgKIL", "opponentWsId": "0002:UFBW", "roomId": 1, "time": 1604446797449}
WRTC Setting call status to  received_offer    
WRTC CallHandler initialized
ws:out  {"action":"replyCall","connId":"YZnbgKIL","content":{"browser":"Chrome 86"},"messageId":1} 
rsok33GN CallHandler initialized
rsok33GN:0005:EJAd Created CallSenderPeerConnection
WRTC Setting call status to  accepted
WRTC capturing input
WRTC navigator.mediaDevices.getUserMedia({audio, video})
ws:out  {"action":"acceptCall","connId":"YZnbgKIL","messageId":2}
YZnbgKIL:0002:UFBW Connect to remote  
rsok33GN:0005:EJAd Creating RTCPeerConnection
YZnbgKIL:0002:UFBW Sending local stream to remote
rsok33GN:0005:EJAd Creating offer...
rsok33GN:0005:EJAd Created offer, setting local description
rsok33GN:0005:EJAd Sending offer to remote
YZnbgKIL:0002:UFBW onicecandidate
...
YZnbgKIL:0002:UFBW onicecandidate
rsok33GN:0005:EJAd onsendRtcData
rsok33GN:0005:EJAd answer received
rsok33GN:0005:EJAd onaddstream
rsok33GN:0005:EJAd onsendRtcData

Receiver:

WRTC capturing input
WRTC navigator.mediaDevices.getUserMedia({audio, video})
WRTC got local stream  MediaStream {id: "0IeyYT9LxHRidUZaw7XSVnXEPWYimm4KDmJB", active: true, onaddtrack: null, onremovetrack: null, onactive: null, …}
WRTC Setting call status to  sent_offer
ws:out  {"action":"offerCall","roomId":1,"content":{"browser":"Chrome 86"},"messageId":1}
ws:in {"action": "setConnectionId", "handler": "void", "connId": "YZnbgKIL", "messageId": 1, "time": 1604446797449}   
rsok33GN CallHandler initialized
rsok33GN:0004:oIc5 Created CallReceiverPeerConnection
YZnbgKIL:0001:qobF Connect to remote
rsok33GN:0004:oIc5 Creating RTCPeerConnection
YZnbgKIL:0001:qobF Sending local stream to remote
rsok33GN:0004:oIc5 onsendRtcData
rsok33GN:0004:oIc5 Creating answer
rsok33GN:0004:oIc5 onaddstream
rsok33GN:0004:oIc5 Sending answer
rsok33GN:0004:oIc5 onsendRtcData
rsok33GN:0004:oIc5 onsendRtcData

The string rsok33GN:0005:EJAd describes:

TO see current connections and their info check chrome://webrtc-internals/ Read this article to understand how JSEP architecture works. SEE WEBRTC_CONFIG at development.json. I personally use turn server coturn, It needs ports 3478 to be exposed.

Frontend Stack

The technologies stack used in project:

builder.js is used to build project. Take a look at it to understand how source files are being processed. Its start point is entry: ['./src/main.ts']. Everything is imported in this files are being processed by section loaders.

Every vue component has injected .$logger object, to log something to console use this.logger.log('Hello {}', {1:'world'})(); Note calling function again in the end. Logger is disabled for production. For more info visit lines-logger

This project uses vue-property-decorator (that's has a dependency vue-class-component) vuex-module-decorators. You should write your component as the following:

import { Vue, Component, Prop, Watch, Emit, Ref } from 'vue-property-decorator'
import {userModule, State} from '@/utils/storeHolder'; // vuex module example


@Component
export class MyComp extends Vue {
  
  @Ref
  button: HTMLInputElement;

  @Prop readonly propA!: number;
  
  @State
  public readonly users!: User[];

  @Watch('child')
  onChildChanged(val: string, oldVal: string) { }

  @Emit() 
  changedProps() {}

  async created() {
    userModule.setUsers(await this.$api.getUsers());
  }
}

Frontend config

development.json and production.json have the following format:

{
  "BACKEND_ADDRESS": "e.g. pychat.org:443, protocol shouldn't be there, note there's no trailing slash, you can specify '{}' to use the same host as files served with",
  "IS_DEBUG": "if true, build won't be uglifies, logs will be set to trace, window object will be added with useful data and etc",
  "GOOGLE_OAUTH_2_CLIENT_ID" : "check chat/settings_example.py",
  "FACEBOOK_APP_ID": "check chat/settings_example.py",
  "RECAPTCHA_PUBLIC_KEY": "check chat/settings_example.py RECAPTCHA_SITE_KEY",
  "AUTO_REGISTRATION": "if set to true, for non loggined user registration page will be skipped with loggining with random generated username. Don't use RECAPTCHA with this key",
  "PUBLIC_PATH": "Set this path if you have different domains/IPs for index.html and other static assets, e.g. I serve index.html directly from my server and all sttatic assets like main.js from CDN, so in my case it's 'https://static.pychat.org/' note ending slash",
  "ISSUES": "if true navigation bar will display link to reporting a issue page",
  "GIPHY_API_KEY": "Api keys that is used to fetch gifs from https://giphy.com/. Be aware, this key is gonna be exposed to frontend. So anyone can steal it. To get those sign up in https://developers.giphy.com/, create a new app and replaced with its key.",
  "GITHUB_LINK": "an external link to project source files, in my case https://github.com/Deathangel908/pychat . Set to false if you don't wanna see it in the navbar",
  "FLAGS": "if true, a user name will contain a country icon on the right. User names are shown on the right section of the screen",
  "WEBRTC_CONFIG": "This variable defines the first argument of RtcPeerConnection constructor. Sometimes webrtc stun server doesn't work in establishing a connection. Especially for this you can use turn server instead of it. Docker prod docker image already comes with a turn server, example of configuration for it  `{iceServers:[{urls:['turn:YOUR_DOMAIN'],username:'pychat',credential:'pypass'}]}`. replace YOUR_DOMAIN with your real domain name/public ip. You other scenarios use your server like coturn (https://github.com/coturn/coturn). See more info of this variable at docs: https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/RTCPeerConnection#RTCConfiguration_dictionary"
}

Github actions

In order to setup continuous delivery via github:

Cmnd_Alias RESTART_TORNADO = /usr/bin/systemctl restart tornado
http ALL=(ALL) NOPASSWD: RESTART_TORNADO

TODO