Home

Awesome

serial-port-json-server

Version 1.96

Binaries for Download

You can now always check the Releases page on Github for the latest binaries. https://github.com/chilipeppr/serial-port-json-server/releases

Summary

A serial port JSON websocket & web server that runs from the command line on Windows, Mac, Linux, Raspberry Pi, or Beagle Bone that lets you communicate with your serial port from a web application. This enables web apps to be written that can communicate with your local serial device such as an Arduino, CNC controller, or any device that communicates over the serial port. Since version 1.82 you can now also program your Arduino by uploading a hex file.

The app is written in Go. It has an embedded web server and websocket server. The server runs on the standard port of localhost:8989. You can connect to it locally with your browser to interact by visiting http://localhost:8989. The websocket is technically running at ws://localhost/ws. You can of course connect to your websocket from any other computer to bind in remotely. For example, just connect to ws://192.168.1.10/ws if you are on a remote host where 192.168.1.10 is your devices actual IP address.

The app is one executable with everything you need and is available ready-to-go for every major platform. It is a multi-threaded app that uses all of the cool techniques available in Go including extensive use of channels (threads) to create a super-responsive app.

If you are a web developer and want to write a web application that connects to somebody's local or remote serial port server, then you simply need to create a websocket connection to the localhost or remote host and you will be directly interacting with that user's serial port.

For example, if you wanted to create a Gcode Sender web app to enable people to send 3D print or milling commands from your site, this would be a perfect use case. Or if you've created an oscilloscope web app that connects to an Arduino, it would be another great use case. Finally you can write web apps that interact with a user's local hardware.

Thanks go to gary.burd.info for the websocket example in Go. Thanks also go to tarm/goserial for the serial port base implementation. Thanks go to Jarret Luft at well for building the Grbl buffer and helping on global code changes to make everything better.

Front-End Javascript Client

There is a very thorough front-end Javascript client available called "widget-spjs" located at http https://github.com/chilipeppr/widget-spjs. This widget is good if you want your web page to talk to the Serial Port JSON Server (SPJS). This widget enables numerous pubsub signals via amplify.js so you can publish to SPJS and receive data back when you subscribe to the appropriate signals.

Example Use Case

Here is a screenshot of the Serial Port JSON Server being used inside the ChiliPeppr Serial Port web console app. http://chilipeppr.com/serialport <img src="http://chilipeppr.com/img/screenshots/serialportjsonserver2.png">

This is the Serial Port JSON Server being used inside the TinyG workspace in ChiliPeppr. http://chilipeppr.com/tinyg <img src="http://chilipeppr.com/img/screenshots/serialportjsonserver3.png">

There is also a JSFiddle you can fork to create your own interface to the Serial Port JSON Server for your own project. http://jsfiddle.net/chilipeppr/vetj5fvx/ <img src="http://chilipeppr.com/img/screenshots/serialportjsonserver_jsfiddle.png">

Running

From the command line issue the following command:

Verbose logging mode:

Running on alternate port:

Filter the serial port list so it has relevant ports in the list:

Garbage collect mode (deprecated):

Override the default hostname:

Here's a screenshot of a successful run on Windows x64. Make sure you allow the firewall to give access to Serial Port JSON Server or you'll wonder why it's not working. <img src="http://chilipeppr.com/img/screenshots/serialportjsonserver_running.png">

Feed Rate Override

There is a new feature available as of version 1.83 which is Feed Rate Override. It can be triggered by sending in a command like the following:

fro COM4 0.5 fro /dev/ttyUSB0 0.5

This command asks SPJS to override the existing feed rate and reduce it by half. If you have a feedrate of 200 then the command above would bring it to 100 by multiplying 200 * 0.5 = 100. To turn off the feed rate override set it back to 0 with a command such as:

fro COM4 0.0 fro /dev/ttyUSB0 0.0

To increase speed 2x

fro COM4 2

Binaries for Download

https://github.com/chilipeppr/serial-port-json-server/releases

How to Build

You do not need to build this. Binaries are available in the releases section of this Github repo at https://github.com/chilipeppr/serial-port-json-server/releases. However, if you still want to build...

Video tutorial of building SPJS on a Mac: https://www.youtube.com/watch?v=4Hou06bOuHc

  1. Install Go (http://golang.org/doc/install)
  2. If you're on a Mac, install Xcode from the Apple Store because you'll need gcc to compile the native code for a Mac. If you're on Windows, Linux, Raspberry Pi, or Beagle Bone you are all set.
  3. Get go into your path so you can run "go" from any directory: On Linux, Mac, Raspberry Pi, Beagle Bone Black export PATH=$PATH:/usr/local/go/bin On Windows, use the Environment Variables dialog by right-click My Computer
  4. Define your GOPATH variable and create the folder to match. This is your personal working folder for all yourGo code. This is important because you will be retrieving several projects from Github and Go needs to know where to download all the files and where to build the directory structure. On my Windows computer I created a folder called C:\Users\John\go and set GOPATH=C:\Users\John\go On Mac export GOPATH=/Users/john/go On Linux, Raspberry Pi, Beagle Bone Black, Intel Edison export GOPATH=/home/john/go On Windows, use the Environment Variables dialog by right-click My Computer to create GOPATH
  5. Change directory into your GOPATH
  6. Type go get github.com/chilipeppr/serial-port-json-server. This will retrieve this Github project and all dependent projects. It takes some time to run this.
  7. Then change direcory into src/github.com/chilipeppr/serial-port-json-server.
  8. Type go build when you're inside that directory and it will create a binary called serial-port-json-server
  9. Run it by typing ./serial-port-json-server or on Windows run serial-port-json-server.exe
  10. If you have a firewall on the computer running the serial-port-json-server you must allow port 8989 in the firewall.

Supported Commands

CommandExampleDescription
listLists all available serial ports on your device
open portName baudRate [bufferAlgorithm]open /dev/ttyACM0 115200 tinygOpens a serial port. The comPort should be the Name of the port inside the list response such as COM2 or /dev/ttyACM0. The baudrate should be a rate from the baudrates command or a typical baudrate such as 9600 or 115200. A bufferAlgorithm can be optionally specified such as "tinyg" (or in the future "grbl" if somebody writes it) or write your own.
sendjson {}{"P":"COM22","Data":[{"D":"!~\n","Id":"234"},{"D":"{"sr":""}\n","Id":"235"}]}See Wiki page at https://github.com/johnlauer/serial-port-json-server/wiki
send portName datasend /dev/ttyACM0 G1 X10.5 Y2 F100\nSend your data to the serial port. Remember to send a newline in your data if your serial port expects it.
sendnobuf portName datasend COM22 {"qv":0}\nSend your data and bypass the bufferFlowAlgorithm if you specified one.
close portNameclose COM1Close out your serial port
bufferalgorithmsList the available bufferAlgorithms on the server. You will get a list such as "default, tinyg"
baudratesList common baudrates such as 2400, 9600, 115200
restartRestart the serial port JSON server
exitExit the serial port JSON server
frofro COM 1.5Multiplies the current feed rate by the value passed in for the specific serial port. (This is specific to Gcode, so if using SPJS for non-Gcode work this command won't mean much.)
memstatsSend back data on the memory usage and garbage collection performance
broadcast stringbroadcast my dataSend in this command and you will get a message reflected back to all connected endpoints. This is useful for communicating with all connected clients, i.e. in a CNC scenario is a pendant wants to ask the main workspace if there are any settings it should know about. For example send in "broadcast this is my custom cmd" and get this reflected back to all connected sockets {"Cmd":"Broadcast","Msg":"this is my custom cmd\n"}
versionGet the software version of SPJS that is running
hostnameGet the hostname of the current SPJS instance
program portName core:architecture:name $path/to/filenameprogram com3 arduino:avr:uno c:\myfiles\grbl_9i.hexSend a hex file to your Arduino board to program it.
programfromurl portName core:architecture:name urlprogramfromurl /dev/ttyACM0 arduino:sam:arduino_due_x http://synthetos.github.io/g2/binaries/TinyG2_Due-edge-078.03-default.binDownload a hex/bin file from a URL and then send it to your Arduino board to program it.
cayenn-sendudpcayenn-sendudp 192.168.1.12 any-msg-to-end-of-lineSend this command into SPJS and the content after the IP address will be forwarded to the IP address you provided via UDP to port 8988 of the device. This enables IoT communication from the browser since browsers can't message with UDP directly. You can also send to the network broadcast address with UDP.
cayenn-sendtcpcayenn-sendtcp 192.168.1.12 any-msg-to-end-of-lineSend this command into SPJS and the content after the IP address will be forwarded to the IP address you provided via TCP to port 8988 of the device. This enables IoT communication from the browser since browsers can't message with TCP directly.
disablecayennAdd this command line flag to disable the loading of the Cayenn TCP/UDP server on port 8988. Defaults to on.
usblistusblistSend this command to get a list of USB devices. Currently only works on Linux ARM. Typically used to find webcams on your Raspberry Pi. (Available in version 1.91 and later)
execruntimeexecruntimeGet the runtime operating system and processor platform for the host running SPJS. Used to figure out if specific commands or features are available on the host especially when used in conjunction with the "exec" command.
execexec id:123 user:pi pass:blahUsed to execute a shell command on the host. You must specificy a user/password.
createstartupscriptRun this command line switch on Linux to have SPJS create an /etc/init.d/serial-port-json-server startup script. You must run as root for this command to work.

Exec and Execruntime

SPJS now supports the exec and execruntime commands. These were added to enable users to fully control their device running SPJS. For example, the Cam widget in ChiliPeppr uses this command to install a WebRTC server on the SPJS host so you don't have any work to do configuring your host. To solve security concerns you have to specify a username/password combination in the command. Keep in mind that this user/pass is sent over the websocket which is cleartext, so only use this feature when behind a firewall or over a VPN. You may also specify the -allowexec option (which is not on by default for security) on the command line when launching SPJS to bypass the requirement for the user/pass to be provided for each exec command.

The exec command was necessary because the growing number of devices being used to create one overall solution is increasing and the configuration and management of those devices is becoming a problem. For example, in the ChiliPeppr world users are working on a Pick and Place machine. The solution will use multiple Raspberry Pi's to host a webcam and do OpenCV machine vision processing. Configuring and aggregating those devices needs to be done by the front-end Javascript widgets and thus these commands solve that problem.

Example of asking SPJS to tell you the runtime environment of the host computer.

execruntime
{"ExecRuntimeStatus":"Done","OS":"linux","Arch":"arm","Goroot":"/home/pi/go","NumCpu":4}

Example of executing the echo command with the parameter of done.

exec id:123 user:pi pass:blah echo "done"
{"ExecStatus":"Progress","Id":"123","Cmd":"/bin/bash","Args":["echo \"done\""],"Output":"done"}
{"ExecStatus":"Done","Id":"123","Cmd":"/bin/bash","Args":["echo \"done\""],"Output":"done"}

Programming Your Arduino from SPJS

The ability to program your board is now available within Serial Port JSON Server (SPJS). This feature was developed by the folks at Arduino because they are looking to use SPJS inside their upcoming Web IDE project. Therefore you can expect great support for this feature into the future as it will be the main way the IDE programs the boards. For folks using SPJS in other environments like ChiliPeppr, this means you'll be able to do firmware updates on your boards without much effort.

There are two new commands:

program [portName] [core:architecture:name] [$path/to/filename]
programfromurl [portName] [core:architecture:name] [url]

These commands are identical except for one parameter that specifies where the binary hex/bin file is. With the program command you specify a file path. With programfromurl you specify a public URL.

This example command will update your Arduino Due with the latest TinyG G2 firmware for your CNC machine. It will download the bin file from Github and flash it to an Arduino Due running on the ttyACM0 serial port on a Raspberry Pi 2.

programfromurl /dev/ttyACM0 arduino:sam:arduino_due_x http://synthetos.github.io/g2/binaries/TinyG2_Due-edge-078.03-default.bin

This example will update your Arduino Uno running on a Windows computer with the latest version of Grbl from a public URL.

programfromurl com12 arduino:avr:uno https://raw.githubusercontent.com/grbl/grbl-builds/master/builds/grbl_v0_9i_atmega328p_16mhz_115200.hex

The 2nd parameter of core:architecture:name specifies which board you're trying to program so that SPJS can figure out what programmer and parameters should be used to send the hex/bin file up to your device. The choices can be seen in the boards.txt file in the distribution, but here is a partial list for quick reference:

<!-- Garbage collection ------- On slower devices like Raspberry Pis (not the new Raspberry Pi 2) it is evident that the slowness of the CPU can cause some issues. In particular, on a Tinyg so much data can flow back from the serial device that it can overwhelm the Raspberry Pi such that serial data is lost if the Pi can't process it quick enough. This usually isn't a problem until a garbage collection process is triggered by golang for SPJS. Garbage collection does a "stop the world" technique which on the Raspi is so slow that SPJS may be unresponsive for 5 or even 10 seconds. This is long enough that data starts spilling off the serial port buffer inside the TinyG. On faster hosts like Windows or Mac this doesn't happen. Therefore some additional tricks have been added to SPJS to try to alleviate this problem from rearing it's ugly head. SPJS by default will start in gc=std mode. This means SPJS will simply use the default garbage collection from Golang. You could instead try gc=max. This means SPJS will forcibly garbage collect non-stop on each receive and send on the serial port. This essentially doubles or triples SPJS's CPU usage, but it reduces the chance for the stopping of the world. It is recommended to keep gc=std as the default, but you could try your own settings including trying gc=off which means all garbage collection is turned off and thus you'll eventually run out of memory. You can send in a "gc" into SPJS via the websocket to force manual garbage collection in this instance. -->

Broadcast Command

There is a growing need for end-clients of SPJS to be able to chat with eachother. Therefore a new command has been added called "broadcast". It's not a very sophisticated feature because it simply regurgitates out whatever is after the broadcast command back to all connected clients. This simplistic approach means any user can implement any command they would like via the broadcast command and create unique solutions via SPJS.

For example, if a pendant controller for your CNC is connected to SPJS and trying to figure out if the ChiliPeppr main workspace has some stored settings for your pendant, it could send out a command like: broadcast get-settings

And SPJS would regurgitate the command to all connected sockets like: {"Cmd":"Broadcast","Msg":"get-settings\n"}

And if the ChiliPeppr workspace were listening for all incoming {"Cmd":"Broadcast","Msg":...} signals and specifically the "get-settings" command then it could respond with something like: broadcast settings x:1, y:10, z:4

Interesting Branches of SPJS

https://github.com/arduino/arduino-create-agent

The Arduino team is basing their new web IDE on SPJS. That's just awesome! They've definitely taken SPJS to new heights and in different directions. The two projects have branched enough that pull requests aren't clean anymore, but we can still borrow nicely from eachother with new features that either project adds.

https://github.com/benjamind/gpio-json-server/

This is a very interesting branch on this project where Ben took the basic code layout, websocket, and command structure and created a GPIO server version of this app. It's such an interesting and awesome project, it makes me want to combine his code into SPJS to make a full-blown version of serving up hardware ports via JSON and websockets--whether they're serial ports or GPIO ports. Something about that just feels right. The only downside is that no Windows or Mac machines have GPIO, so it would be a very Raspberry Pi specific feature.

FAQ

Startup Script for Linux

Here's a really lightweight /etc/init.d startup script for use on Linux like with a Raspberry Pi, Beable Bone Black, Odroid, Intel Edison, etc.

Create a text file inside /etc/init.d called serial-port-json-server, for example:

sudo nano /etc/init.d/serial-port-json-server

Then make sure the file contents contain the following script, but make sure to update the path to your serial-port-json-server binary. This example has the binary in /home/pi but yours may differ.

<pre> #! /bin/sh ### BEGIN INIT INFO # Provides: serial-port-json-server # Required-Start: $all # Required-Stop: # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Manage my cool stuff ### END INIT INFO PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin . /lib/init/vars.sh . /lib/lsb/init-functions # If you need to source some other scripts, do it here case "$1" in start) log_begin_msg "Starting Serial Port JSON Server service" # do something /home/pi/serial-port-json-server_linux_arm/serial-port-json-server -regex usb|acm & log_end_msg $? exit 0 ;; stop) log_begin_msg "Stopping the Serial Port JSON Server" # do something to kill the service or cleanup or nothing killall serial-port-json-server log_end_msg $? exit 0 ;; *) echo "Usage: /etc/init.d/serial-port-json-server {start|stop}" exit 1 ;; esac </pre>

Make your script executable

sudo chmod +x /etc/init.d/serial-port-json-server

Then you need to run the following command to setup your /etc/init.d script so it starts on boot up of your computer...

sudo update-rc.d serial-port-json-server defaults

And of course to manually start/stop the service:

<pre> sudo service serial-port-json-server stop sudo service serial-port-json-server start </pre>

Revisions

Changes in 1.95

Changes in 1.94

Changes in 1.93

Changes in 1.92

Changes in 1.91

Changes in 1.89

Changes in 1.88

Changes in 1.87

Changes in 1.88

Changes in 1.85

Changes in 1.84

Changes in 1.83

Changes in 1.82

Changes in 1.81

Changes in 1.80

Changes in 1.77

Changes in 1.76

Changes in 1.75

Changes in 1.7

Changes in 1.6

Changes in 1.5

Changes in 1.4

Changes in 1.3

Changes in 1.2