Awesome

Introduction

The reMarkable tablets provide a very good experience for digital writing/drawing. Unfortunately, sharing drawing in real-time is difficult, especially on Linux, where the official reMarkable cloud streaming does not work.

Since the reMarkable uses an e-paper display, it's possible to extract high quality damage tracking information from the software responsible for driving the display. This repository uses that information and libvncserver to implement a simple well-performing VNC server. This allows streaming the display contents off the device to any VNC/RFB client. Performance is quite good, at least when using the ZRLE compression built into libvncserver; over USB connections, drawing is quite smooth, with negligible latency. Large framebuffer updates, such as opening or closing a notebook, are, in fact, frequently displayed on the client before the reMarkable screen has finished refreshing. The code has been less well tested over WiFi connections; drawing over an SSH tunnel over a WiFi connection seems to occasionally stutter due to unpredictable latency, although unencrypted connections still seem to perform well.

Dependent projects provide the damage-tracking information that is required. For the rM1, the required information is extracted from the kernel, using mxc_epdc_fb_damage. For the rM2, the epaper display is driven from userspace software, rather than from the integrated e-paper display controller on the processor (which the rM1 used), and so damage information is unfortunately not available outside of the process driving the display. Currently, all known deployed drivers for the display (most noticeably xochitl, the official interface on the rM), use a proprietary library libqsgepaper.a to draw to the display. The rM2 version of the server therefore uses libqsgepaper-snoop, which injects a damage-reporting stub into processes that use libqsgepaper.a to control the rM2 framebuffer.

Input support

The VNC server also supports injecting input from the VNC clients into the reMarkable's input devices, using rM-input-devices. Cursor movements with button 3 active are sent to the tablet as touch events. Cursor movements with button 1 active are sent as wacom stylus contacts at full pressure (allowing rudimentary drawing). Cursor movements with button 2 active are sent as wacom stylus movement-within-proximity-but-without-contact events. Keyboard keys left/up/page up are sent as a left key, which moves back a page in the current notebook; right/down/page down similarly are sent as a right key to move forward a page. Home and escape are sent as the home key, which generally returns to the home page of xochitl.

Cursor location reporting

If a VNC client supports the cursor position pseudo-encoding, the position of the wacom stylus is sent to it. Unfortunately, sending pen status such as in/out-of-proximity/contact events is not easy to do in the current protocol; a protocol extension may be warranted if clients would find such information useful.

Building

The supported way to build this is via the Nix package manager, through the nix-remarkable expressions. To build just this project via nix build from this repo, download it into the pkgs/ directory of nix-remarkable.

There are four variants that can be built: for either rM1 or rM2, and in a "standalone" configuration or not. The standalone configuration will statically link against all libraries not installed on the reMarkable by default, and will bundle necessary kernel modules and tools; it is recommended unless you have a particular need for sharing. By default, the Nix expressions will build all variants.

For other systems, the commands needed to compile and link are given in the Makefile. Build dependencies depend on the configuraiton being built, but all versions require libvncserver (built for the reMarkable) and the reMarkable kernel headers as well as the rM-input-devices.h from rM-input-devices.

For the reMarkable 1, the mxc_epdc_fb_damage.h file from mxc_epdc_fb_damage is also required; for the reMarkable 2, the libqsgepaper-snoop.h file from libqsgepaper-snoop is needed instead.

For all versions, the -standalone versions require the standalone static library versions of the relevant dependencies---rM-input-devices for both, mxc_epdc_fb_damage for the rM1, and libqsgepaper-snoop for the rM2.

Prebuilt binaries are available in the Releases tab.

Usage

Copy the relevant executable to the device and run it; this will start a vnc server listening on port 5900. Any VNC client should be able to provide a (view-only) view of the tablet's screen when pointed at the reMarkable's IP address and standard VNC port (5900). Vinagre, Remmina, and gst-libvncclient-rfbsrc have been tested.

Note that the server will bind to all available network interfaces by default, so only run this if the tablet's WiFi is disconnected or connected to a trusted network. iptables rules could be used to restrict traffic.