Awesome
Introduction
The reMarkable tablets provide a very good experience for digital writing/drawing. Unfortunately, sharing drawing in real-time is difficult, especially on Linux, where the official reMarkable cloud streaming does not work.
Since the reMarkable uses an e-paper display, it's possible to extract high quality damage tracking information from the software responsible for driving the display. This repository uses that information and libvncserver to implement a simple well-performing VNC server. This allows streaming the display contents off the device to any VNC/RFB client. Performance is quite good, at least when using the ZRLE compression built into libvncserver; over USB connections, drawing is quite smooth, with negligible latency. Large framebuffer updates, such as opening or closing a notebook, are, in fact, frequently displayed on the client before the reMarkable screen has finished refreshing. The code has been less well tested over WiFi connections; drawing over an SSH tunnel over a WiFi connection seems to occasionally stutter due to unpredictable latency, although unencrypted connections still seem to perform well.
Dependent projects provide the damage-tracking information that is
required. For the rM1, the required information is extracted from the
kernel, using
mxc_epdc_fb_damage.
For the rM2, the epaper display is driven from userspace software,
rather than from the integrated e-paper display controller on the
processor (which the rM1 used), and so damage information is
unfortunately not available outside of the process driving the
display. Currently, all known deployed drivers for the display (most
noticeably xochitl
, the official interface on the rM), use a
proprietary library libqsgepaper.a
to draw to the display. The rM2
version of the server therefore uses
libqsgepaper-snoop,
which injects a damage-reporting stub into processes that use
libqsgepaper.a
to control the rM2 framebuffer.
Input support
The VNC server also supports injecting input from the VNC clients into the reMarkable's input devices, using rM-input-devices. Cursor movements with button 3 active are sent to the tablet as touch events. Cursor movements with button 1 active are sent as wacom stylus contacts at full pressure (allowing rudimentary drawing). Cursor movements with button 2 active are sent as wacom stylus movement-within-proximity-but-without-contact events. Keyboard keys left/up/page up are sent as a left key, which moves back a page in the current notebook; right/down/page down similarly are sent as a right key to move forward a page. Home and escape are sent as the home key, which generally returns to the home page of xochitl.
Cursor location reporting
If a VNC client supports the cursor position pseudo-encoding, the position of the wacom stylus is sent to it. Unfortunately, sending pen status such as in/out-of-proximity/contact events is not easy to do in the current protocol; a protocol extension may be warranted if clients would find such information useful.
Building
The supported way to build this is via the
Nix package manager, through the
nix-remarkable
expressions. To build just this project via nix build
from this
repo, download it into the pkgs/
directory of nix-remarkable
.
There are four variants that can be built: for either rM1 or rM2, and in a "standalone" configuration or not. The standalone configuration will statically link against all libraries not installed on the reMarkable by default, and will bundle necessary kernel modules and tools; it is recommended unless you have a particular need for sharing. By default, the Nix expressions will build all variants.
For other systems, the commands needed to compile and link are given
in the Makefile. Build dependencies depend on the
configuraiton being built, but all versions require libvncserver
(built for the reMarkable) and the reMarkable kernel headers as well
as the rM-input-devices.h
from
rM-input-devices.
For the reMarkable 1, the mxc_epdc_fb_damage.h
file from
mxc_epdc_fb_damage
is also required; for the reMarkable 2, the libqsgepaper-snoop.h
file from
libqsgepaper-snoop
is needed instead.
For all versions, the -standalone versions require the standalone static library versions of the relevant dependencies---rM-input-devices for both, mxc_epdc_fb_damage for the rM1, and libqsgepaper-snoop for the rM2.
Prebuilt binaries are available in the Releases tab.
Usage
Copy the relevant executable to the device and run it; this will start a vnc server listening on port 5900. Any VNC client should be able to provide a (view-only) view of the tablet's screen when pointed at the reMarkable's IP address and standard VNC port (5900). Vinagre, Remmina, and gst-libvncclient-rfbsrc have been tested.
Note that the server will bind to all available network interfaces by default, so only run this if the tablet's WiFi is disconnected or connected to a trusted network. iptables rules could be used to restrict traffic.