Awesome
ImJoy Jupyter Extension
Using ImJoy with Jupyter Notebook and JupyterLab
This extension utilizes the imjoy-rpc library to enable bidirectional RPC calls between the ImJoy core and plugins.
The library has an abstract transport interface that can support different types of transport. For now, we support Jupyter comms message which is a custom message protocol used in Jupyter notebooks (for example powers jupyter widgets).
To use it, you need to install the imjoy-jupyter-extension
in Python which will also include the imjoy-rpc as a dependency (see below).
Install the extension for Jupyter Notebook or JupyterLab
For Jupyter Notebooks
pip install -U imjoy-jupyter-extension
You can verify the installation by running:
jupyter nbextension list
And you should see something like imjoy_jupyter_extension/index enabled
in the output.
If you still have issue, you can try to install it manually.
git clone https://github.com/imjoy-team/imjoy-jupyter-extension.git
cd imjoy-jupyter-extension
jupyter nbextension install imjoy_jupyter_extension --py # add --sys-prefix if using virtualenv or conda
jupyter nbextension enable imjoy_jupyter_extension --py # add --sys-prefix if using virtualenv or conda
jupyter nbextension list
For Jupyter Lab
pip install -U imjoy-jupyter-extension
jupyter labextension install imjoy-jupyter-extension
Use ImJoy plugins inside Jupyter notebooks
Now you can start a jupyter notebook via for example jupyter notebook
command, you should be able to see an ImJoy icon in the toolbar if everything goes well.
For JupyterLab, start it via jupyter lab
command, then create a notebook and you should see an ImJoy button in the toolbar:
Now run ImJoy plugins in a cell, see an example that uses itk-vtk-viewer to visualize images:
import imageio
import numpy as np
from imjoy_rpc import api
class ImJoyPlugin():
def setup(self):
api.log('plugin initialized')
async def run(self, ctx):
viewer = await api.showDialog(src="https://kitware.github.io/itk-vtk-viewer/app/")
# show a 3D volume
image_array = np.random.randint(0, 255, [10,10,10], dtype='uint8')
# show a 2D image
# image_array = imageio.imread('imageio:chelsea.png')
await viewer.setImage(image_array)
api.export(ImJoyPlugin())
With the above code, you created an ImJoy plugin. To run it, click the Run button with the ImJoy icon. It will then call the run function of your plugin.
- GIF Demo: Visualizing 2D image
- GIF Demo: Visualizing 3D volume
- GIF Demo: Load ImageAnnotator
You can also do the reverse by running a notebook inside ImJoy, to do that, please first create an empty jupyter notebook. Then copy and paste the url into the "+ PLUGINS" dialog, press enter and install the plugin. Click the newly installed plugin and you will get a notebook page open in ImJoy.
Try to copy and paste the itk-vtk-viewer example to a cell and execute it. Similarily, if you now click the run ImJoy button in the toolbar, you will get the viewer open insided ImJoy.
Load plugin dynamically in a notebook
There are different ways to load a plugin in a notebook, you can load from URL for the source code of the plugin (e.g. gist or github file url) or a hosted web app (e.g. the itk-vtk-viewer app used above). Besides that, you can also write a window plugin directly in a notebook. This is practical especially for debugging window plugin for your GUI.
You can directly pass a string with the plugin source code to api.getPlugin
, api.createWindow(src=...)
or api.showDialog(src=...)
, however, that doesn't give you syntax highliting in a Jupyter notebook cell. A workaround solution (which also works for Google Colab) is to write plugin code in a seperate cell startting with ## ImJoy Plugin
and wrap the source code with IPython.display.HTML()
. For example:
## ImJoy Plugin
from IPython.display import HTML
my_plugin_source = HTML('''
<docs lang="markdown">
[TODO: write documentation for this plugin.]
</docs>
<config lang="json">
{
"name": "Untitled Plugin",
"type": "window",
"tags": [],
"ui": "",
"version": "0.1.0",
"cover": "",
"description": "[TODO: describe this plugin with one sentence.]",
"icon": "extension",
"inputs": null,
"outputs": null,
"api_version": "0.1.8",
"env": "",
"permissions": [],
"requirements": [],
"dependencies": [],
"defaults": {"w": 20, "h": 10}
}
</config>
<script lang="javascript">
class ImJoyPlugin {
async setup() {
api.log('initialized')
}
async run(ctx) {
}
}
api.export(new ImJoyPlugin())
</script>
<window lang="html">
<div>
<p>
Hello World
</p>
</div>
</window>
<style lang="css">
</style>
''')
Then in another cell, you can passing the plugin source code to api.getPlugin
, api.createWindow(src=...)
or api.showDialog(src=...)
to make an actually plugin:
from imjoy_rpc import api
class ImJoyPlugin():
async def setup(self):
pass
async def run(self, ctx):
# for regular plugin
# p = await api.getPlugin(my_plugin_source)
# or for window plugin
# await api.createWindow(src=my_plugin_source)
await api.showDialog(src=my_plugin_source)
api.export(ImJoyPlugin())
Development
To develop the JupyterLab extension, open a terminal and run:
cd imjoy_jupyterlab_extension
npm run watch
In another terminal window run:
cd imjoy_jupyterlab_extension
npm run build
jupyter labextension link .
jupyter lab