Home

Awesome

Clojars Project

Hanami

Interactive arts and charts visualizations with Clojure(Script), Vega-lite, and Vega. Flower viewing 花見 (hanami)

<a href="https://jsa-aerial.github.io/aerial.hanami/index.html"><img src="https://github.com/jsa-aerial/hanami/blob/master/resources/public/Himeji_sakura.jpg" align="left" hspace="10" vspace="6" alt="hanami logo" width="150px"></a>

Hanami is a Clojure(Script) library and framework for creating interactive visualization applications based in Vega-Lite (VGL) and/or Vega (VG) specifications. These specifications are declarative and completely specified by data (JSON maps). VGL compiles into the lower level grammar of VG which in turn compiles to a runtime format utilizting lower level runtime environments such as D3, HTML5 Canvas, and WebGL. In addition to VGL and VG, Hanami is built on top of Reagent and Re-Com.

Table of Contents

toc

Overview

Hanami is a Clojure(Script) library and framework for creating interactive visualization applications based in Vega-Lite (VGL) and/or Vega (VG) specifications. These specifications are declarative and completely specified by data (JSON maps). VGL compiles into the lower level grammar of VG which in turn compiles to a runtime format utilizting lower level runtime environments such as D3, HTML5 Canvas, and WebGL.

In keeping with the central data oriented tenet, Hanami eschews the typical API approach for generating specifications in favor of using recursive transforms of parameterized templates. This is also in keeping with the data transformation focus in functional programming, which is espcially nice in Clojure(Script).

An important aspect of this approach is that parameterized templates can be used to build other such templates by being higher level substitutions. In addition templates can be composed and this is an important idiomatic use. Furthermore, templates may be merged, though typically this is after transformation. The overall result enables the construction of sharable libraries of templates providing reusable plots, charts, and entire visualizations. Generally these will be domain and/or task specific. Hanami itself provides only a small set of very generic templates, which have proven useful in constructing more domain/task specific end results.

Hence, templates are a means to abstract all manner of visualization aspects and requirements. In this sense they are similar to what Altair provides but without the complications and limitations of an OO class/method based approach.

Installation

To install, add the following to your project :dependencies:

[aerial.hanami "0.15.1"]

Features

Examples

(ns hanami.examples
  (:require [aerial.hanami.common :as hc]
            [aerial.hanami.templates :as ht]
            [aerial.hanami.core :as hmi]
            ...)

In all of the documentation, these namespaces are referred to by the shorthand provided in this require example.

Simple cars

As a first example let's compare a Hanami template with corresponding Altair code. This is a the typical scatter plot example from the Vega-Lite developers used in many tutorials and by others in various places. First, the Altair:

cars = data.cars()

alt.Chart(cars).mark_point().encode(
    x='Horsepower',
    y='Miles_per_Gallon',
    color='Origin',
    ).interactive()

Hanami template version:

(hc/xform ht/point-chart
  :UDATA "data/cars.json"
  :X "Horsepower" :Y "Miles_per_Gallon" :COLOR "Origin")

Which, using the standard default substitution keys transforms to this Vega-Lite JSON specification:

{:data {:url "data/cars.json"},
 :width 400,
 :height 300,
 :background "floralwhite",
 :encoding
   {:x {:field "Horsepower", :type "quantitative"},
    :y {:field "Miles_per_Gallon", :type "quantitative"},
    :color {:field "Origin", :type "nominal"},
    :tooltip
    [{:field "Horsepower", :type "quantitative"}
     {:field "Miles_per_Gallon", :type "quantitative"}]}}

When rendered, both the Altair code and Hanami template, result in the following visualization, where the mouse is hovering over the point given by [132, 32.7]:

Hanami pic 1

Instrumented barchart

Hanami visualizations may be instrumented with external active componets (react/reagent/re-com) to enable external transforms on them. An example of an instrumented chart:

(hc/xform ht/bar-chart
  :USERDATA
  (merge
   (hc/get-default :USERDATA)
   {:vid :bc1
    :slider `[[gap :size "10px"] [label :label "Add Bar"]
              [label :label ~minstr]
              [slider
               :model :m1
               :min ~min, :max ~max, :step 1.0
               :width "200px"
               :on-change :oc1]
              [label :label ~maxstr]
              [input-text
               :model :m1
               :width "60px", :height "26px"
               :on-change :oc2]]})
  :HEIGHT 300, :WIDTH 350
  :X "a" :XTYPE "ordinal" :XTITLE "Foo" :Y "b" :YTITLE "Bar"
  :DATA data)

Which requires the active component implementing the instrument to be coded over on the client side. For this simple example, the test code on the client is a branch inside of a cond testing for the :slider key and value and then processing it:

      (let [cljspec spec
            udata (cljspec :usermeta)
            default-frame {:top [], :bottom [],
                           :left [[box :size "0px" :child ""]],
                           :right [[box :size "0px" :child ""]]}]
        (cond
          ...
          (udata :slider)
          (let [sval (rgt/atom "0.0")]
            (printchan :SLIDER-INSTRUMENTOR)
            (merge default-frame
                   {:top (xform-recom
                          (udata :slider)
                          :m1 sval
                          :oc1 #(do (bar-slider-fn tabid %)
                                    (reset! sval (str %)))
                          :oc2 #(do (bar-slider-fn tabid (js/parseFloat %))
                                    (reset! sval %)))}))

When this chart is rendered, it shows as (left, before slider move; right, after slider move)

Hanami pic 2 Hanami pic 3

Contour plot using Vega template

An example using a Vega template for contour plotting. An important point to recognize here is that Vega specifications are also pure data, so the exact same recursive transformation works on Vega templates as Vega Lite templates:

(hc/xform
  ht/contour-plot
  :MODE "vega"
  :HEIGHT 400, :WIDTH 500
  :X "Horsepower", :XTITLE "Engine Horsepower"
  :Y "Miles_per_Gallon" :YTITLE "Miles/Gallon"
  :UDATA "data/cars.json"
  :XFORM-EXPR #(let [d1 (% :X)
                     d2 (% :Y)]
                 (format "datum['%s'] != null && datum['%s'] !=null" d1 d2)))

This generates far too much to show here, as Vega is a much lower level formal specification language than Vega Lite. This renders as:

Hanami pic 3.1

Tree Layout using Vega template

Another interesting Vega example uses the tree-layout template. Using this template for such layouts abstracts away a good deal of low level complexity. In this example a software system module dependency graph is rendered.

(hc/xform
 ht/tree-layout
  :MODE "vega"
  :WIDTH 650, :HEIGHT 1600
  :UDATA "data/flare.json"
  :LINKSHAPE "line" :LAYOUT "cluster"
  :CFIELD "depth")

Hanami pic 3.2

A number of other examples appear at the end of this README, along with their transformations and renderings.

Templates, Substitution Keys and Transformations

Templates are simply maps parameterized by substitution keys. Generally, templates will typically correspond to a legal VG or VGL specification or a legal subcomponent thereof. For example, a complete VGL specification (rendered as Clojure) is a legal template - even though it has no substitution keys. At the other extreme, templates can correspond to pieces of specifications or subcomponents. These will always have substitution keys - if they didn't there would be no point to them.

Substitution Keys can be considered or thought of in two ways. They are the keys in the substitution map(s) of the recursive transformer hc/xform. There is a default map hc/_defaults (which is an atom containing the map) with various default substitution keys and their corresponding default values. This map can be updated and xform can also be supplied a variadic list of substitution keys and their values for a given invocation.

The second way of thinking about substitution keys is that they are the starting values of keys in templates. So, they represent parameterized values of keys in templates.

As an example consider the following. In a template we have a field and value :field :X. The :X here is a substitution key - it will be replaced as the value of :field during transformation by its value in the current substitution map. In the default substitution map hc/_defaults :X has a value of "x", so the result will be :field "x".

A more complex example would be the field and value :encoding :ENCODING in the predefined subcomponent ht/view-base which lays out the structure of a Vega-Lite view. As before :ENCODING will be replaced during transformation by the value of :ENCODING in the substitution map. However, the default value of :ENCODING is the predefined subcomponent ht/xy-encoding. This value is a map describing the structure of a view's x-y encodings and contains many fields with their own substitution keys. So, to produce a final value, ht/xy-encoding is recursively transformed so that the final value of :encoding is a fully realized x-y encoding for the view being processed. ht/view-base has several other such fields and is itself recursively transformed in the context of the current substitution map. And its final value will be the base of a chart, such as a line chart (ht/line-chart) or area chart (ht/area-chart), or some new plot/chart/layout/etc of your own for some domain specific application.

Function values for substitution keys

A substitution key may have a function of one argument for its value: (fn[submap] ...). The submap parameter is the current substitution map in the transformation. This map contains the special key ::hc/spec whose value is the initial (input) specification as well as all current substitution keys and their current values. This is useful in cases where a substitution key's value should be computed from other values. For example, a typical (Saite does this by default) use case would be to compute the label (display name) of a tab from its id:

        :TLBL #(-> :TID % name cljstr/capitalize)

This takes the tab's id, which here is a keyword, converts to a string and returns the capitalized version.

Subtitution Key Functions

A more general version of the function as value of a substitution key is provided by the hc/subkeyfns map and associated processing. The function hc/update-subkeyfns can be used to register a function for a subtitution key. This is a 'global' function - separate from any value given for the key. These functions take three arguments: (fn[submap, subkey, subval] ...).

Where the submap parameter is the current substitution map in the transformation; the subkey parameter is the substitution key for this function; and the subval parameter is the value of the substitution key.

These functions can be very useful in providing a further level of abstractions is template specifications. For example, it is very typical that a color specification is intended for the mark of a chart (line-chart, bar-chart, point-chart, etc). providing a simple string indicating the data element to facet with colors makes for a cleaner abstraction, but this needs to be converted to correct full form field and type element. Having a subkeyfn for the hc/color-key (default :COLOR) supports this sort of processing:

        color-key
         (fn[xkv subkey subval]
           (if (string? subval)
             (xform ht/default-mark-props (assoc xkv :MPFIELD subval))
             subval))

There is a default set of such keys provided with this particular case being one of them. There are a few others, including one which implements the file data source capability for specifications.

Template local defaults

As of version 0.12.9 there is support for default parameterization per template (where 'template' here means any map whatsoever). A new transformation control key is introduced, :aerial.hanami.templates/defaults which can be used to supply a map, local to the template, containing default values of substitution keys. The substitution keys may be those specific only to the template or any other enclosing template, or for changing global defaults. The values are in effect for the duration of the template's processing, unless a more nested template has its own local defaults which override some of the outer's values. In all cases, the in line (point of call) user specified values to hc/xform take precedent over any template local defaults.

Local defaults can be a nice way to deliver new templates without any need to setup new substitution keys in the global registry or require users to add many key-value pairs to their call to hc/xform using the new templates. Addtionally they can automatically parameterize the underlying templates (those used in constructing a new template) to reflect the value of them required for the form of a new template. As an example, let's look at the following, which will show the construction of a new trend-chart for plotting base data along with a smoothed trend line for it.

;;; Base layer for trend-chart
(def trend-layer
  (assoc ht/line-chart
         :aerial.hanami.templates/defaults
         {:X :data/x> :XTYPE :xtype>
          :Y :data/y> :YTYPE :ytype>
          :YSCALE {:zero false}
          :DATA hc/RMV
          :WIDTH :width> :HEIGHT :height>
          :USERDATA hc/RMV}))

This gives the base form for what the two layers will look like. It is simply the base ht/line-chart parameterized to give the default value of each layer when transformed. The new trend chart will use :data/x> and :data/y> as the parameters for x and y data fields, along with corresponding parameters for the types of the fields :xtype> and :ytype>. Next we use this base layer template to create the trend-chart template:

;;; Trend chart - two layers.  1. base data, 2. loess trend line
;;; Parameters :data/x for x-axis data. :data/y> for y-axis data
;;;            :xtype> and :ytype> for axis types, will default to :XTYPE :YTYPE
;;;            :width>, will default to 700
;;;            :height>, will default to :HEIGHT default
;;;            :trend-color> for loess line, defaults to "firebrick"
(def trend-chart
  (assoc ht/layer-chart
         :description "A two layer plot of base data and its smoothed trend line given by loess transform"
         :aerial.hanami.templates/defaults
         {:LAYER [(hc/xform trend-layer)
                  (hc/xform
                   trend-layer
                   :TRANSFORM [{:loess :data/y> :on :data/x>}]
                   :MCOLOR :trend-color>)]
          :trend-color> "firebrick"
          :xtype> :XTYPE :ytype> :YTYPE
          :width> 700
          :height> (hc/get-defaults :HEIGHT)}))

Here we use the underlying templates of the predefined ht/layer-chart and the new trend-layer. The layer structure is given by the default for :LAYER as being the two transformed cases for trend-layer. The first layer transform just ensures the result has all the new parameters in place. The second also gives the loess transform and provides for a new trend-color for custom coloring the trend line, with a default of "firebrick". The width of the chart gets a new default of 700 while the height defaults to the global :HEIGHT default. The new :xtype> and :ytype> axis types get defaults of whatever the global :XTYPE and :YTYPE values are.

This can now be used in a highly simplified form such as the following:

(let [[cols data] (get-trend-data file)]
 (hc/xform
  trend-chart
  :DATA (mapv #(zipmap cols %) data)
  :data/x> :year :xtype> :temporal
  :data/y> :C))

Resulting in this output:

trend-chart

Basic transformation rules

There are some important rules that guide certain aspects of the recursive transformation process. These aspects are really only important in advanced template contruction, but if you find yourself in such circumstances, they are quite useful and helpful.

Meta data and the :USERDATA key

Vega and Vega-Lite support the inclusion of application specific meta data in specifications via the :usermeta top level key in a specification. To Vega and Vega-Lite, this key is completely transparent and is not used in any way by their processing. This affords applications using them to include application specific data, in particular, control data.

Hanami defaults the :usermeta field to the substitution key :USERDATA, which has a default value of RMV. So, by default, there is no application meta data. However, several aspects of Hanami's messaging system, session management, picture frames and tab system expect and make use of control data supplied via the :usermeta field and associated values for the :USERDATA substitution key. So, if you plan on using any of that, you will need to supply a value for this key - either as a default (via hc/update-defaults) or explicitly in a transform (hc/xform). Of course, if you have your own application specific meta/control data that needs to be supplied to clients, you can provide your own fields and values here.

Hanami understands the following 'special' fields:

    {:export {:png true, :svg true}
     :scaleFactor :SCALEFACTOR
     :editor true
     :source false
     :renderer :RENDERER ; either "canvas" or "svg" - see defaults
     :mode :MODE}        ; either "vega-lite" or "vega" - see defaults

As an example, Saite has an init function as part of its start up which sets the values of :USERDATA and subsequent defining substitution keys as:

:USERDATA
{:tab {:id :TID, :label :TLBL, :opts :TOPTS},
 :frame {:fid :FID, :top :TOP, :bottom :BOTTOM, :left :LEFT, :right :RIGHT}
 :opts :OPTS,
 :vid :VID,
 :msgop :MSGOP,
 :session-name :SESSION-NAME}

:OPTS (hc/default-opts :vgl)
:SESSION-NAME "Exploring"
:TID :expl1
:TLBL #(-> :TID % name cljstr/capitalize)
:TOPTS (hc/default-opts :tab)
:MSGOP :tabs

Where hc/default-opts is:

{:vgl {:export {:png true, :svg true}
       :scaleFactor :SCALEFACTOR
       :editor true
       :source false
       :renderer :RENDERER ; either "canvas" or "svg" - see defaults
       :mode :MODE}        ; either "vega-lite" or "vega" - see defaults
 :tab {:order :ORDER       ; either :row or :col - see defaults
       :eltsper :ELTSPER   ; count of elements per row/col - see defaults
       :size "auto"}}

Example predefined templates

Here are some examples as provided by the name space aerial.hanami.templates which is always referred to in documentation as ht/.

A number of 'fragments':

(def default-tooltip
  [{:field :X :type :XTYPE}
   {:field :Y :type :YTYPE}])

(def default-mark-props
  {:field :MPFIELD :type :MPTYPE})

(def default-row
  {:field :ROW :type :ROWTYPE})

(def data-options
  {:values :VALDATA, :url :UDATA, :name :NDATA})

(def interval-scales
  {:INAME
   {:type "interval",
    :bind "scales", ; <-- This gives zoom and pan
    :translate
    "[mousedown[event.shiftKey], window:mouseup] > window:mousemove!"
    :encodings :ENCODINGS,
    :zoom "wheel!",
    :resolve :IRESOLVE}})

(def mark-base
  {:type :MARK, :point :POINT,
   :size :MSIZE, :color :MCOLOR,
   :filled :MFILLED})

A couple 'subcomponents':

(def xy-encoding
  {:x {:field :X
       :type :XTYPE
       :timeUnit :XUNIT
       :axis :XAXIS
       :scale :XSCALE
       :aggregate :XAGG}
   :y {:field :Y
       :type :YTYPE
       :timeUnit :YUNIT
       :axis :YAXIS
       :scale :YSCALE
       :aggregate :YAGG}
   :opacity :OPACITY
   :row :ROWDEF
   :column :COLDEF
   :color :COLOR
   :size :SIZE
   :shape :SHAPE
   :tooltip :TOOLTIP})

(def view-base
  {:usermeta :USERDATA
   :title :TITLE
   :height :HEIGHT
   :width :WIDTH
   :background :BACKGROUND
   :selection :SELECTION
   :data data-options
   :transform :TRANSFORM
   :encoding :ENCODING})

And some charts.

;; Useful for empty picture frames
(def empty-chart
  {:usermeta :USERDATA})

(def line-chart
  (assoc view-base
         :mark (merge mark-base {:type "line"})))

(def point-chart
  (assoc view-base
         :mark (merge mark-base {:type "circle"})))

(def grouped-bar-chart
  {:usermeta :USERDATA
   :title  :TITLE
   :height :HEIGHT
   :width  :WIDTH
   :background :BACKGROUND
   :selection :SELECTION
   :data data-options

   :mark "bar"
   :encoding :ENCODING

   :config {:bar {:binSpacing 0
                  :discreteBandSize 1
                  :continuousBandSize 1}
            :view {:stroke "transparent"},
            :axis {:domainWidth 1}}})

Example predefined substitution keys

All of these are taken from hc/_defaults They are chosen so as to indicate how some aspects of the above template examples get transformed.

         :USERDATA RMV, :MODE "vega-lite", :RENDERER "canvas", :SCALEFACTOR 1
         :TOP RMV, :BOTTOM RMV, :LEFT RMV, :RIGHT RMV
         :BACKGROUND "floralwhite"
         :OPACITY RMV

         ;; Note that removed things will get Vega/Vega-Lite defaults
         :TITLE RMV, :TOFFSET RMV
         :HEIGHT 300, :WIDTH 400, :DHEIGHT 60

         ;; get-data-vals is a function which handles :DATA and :FDATA
         :VALDATA get-data-vals
         :DATA RMV, :FDATA RMV, :SDATA RMV, :UDATA RMV, :NDATA RMV

         ;; Describes the x encoding in xy-encoding. Similar for y encoding
         :X "x", :XTYPE, "quantitative", :XUNIT RMV
         :XSCALE RMV, :XAXIS {:title :XTITLE, :grid :XGRID, :format :XFORMAT}
         :XTITLE RMV, :XGRID RMV, :XFORMAT RMV

         ;; Aggregation transforms
         :AGG RMV, :XAGG RMV, :YAGG RMV

         ;; Mark properties
         :POINT RMV, :MSIZE RMV, :MCOLOR RMV, :MFILLED RMV
         :MPTYPE "nominal", :SHAPE RMV, :SIZE RMV


         ;; Note that by default then :ROWDEF -> {:field :ROW :type :ROWTYPE} ->
         ;; {:field RMV, :type RMV} -> {} -> RMV. See transformation rules
         ;; Similar for :COLDEF
         :ROWDEF ht/default-row :ROW RMV, :ROWTYPE RMV

         :TOOLTIP ht/default-tooltip
         :ENCODING ht/xy-encoding

         :TRANSFORM RMV
         :SELECTION RMV

Data Sources

Visualizations are based in and so require data to realize them. Specifications provide several means of declaring the source and processing of data to be used in a visualization. The underlying Vega and Vega-Lite systems provide for several of these by various means. Hanami currently directly supports three of these plus a fourth. The keys and expected values are given in this section.

These three are based directly on the underlying Vega and Vega-Lite options. The fourth, file data sources, is provided by Hanami.

File data source

:FDATA <filepath | [filepath, type-vector-or-map]]

The filepath is a full path for the OS and can denote a file with extensions clj, edn, json, or csv. These extensions indicate the file types - no other checking is done. For the clj, edn, and json the content must be a vector of maps, where each map is a record of data fields and their values. CSV files are converted to vectors of maps, each map built from the column names and a row's corresponding values.

For csv, a type-vec-or-map may be provided as the second value of a vector pair (tuple). This can be either a vector of type indicators in 1-1 correspondence with the column names of the csv file, or a map where each key is one of the column names and the value is a type indicator. Type indicators supported are (the strings): "string", "int", "float", and "double". In either case, the strings in a row associated with the type indicators are converted per the indicator. The default type indicator for the case of maps is "string", i.e., return the original string value.

Example:

(hc/xform my-plot-template
  :FDATA ["/Data/RNAseq/Exp-xyz/Out/dge-xyz.csv" {"dge" "float"}])

This will create a vector of maps from the csv content, where the "dge" field in each map will have its value converted to a floating point number.

Walk through example of transformation

It's worth having a look at what happens with the simple car chart template example and its transformations. The value of ht/point-chart is:

(def point-chart
  (assoc view-base
         :mark (merge mark-base {:type "circle"})))

We have already seen (above) what view-base, data-options, and mark-base are, so we know that the starting value of point-chart is:

{:usermeta :USERDATA
 :title :TITLE
 :height :HEIGHT
 :width :WIDTH
 :background :BACKGROUND
 :mark {:type "circle"
        :point :POINT
        :size :MSIZE
        :color :MCOLOR
        :filled :MFILLED},
 :selection :SELECTION
 :data {:values :VALDATA, :url :UDATA, :name :NDATA}
 :transform :TRANSFORM
 :encoding :ENCODING}

We already have seen what the default values of the substitution keys here are. In the transform, explicit values are given for :UDATA, :X, :Y, and :COLOR. We will see about :COLOR a bit later, but with the values for the first three, conceptually, we have a "first version":

{:height 300
 :width 400
 :background "floralwhite"
 :mark {:type "circle"}
 :data {:url "data/cars.json"}
 :encoding {:x {:field "Horsepower"
                :type "quantitative"
                :axis {:title :XTITLE, :grid :XGRID, :format :XFORMAT}}
            :y {:field "Miles_per_Gallon"
                :type "quantitative"
                :axis {:title :YTITLE, :grid :YGRID, :format :YFORMAT}}
            :row {:field :ROW :type :ROWTYPE}
            :column {:field :COLUMN :type :COLTYPE}
            :color :COLOR
            :tooltip [{:field "Horsepower" :type "quantitative"}
                      {:field "Miles_per_Gallon" :type "quantitative"}]}}

By the second rule of transformation, we also know that the :axis, :row, and :column fields will be replaced first by {} (since all of their fields default to RMV) and then by the third rule they will be removed. So, conceptually, the 'next' version is:

{:height 300, :width 400, :background "floralwhite"
 :mark {:type "circle"}
 :data {:url "data/cars.json"}
 :encoding {:x {:field "Horsepower" :type "quantitative"}
            :y {:field "Miles_per_Gallon" :type "quantitative"}
            :color :COLOR
            :tooltip [{:field "Horsepower" :type "quantitative"}
                      {:field "Miles_per_Gallon" :type "quantitative"}]}}

OK, :COLOR is all we have left. As shown in the section on substitution key functions Hanami has a default such function for the hc/color-key (default :COLOR). If the subval of the key is a string, then this function returns the value (xform ht/default-mark-props (assoc xkv :MPFIELD subval)). As we know from above, the value of ht/default-mark-props is {:field :MPFIELD :type :MPTYPE} and so the function here returns {:field "Origin" :type "nominal"}. So, the final value of the transformation is:

{:height 300, :width 400, :background "floralwhite"
 :mark {:type "circle"}
 :data {:url "data/cars.json"}
 :encoding {:x {:field "Horsepower" :type "quantitative"}
            :y {:field "Miles_per_Gallon" :type "quantitative"}
            :color {:field "Origin" :type "nominal"}
            :tooltip [{:field "Horsepower" :type "quantitative"}
                      {:field "Miles_per_Gallon" :type "quantitative"}]}}

The reason for caveat uses of 'conceptually' in the above description is the final rule of transformation. Which says that the actual transformation process is via a depth first walk and replacement. So, for example, :encoding would have only had its full final value, not any intermediate value(s).

Application Construction

Hanami is an visualization application developement enabler. It is a library with optional framework aspects provided for both server side (Clojure) and client side (ClojureScript) development. The client is most typically expected to be in the browser, but technically may not be.

Library overview

The library portion of Hanami centers on

These bits are not opionated in how to go about developing a domain specific visualization application. There is no constraint on how page(s) are to be laid out, what sort of ancillary and support components should be used, what CSS is used, etc. You can use Hiccup or Re-Com or whatever to layout and structure your application. Nor is there any opinion about the structure of the server side. And while there are a set of defaults, there are no requirements or expectations about the form, makeup, or content of templates and substitution keys. You can replace, augment, or change any or all of the defaults.

Framework overview

The framework portion of Hanami is opinionated, though not too stringently. It consists of

These combine to form the basic page layout from this 'framework perspective' as shown here:

framework topology graphic

Hanami framework layout

The header area is constructed by the :header-fn argument of the client start function. The tab bar is dynamically constructed via :tabs messages or by explict calls to the hmi/tabs client function. The content of each tab's body is also constructed dynamically via these same means. If the tab doesn't exist, it will be created and added to the tab bar at the time it's body is also rendered. Updates to a tab will simply update the existing tab's body.

As an example Saite makes use of the framework aspects of Hanami and here is an example page layout from a session in it.

Hanami framework layout

Resource requirements

The basic resource requirements are the various stylesheets, mostly in support of re-com, that are shown in the development 'landing page' index.html. When you create your own index.html landing page - or generate it dynamically - you will need to include these resources.

Also, as indicated in the development index.html, you will need to indicate where your final compiled JavaScript implementing your application will be and what it is called.

NOTE: If you will be running serverless, you will need to make sure that the resources and final javascript are located relative to where you load your landing page from or that all the links are absolute.

Header

The client side start function has a :header-fn parameter which is a function of a single parameter that should return the main application header area. The default function for this returns a header which supports multiple named session groups. This is one mechanism where by you can insert global controls, logos, application 'avatars', etc. Alternatively you may choose to have this return nothing and wait for the application initialization message, to perform the application setup. The latter may be more appealing if you compute a number of fields that support the contruction of a more sophisticated header.

Tabs

The tab system of Hanami is intended to be a fairly general purpose document structuring and layout capability for a variety of visualization applications. But that very fact means it has some level of 'opinion' on such layouts and so you may not want to use it if you need total general / manual layouts. But for the very large subset of cases where it fits nicely it can be very helpful. There is a set of functions and components of the client API for dealing with the tab system. If you use the standard framework main components, you will not have to use this API - it is all handled for you. The API is given in case you need / want to set up your own components which make use of it.

Basics

As depicted in the framework topology graphic, the tab system has a dedicated area for the tab bar. Tabs are dynamically added to the tab bar from left to right. They may also be deleted. Any selected tab will have its body displayed in the dedicated content area. If you use Hanami's standard framework, the main components will drive all tab actions and updates.

Configuration and behavior

To use tabs, the :usermeta field of specifications must contain the :tab key whose value must be a map. The keys and values of this map are detailed in the tab section of the USERDATA section of this documentation. The :id field is the primary key for identifying and manipulating a tab and its body. The :label field is for human readable naming of the tab. The :opts field details the auto layout format for the tab's body. It is worth noting that if the :eltsper field has a value of 1, the effect is to have the body layed out linearly from top to bottom in the order the specifications were given to the tab.

Tabs are added and updated via the update-tabs client core function. This function takes a vector of specifications and groups them by the tab id for each. Then for each such group,preprocesses the specifications (including instrumentation and frames) and places this information into the tab database. This database update triggers the Reagent components responsible for rendering any such changes.

Extension tabs

In addition to 'standard tabs', those described above, the tab system may be extended with 'extension tabs'. The defining characteristic of such tabs is that they have an extra field: [:opts :extfn]. The value of this field must be a Reagent form function. Typically it will be a form-2 and will implement the body of the tab. This functionality is seamlessly integrated with standard tabs and will be implicitly invoked when the tab is selected.

Extension tabs are intended to be built and added on the client side. add-tab can be used to manually add such tabs, and the :app-init user-msg is a convenient place to perform this action (as part of any other application initialization). NOTE in client only you can explicitly fire this message as part of on page load code start.

Incremental bodies

As of version 0.9.0, there is support for incrementally adding to a tab's content area, which is also known as it's body. Previously the only way to effect a change on a standard (non extention) tab's body was to perform a full update on it via update-tabs or more directly via vis-list. Both of these required the full list of specifications to be rendered. There are applications where this is simply too restrictive. You may want to simply insert a picture frame at a given point in a page. Or simply extend it, without needing to manage the entire sequence of specifications. There are now two new client core functions add-to-tab-body and remove-from-tab-body which can adjust the content of a tab at the granularity of picture frames.

Wrapping functions

As of version 0.9.0 there is now support for per tab user wrapping functions. This supports custom layout encompassing or encasing the tab's fully structured grid layout of the tab's sequence of specifications. Wrapping functions are configured by means of the :wrapfn key/value pair specifiable in the tab's options. This provides a level of customization that is in between fully custom extension tabs and standard tabs. Wrapping functions must accept one argument, gridhc, which is the fully computed hiccup for the grid layout of the tab, and return a hiccup/re-com element which includes the gridhc element somewhere inside it (directly or in some child). The result will then be rendered in the tab's content area.

Sessions

The server side start-server function has an :idfn parameter which is a parameterless function which returns session names for client connections. Upon a client connetion (initial websocket connect), the server will send a registration message to the client. Among other fields in the data sent will be a :session-name. This is the name of the session group the client has been placed in according to the idfn function.

It is possible for the client to "opt out" of this session group in favor of another - including a totally new one of it's own making - by sending a set session message to the server. This also implies that session groups need only contain one session.

As a simple example of how this can be done, the client start function's default header-fn function uses a two stage input area to support changing the server's default assigned session group to some other such group (including a new one if the input does not name an existing group):

(defn set-session-name [name]
  (let [old-uid (get-adb [:main :uid])
        name (if (= name "") (old-uid :name) name)]
    (sp/setval [sp/ATOM :main :session-name sp/ATOM] name app-db)
    (when (not= name (old-uid :name))
      (update-adb [:main :uid :name] name)
      (send-msg {:op :set-session-name
                 :data {:uid old-uid
                        :new-name name}}))))

(defn session-input []
  (if (not= "" (sp/select-one [sp/ATOM :main :session-name sp/ATOM] app-db))
    [:p]
    [input-text
     :model (get-adb [:main :session-name])
     :on-change set-session-name
     :placeholder (get-adb [:main :uid :name])
     :width "100px"]))

(defn default-header-fn []
  [h-box :gap "10px" :max-height "30px"
   :children [[gap :size "5px"]
              [:img {:src (get-adb [:main :logo])}]
              [title
               :level :level3
               :label [:span.bold (get-adb [:main :title])]]
              [session-input]
              [gap :size "5px"]
              [title
               :level :level3
               :label [:span.bold (get-adb [:main :uid :name])]]
              [gap :size "30px"]]])

On connection the client's header contains a header with the default assigned session group name listed along side an input text box. The user can decide to change the assigned group, by typing in a new name. This will fire a set-session-name message to the server to reassign this session to the desired group.

message overivew

Any message sent from the server to a named session group, will result in all current session members getting the message. In particular, messages updating the visualizations and picture frame content of tab bodies, will be sent to all such session members. This is equally true for application specific messages sent via hmi/send-msg

Messages

There are a few builtin messages which are sent between the server and client. These are generally in support of the framework aspects of Hanami. In general, domain specific applications will have a set of their own messages. These can be sent via send-msg, available on both the client and server, and processed by user-msg, also available on both the client and server

message overivew

The messages listed in that graphic overview are roughly in the order they occur from top to bottom. Let's walk through them so that we know what happens as they occur.

Connection

Upon a client opening a connection, the server creates a session/app initialization data package, which is sent to the client via a :register message. This package is a map composed of two groups of data:

The client side dispatches two calls upon reception of this message. Both are passed this final initialization data map. The first is hc/register which takes the standard fields and sets up the initial client side database. The second is the multimethod user-msg where the msg given is {:op :app-init :data init-data-map}. If your application implements this method for :app-init, it will be called and any application specific work can be done at that point.

Session group

As described in the section on sessions and session groups, the client may change the group its session belongs to (named by the idefn function of start-server) as described in the section on connections. This is achieved by sending the server a :set-session-name message, which has the form:

{:op :set-session-name
 :data {:uid old-uid
        :new-name name}}

Where

Upon receipt of such a message, the server will update its database to reflect the change: the session's uuid will be moved to the requested group named by the :new-name field.

Tab updates

The :tabs message is the main way to update one or more tabs in a current sesion if you are using the tab system in a client / server setup. The simplest way to send these messages is via the server sv! function. Typically this will be from within a server side editor / IDE.

In client only applications, you can achieve the exact same effect by calling the client side 'mirror' function sv!.

As an example, the simple cars case listed in the opening examples was dispatched as:

(-> (hc/xform ht/point-chart
      :UDATA "data/cars.json"
      :X "Horsepower" :Y "Miles_per_Gallon" :COLOR "Origin")
    hmi/sv!)

User messages

User, also know here as "application specific", messages are sent via the send-msg function. This function exists in both the client and server hmi name space. The entire point of these messages is to support extension messages that are specific to the needs of an application. All such messgaes will be caught by the receiving party and dispatched to the multimethod user-msg. There is one such message which is dispatched on the client side implicitly - the :app-init msg on client connection. This supports the any application specific client initialization processing.

Two examples of from Saite

The first is for the :app-init message.

(defmethod user-msg :app-init [msg]
  (update-adb [:main :convert-chan] (async/chan))
  (add-tab {:id :xvgl
            :label "<->"
            :opts {:extfn (tab<-> :NA)}}))

So, we add a Saite specific async channel to the database in support of conversions and popup renderings. Then we add the extension tab for the conversion / popup renderer capability.

The second (client / server is for messages specifically required by Saite to perform data conversion and transformation operations for the client.

;;; Client sends two versions of :read-clj msg to server.
;;;  One for conversion and one for popup rendering
;;; The popup case
(let [...
      msg {:op :read-clj
           :data {:session-name nm
                  :render? true
                  :cljstg inspec}}
      _ (send-msg msg)
      ...]
      ...
      )

;;; Server user-msg method for this:
(defmethod hmi/user-msg :read-clj [msg]
  (let [{:keys [session-name cljstg render?]} (msg :data)
        clj (try
              (let [clj (->> cljstg clojure.core/read-string)]
                (swap! dbg (fn[m] (assoc m :clj clj)))
                (->> clj xform-cljform eval final-xform))
              (catch Exception e
                {:error (format "Error %s" (or (.getMessage e) e))})
              (catch Error e
                {:error (format "Error %s" (or (.getMessage e) e))}))]
    (swap! dbg (fn[m] (assoc m :xform-clj clj)))
    (hmi/print-when [:pchan :umsg] :CLJ clj)
    (hmi/send-msg session-name :clj-read (assoc clj :render? render?))))

;;; NOTE the send-msg to client with :cli-read

;;; Client user-msg method for this:
(defmethod user-msg :clj-read [msg]
  (let [data (msg :data)
        render? (data :render?)
        clj (dissoc data :render?)
        result (if render?
                 clj
                 (try (-> clj clj->js (js/JSON.stringify nil, 2))
                      (catch js/Error e (str e))))
        ch (get-adb [:main :convert-chan])]
    (go (async/>! ch result))))

Picture Frames

Picture frames are simply a way to automatically encase your visualizations with 'meta' level instrumentation and / or arbitrary annotations. They are composed of four individual parts corresponding to the top, bottom, left, and right quadrants. Picture frames (or simply frames) are specified in the usermeta data of a specification via the :frame key. By default Hanami uses the substitution key :USERDATA for this. So, the format is:

{...
 :USERDATA {...
            :frame {:fid ...
                    :top ...
                    :bottom ...
                    :left ...
                    :right ...}
            ...
            }
}

Hanami picture frame

All of the quadrants are optional and :frame {} is legal, which by the third rule of transformation will result in its removal.

The value of :fid should be a globally unique id (keyword or string). If it is not specified the field will be removed and the frame will not have an id. This id will be used as the frame's DOM id as well as the corresponding spec's id. So, if it is not supplied the frame will not be accessible (in particular, it won't be accessible when using incremental bodies.

The value of a quadrant can be any legal mix of strings, hiccup, markdown (MD) and / or active components. Where 'legal' here means 'yields a legal DOM branch'. A great resource for active components, which Hanami provides as part of its package, is Re-Com. This is especially true if you are not a CSS/flex and / or component savant.

As of version 0.5.0, picture frames may be 'empty' in that they do not need to have an associated visualization. To make this a bit simpler, there is a new template for these cases ht/empty-chart.

As of version 0.9.0, markdown is directly available via the md active component, as such it is an extension to the hiccup used by Hanami (from Reagent). You would use this like / where you would use any other hiccup / re-com component. The form is [md <optional style map> <string of markdown>]. The string can include newlines. The style map is for things such as font-size, color, etc.

ASSIDE: LaTex is specifically not included in Hanami. This decision is based on three reasons: 1. Many (most?) applications do not need LaTex math rendering. 2. MathJax is the go to JS engine for this and is a heavy dependency that would not be used in these applications. 3) React (as of version 16+) has a 'bug' preventing proper running of engines like MathJax (and Google Translate and many many other such rendering engines). It is possible to get around issue 3. but it requires 'monkey patching' React. If you need LaTex, you should consider using Saite which is an application (built on Hanami) which does include MathJax, fixes 3., and is an application for general graphics and live document creation and sharing. Of course, if you are creating your own domain specific application and need LaTex for it, you will need to navigate issue 3. in your app. However, note that the md tag (see above markdown discussion) supports LaTex, so you can include it in your markdown (again, see Saite for examples).

You can specifiy frames from either the server or client side of your application. Working on the client side from within ClojureScript can make this more 'natural', as you are in the actual environment (browser/DOM) where things are running and rendering. However, specifying from the server side is fully supported, as long as Re-Com use is quoted (or more typical and useful, backquoted).

NOTE: if you are instrumenting your visualization (using active components - input box, dropdowns, selection lists, etc.) the functions updating the relevant model(s) of / for these components must be written over on the ClojureScript side (client). This is because, Hanami does not use a self hosted ClojureScript, but rather the cross compiled (with Google Closure) variant. Hence, you cannot, for example, write function code on the server side and have it eval'd on the client! (NOTE: this restriction may be lifted!)

Picture frames are fully automatic if you use the default tab system. If you use custom tabs and want to make use of frames, you should call hmi/vis-list on the client side (Cljs) for automatic rendering. If you are doing completely custom layouts, you will want to call hmi/frameit client function. As always, if you do not want to use any of this, you should 'manually' use the hmi/vgl reagent vega/vega-lite component.

A couple of examples. These are actually taken from Saite, which is an interactive, exploratory and ad-hoc visualization application written with Hanami. Worth noting is that Saite assumes an interactive REPL model of exploration on the server side, pushing visualizations to the client. Hence, the only active components that can be used are those that are self contained, like information buttons or modal panels.

(let [_ (hc/add-defaults
         :NAME #(-> :SIDE % name cljstr/capitalize)
         :STYLE hc/RMV)
      frame-template {:frame
                      {:SIDE `[[gap :size :GAP]
                               [p {:style :STYLE}
                                "This is the " [:span.bold :NAME]
                                " 'board' of a picture "
                                [:span.italic.bold "frame."]]]}}
      frame-top (hc/xform
                 frame-template :SIDE :top :GAP "10px")

      frame-left (hc/xform
                  frame-template :SIDE :left :GAP "10px"
                  :STYLE {:width "100px" :min-width "50px"})
      frame-right (merge-with merge
                   (hc/xform
                    frame-template :SIDE :right :GAP "2px"
                    :STYLE {:width "100px" :min-width "50px"})
                   (hc/xform
                    frame-template :SIDE :left :GAP "2px"
                    :STYLE {:width "100px" :min-width "50px"
                            :color "white"}))
      frame-bot (hc/xform
                 frame-template :SIDE :bottom :GAP "10px")]
  (->> (mapv #(hc/xform ht/point-chart
                :HEIGHT 200 :WIDTH 250
                :USERDATA (merge (hc/get-default :USERDATA) %)
                :UDATA "data/cars.json"
                :X "Horsepower" :Y "Miles_per_Gallon" :COLOR "Origin")
             [frame-top frame-left frame-bot frame-right])
       hmi/sv!))

Hanami picture frame

(let [text "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quod si ita est, sequitur id ipsum, quod te velle video, omnes semper beatos esse sapientes. Tamen a proposito, inquam, aberramus."
      top `[[gap :size "150px"]
            [p "An example showing a "
             [:span.bold "picture "] [:span.italic.bold "frame"]
             ". This is the top 'board'"
             [:br] ~text]]
      left `[[gap :size "10px"]
             [p {:style {:width "100px" :min-width "50px"}}
              "Some text on the " [:span.bold "left:"] [:br] ~text]]
      right `[[gap :size "2px"]
              [p {:style {:width "200px" :min-width "50px"
                          :font-size "20px" :color "red"}}
               "Some large text on the " [:span.bold "right:"] [:br]
               ~(.substring text 0 180)]]
      bottom `[[gap :size "200px"]
               [title :level :level3
                :label [p {:style {:font-size "large"}}
                        "Some text on the "
                        [:span.bold "bottom"] [:br]
                        "With a cool info button "
                        [info-button
                         :position :right-center
                         :info
                         [:p "Check out Saite Visualizer!" [:br]
                          "Built with Hanami!" [:br]
                          [hyperlink-href
                           :label "Saite "
                           :href  "https://github.com/jsa-aerial/saite"
                           :target "_blank"]]]]]]]
  (->> [(hc/xform ht/point-chart
          :TID :picframes
          :TOP top :BOTTOM bottom :LEFT left :RIGHT right
          :UDATA "data/cars.json"
          :X "Horsepower" :Y "Miles_per_Gallon" :COLOR "Origin")]
       hmi/sv!))

Hanami picture frame

Empty Frames

As of version 0.5.0, picture frames can now be 'empty', i.e., they do not need an associated visualization. The basic layout is the same as for frames with visualizations:

Hanami empty frame

To make the use of these simpler (and easier) there is a new standard template ht/empty-chart which can be used to create them. Empty frames enable a more general document structure supporting paragraphs of html/'text oriented' material along side or in place of standard visualizations with or without frames. Here are a couple of examples showing the basic capability:

This example shows an empty frame with all 4 picture frame elements (areas) with various html/text information

(let [text "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quod si ita est, sequitur id ipsum, quod te velle video, omnes semper beatos esse sapientes. Tamen a proposito, inquam, aberramus."
      top `[[gap :size "50px"]
            [p {:style {:width "600px" :min-width "50px"}}
             "An example empty picture frame showing all four areas."
             " This is the " [:span.bold "top"] " area. "
             ~text ~text ~text]]
      left `[[gap :size "50px"]
             [p {:style {:width "300px" :min-width "50px"}}
              "The " [:span.bold "left "] "area as a column of text. "
              ~text ~text ~text ~text]]
      right `[[gap :size "70px"]
              [p {:style {:width "300px" :min-width "50px"}}
               "The " [:span.bold "right "] "area as a column of text. "
               ~text ~text ~text ~text]]
      bottom `[[gap :size "50px"]
               [v-box
                :children
                [[p {:style {:width "600px" :min-width "50px"}}
                  "The " [:span.bold "bottom "]
                  "area showing a variety of text. "
                  [:span.italic ~text] [:span.bold ~text]]
                 [p {:style {:width "400px" :min-width "50px"
                             :font-size "20px"}}
                  "some TeX: " "\\(f(x) = \\sqrt x\\)"]
                 [md {:style {:font-size "16px" :color "blue"}}
                  "#### Some Markup
* **Item 1** Lorem ipsum dolor sit amet, consectetur adipiscing elit.
* **Item 2** Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quod si ita est, sequitur id ipsum, quod te velle video, omnes semper beatos esse sapientes. Tamen a proposito, inquam, aberramus."]
                 [p {:style {:width "600px" :min-width "50px"
                             :color "red"}}
                  ~text]]]]]
  (->> (hc/xform ht/empty-chart
        :TID :picframes :TOP top :BOTTOM bottom :LEFT left :RIGHT right)
       hmi/sv!))

Hanami empty picframe

This next eample shows a tab/page with two picture frames. The left being a frame with an associated chart, while the one on the right shows how you can structure text in columns.

(let [text "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quod si ita est, sequitur id ipsum, quod te velle video, omnes semper beatos esse sapientes. Tamen a proposito, inquam, aberramus."
      top `[[gap :size "50px"]
            [p "Here's a 'typical' chart/plot filled picture frame."
             "It only has the top area"
             [:br] ~text]]
      left `[[gap :size "20px"]
             [p {:style {:width "200px" :min-width "50px"}}
              "This is an empty frame with a " [:span.bold "left "]
              "column of text" [:br] ~text ~text ~text ~text]]
      right `[[gap :size "30px"]
              [p {:style {:width "200px" :min-width "50px"}}
               "And a " [:span.bold "right "]
               "column of text"
               [:br] ~text ~text ~text ~text]]]
  (->> [(hc/xform ht/point-chart
          :TID :picframes :UDATA "data/cars.json"
          :TOP top
          :X "Horsepower" :Y "Miles_per_Gallon" :COLOR "Origin")
        (hc/xform ht/empty-chart
          :TID :picframes
          :LEFT left :RIGHT right)]
       hmi/sv!))

Hanami picframe chart and empty

Data Streaming

Client Only Apps

While the most typical use case scenario involves clients and server working together, Hanami also enables the development of client only applications. In this case, the Clojure (server) side of the library (and framework aspects) is not used and all development is done on the client (typically browser) side.

In client only applications, there are no explicit messages, nor is the usual client side start function used. Application initialization is no longer implicit (via the server to client register message); you will need to explicitly code your own specific initialization.

As noted in resource requirements, you still need to create a landing page (index.html) with all the required resources and to have your directory structure setup for relative loading or use absolute URLs in the links.

When running client only applications, the full template system is available on the client side. Both the hc and ht namespaces may be required and their resources used. So, you can make full use of templates, substitution keys, and transformations. Further, all the default templates are available for use.

Basic client app requirements

Require resources

You need to require the basic Hanami resources in your name space definition

(ns clientex.core
  (:require
  ...
   [aerial.hanami.core
    :as hmi
    :refer [printchan user-msg
            re-com-xref xform-recom
            default-header-fn default-instrumentor-fn make-instrumentor
            update-adb get-adb
            init-tabs
            hanami-main]]
   [aerial.hanami.common :as hc :refer [RMV]]
   [aerial.hanami.templates :as ht]
   ...))

The exact resources you refer will be up to how you typically do such referals. You may just use the :as hmi and reference all resources with hmi/.... If you want to use re-com capabilities, you will need to require those as well. For example, something like

 [re-com.core
    :as rcm
    :refer [h-box v-box box gap line h-split v-split scroller
            button row-button md-icon-button md-circle-icon-button info-button
            input-text input-password input-textarea
            label title p
            single-dropdown
            checkbox radio-button slider progress-bar throbber
            horizontal-bar-tabs vertical-bar-tabs
            modal-panel popover-content-wrapper popover-anchor-wrapper]]

But again, what you require and refer will be up to what you need for your application and how you want to structure it.

Application initialization

You will need to code your application setup and initialization functions. If you use the tab system you will need to call init-tabs as part of this. If you want to use hanami-main you will also need to setup the database to have the expected fields. The example client only app does this here

Landing page - index.html

The most typical way of getting the application loaded is via an explict index.html file which is correctly located in the directory resources/public of your application. The example application uses this index.html

Once you have your code compiled and located per the <script ...> tag in your index.html, you can simply use a browser to navigate to this file and your application will be active. If you want to have an interactive session(s) with the application you will need to run figwheel or an equivalent type of capability. The client only example app is set up to use figwheel (plus cljs-repl and piggyback)

Application start

Your code should have a final block which will start the application upon page load. Generally this will test for a specific DOM element upon which to render (via reagent) your main component. An example of this from the client only example is here. In this case the startup first sets several default substitution keys, then calls app-init which sets up the application. At this point you will be able to render visualizations just like from the server, using the same hc/xform transformer of templates, and hmi/sv!

Example Client Only Application

Hanami's github repository contains a client only example application. Found in the examples directory here. To run this you can simply change directory to the examples/ClientOnly directory. Then run lein repl. When repl is up and running, issue (use 'figwheel-sidecar.repl-api) and when ready issue (start-figwheel!). This will open a browser window and you will see:

Hanami pic 1

You can also, at this point, pull up the clientex.core.cljs in emacs, connect to the repl, and once connected issue (cljs-repl). Switch buffers to the clientex.core name space file, and evaluate the buffer. At this point you will be able to interactively transform and render other specifications (see the 'rich' comment for some other examples)

API

As noted, with respect to abstracting visualizations (ala' something like Altair) there isn't much of an API and no classes, objects, or methods are involved. Most of what API there is centers on application development and start up. This is split across the server and the client side.

Templates and Substitution keys

This applies across both the server and client - the facilities are available in both server and client. They are in aerial.hanami.common, in this documentation aka hc. NOTE this namespace, along with aerial.hanami.templates (aka ht), is available on both the client and server.

Startup

There are two main start functions. One each for the server and client.

Server start

(defn start-server
  [port & {:keys [route-handler idfn title logo img connfn]
           :or {route-handler (hanami-handler (hanami-routes))
                idfn (partial gensym "hanami-")
                connfn identity
                title "花見 Hanami"
                logo "logo.png"
                img "Himeji_sakura.jpg"}}]
  ...)

Client start

(defn start [& {:keys [elem port header-fn instrumentor-fn frame-cb]
                :or {header-fn default-header-fn
                     instrumentor-fn default-instrumentor-fn
                     frame-cb default-frame-cb
                     symxlate-cb identity}}]
  ...)

Message system

This applies across both the server and client - the facilities are available in both server and client. They are in aerial.hanami.core, in this documentation aka hmi. NOTE this namespace, exists on both the client and server.

send msg

In both cases, the receiving party will have their user-msg multimethod dispatched on the msg op key.

user msg

Client core

Visualization

Tab system

{:label <tab label name>
 :id <tab id>
 :opts <full options - see hc/default-opts>
 :specs <current set of specs for tab body>
 :spec-frame-pairs <current set of (spec,frame) pairs rendered in body>
 }

Hanami main

The main (top level) Reagent component for driving an application using the standard framework aspects

Creates a Re-Com v-box consisting of the framework header (using the value of calling the client start header function), tab bar area (via the active-tabs component, and tab body area (via the tabs component). Each is separated by a line.

If you are writing a client only application you could render this as part of your applications initialization if you want to use Hanami framework capabilities.

Server core

Example Transform 'Gallery'

A gallery of visualization examples exploiting the abstraction power of Hanami's templates and recursive transformations. This will likely keep growing as neat new things are built.

** If anyone has their own nice examples that they would like to pass along, please don't hesitate to submit a PR!!** Thanks!

Observed vs Binomial Model

Here is the same data (observed distribution vs binomial models) as row and column grouped (faceted) charts.

(hc/xform ht/grouped-bar-chart
  :TITLE "Real distribution vs Binomials", :TOFFSET 10
  :HEIGHT 80, :WIDTH 450
  :DATA ...
  :X "cnt" :XTYPE "ordinal" :XTITLE ""
  :Y "y" :YTITLE "Probability"
  :COLOR ht/default-color :CFIELD :ROW :CTYPE :ROWTYPE
  :ROW "dist" :ROWTYPE "nominal")

(hc/xform ht/grouped-bar-chart
  :TITLE "Real distribution vs Binomials", :TOFFSET 40
  :WIDTH (-> 550 (/ 6) double Math/round (- 15))
  :DATA ...
  :X "dist" :XTYPE "nominal" :XTITLE ""
  :Y "y" :YTITLE "Probability"
  :COLOR ht/default-color
  :COLUMN "cnt" :COLTYPE "ordinal")

Both of these transform into similar amounts of VGL output, but the first is somewhat more interesting. Note the :color mark propery value and the input for it in the above code.

{:title {:text "Real distribution vs Binomials", :offset 10},
 :height 80,
 :width 450,
 :background "floralwhite",
 :mark "bar",
 :encoding
 {:x {:field "cnt", :type "ordinal", :axis {:title ""}},
  :y {:field "y", :type "quantitative", :axis {:title "Probability"}},
  :row {:field "dist", :type "nominal"},
  :color
  {:field "dist",
   :type "nominal",
   :scale {:scheme {:name "greenblue", :extent [0.4 1]}}},
  :tooltip
  [{:field "cnt", :type "ordinal"}
   {:field "y", :type "quantitative"}]},
  :view {:stroke "transparent"},
  :axis {:domainWidth 1}},
 :data {:values ...}
 :config
 {:bar {:binSpacing 0, :discreteBandSize 1, :continuousBandSize 1}}

And the rendered visualizations are:

Hanami pic 4.1

Hanami pic 4.2

This is a nice example of how one visualization (the row grouping) can bring out the salient information so much better than another (the col grouping)

Lowess smoothing of Tn-Seq fitness data

The next is a visualization for an investigation into using lowess smoothing of Tn-Seq fitness data.

(hc/xform ht/layer-chart
  :TITLE "Raw vs 1-4 lowess smoothing" :TOFFSET 5
  :HEIGHT 500 :WIDTH 700
  :DATA (concat base-xy lowess-1 lowess-2 lowess-3 lowess-4)
  :LAYER (mapv #(hc/xform ht/gen-encode-layer
                  :MARK (if (= "NoL" %) "circle" "line")
                  :TRANSFORM [{:filter {:field "NM" :equal %}}]
                  :COLOR "NM"
                  :XTITLE "Position", :YTITLE "Count")
               ["NoL" "L1" "L2" "L3" "L4"]))

This one is interesting in that it combines some nice straight ahead Clojure data mapping with the template system. Here, we create five layers - but they are all different data sets and so VGL's repeat would not apply. But, Clojure's mapv combined with Hanami's xform does the trick. Of course, any number of layers could be so constructed.

{:title {:text "Raw vs 1-4 lowess smoothing", :offset 5},
 :height 500,
 :width 700,
 :background "floralwhite",
 :layer
 [{:transform [{:filter {:field "NM", :equal "NoL"}}],
   :mark "circle",
   :encoding
   {:x {:field "x", :type "quantitative", :axis {:title "Position"}},
    :y {:field "y", :type "quantitative", :axis {:title "Count"}},
    :color {:field "NM", :type "nominal"},
    :tooltip
    [{:field "x", :type "quantitative"}
     {:field "y", :type "quantitative"}]}}
  {:transform [{:filter {:field "NM", :equal "L1"}}],
   :mark "line",
   :encoding
   {:x {:field "x", :type "quantitative", :axis {:title "Position"}},
    :y {:field "y", :type "quantitative", :axis {:title "Count"}},
    :color {:field "NM", :type "nominal"},
    :tooltip
    [{:field "x", :type "quantitative"}
     {:field "y", :type "quantitative"}]}}
  {:transform [{:filter {:field "NM", :equal "L2"}}],
   :mark "line",
   :encoding
   {:x {:field "x", :type "quantitative", :axis {:title "Position"}},
    :y {:field "y", :type "quantitative", :axis {:title "Count"}},
    :color {:field "NM", :type "nominal"},
    :tooltip
    [{:field "x", :type "quantitative"}
     {:field "y", :type "quantitative"}]}}
  {:transform [{:filter {:field "NM", :equal "L3"}}],
   :mark "line",
   :encoding
   {:x {:field "x", :type "quantitative", :axis {:title "Position"}},
    :y {:field "y", :type "quantitative", :axis {:title "Count"}},
    :color {:field "NM", :type "nominal"},
    :tooltip
    [{:field "x", :type "quantitative"}
     {:field "y", :type "quantitative"}]}}
  {:transform [{:filter {:field "NM", :equal "L4"}}],
   :mark "line",
   :encoding
   {:x {:field "x", :type "quantitative", :axis {:title "Position"}},
    :y {:field "y", :type "quantitative", :axis {:title "Count"}},
    :color {:field "NM", :type "nominal"},
    :tooltip
    [{:field "x", :type "quantitative"}
     {:field "y", :type "quantitative"}]}}],
 :data {:values ...}
 :config
 {:bar {:binSpacing 1, :discreteBandSize 5, :continuousBandSize 5}}}

Hanami pic 5

With Overview + Detail

We can do something more interesting here in this case, as we may want to get close ups of various sections of such a plot. Instead of looking at the entire genome, we can focus on chunks of it with selection brushes using an overlay+detail display:

(hc/xform ht/vconcat-chart
   :TITLE "Raw vs 1-4 lowess smoothing" :TOFFSET 5
   :DATA (concat base-xy lowess-1 lowess-2 lowess-3 lowess-4)
   :VCONCAT [(hc/xform ht/layer-chart
               :LAYER
               (mapv #(hc/xform
                       ht/gen-encode-layer :WIDTH 1000
                       :MARK (if (= "NoL" %) "circle" "line")
                       :TRANSFORM [{:filter {:field "NM" :equal %}}]
                       :XSCALE {:domain {:selection "brush"}}
                       :COLOR "NM", :XTITLE "Position", :YTITLE "Count")
                     ["NoL" "L1" "L2" "L3"]))
             (hc/xform ht/gen-encode-layer
               :MARK "circle"
               :WIDTH 1000, :HEIGHT 60
               :TRANSFORM [{:filter {:field "NM" :equal "NoL"}}]
               :SELECTION {:brush {:type "interval", :encodings ["x"]}}
               :COLOR "NM" :XTITLE "Position", :YTITLE "Count")])

Here are two snapshots of the resulting interactive visualization:

Hanami pic 5.1 Hanami pic 5.2

Interactive Cross plot Differential Gene Expression

And lastly a quite involved example from a real application for RNASeq Differential Gene Expression:

(let [data (->> DGE-data (filter #(-> "padj" % (<= 0.05))))
       mdwn-brush (hc/xform ht/interval-brush-mdwn :MDWM-NAME "brush" :IRESOLVE "global")
       color {:field "NM" :type "nominal" :scale {:range ["#e45756" "#54a24b" "#4c78a8"]}}
       size {:condition {:selection {:not "brush"} :value 40} :value 400}
       tooltip `[{:field "Gene", :type "nominal"} ~@ht/default-tooltip {:field "pvalue", :type "quantitative"}]]
   (hc/xform
    ht/hconcat-chart
    :TITLE
    "RNASeq Exp 180109_NS500751_0066 DGE for aT4-024V10min vs aT4-024V30min"
    :TOFFSET 40
    :DATA :data
    :HCONCAT[(hc/xform
              ht/point-chart
              :TITLE "MA Plot"
              :MSIZE 40
              :SELECTION (merge  (hc/xform ht/interval-scales :INAME "grid1")
                                 mdwn-brush)
              :X "baseMean", :Y "log2FoldChange"
              :COLOR color, :SIZE size, :TOOLTIP tooltip)
             (hc/xform
              ht/point-chart
              :TITLE "Volcano Plot"
              :MSIZE 40
              :SELECTION (merge (hc/xform ht/interval-scales :INAME "grid2")
                                mdwn-brush)
              :X "log2FoldChange", :Y "-log10(pval)"
              :COLOR color, :SIZE size :TOOLTIP tooltip)]))

Transforms to:

{:hconcat
 [{:encoding
   {:x {:field "baseMean", :type "quantitative"},
    :y {:field "log2FoldChange", :type "quantitative"},
    :color
    {:field "NM",
     :type "nominal",
     :scale {:range ["#e45756" "#54a24b" "#4c78a8"]}},
    :size
    {:condition {:selection {:not "brush"}, :value 40}, :value 400},
    :tooltip
    [{:field "Gene", :type "Nominal"}
     {:field "baseMean", :type "quantitative"}
     {:field "log2FoldChange", :type "quantitative"}
     {:field "pvalue", :type "quantitative"}]},
   :mark {:type "circle", :size 40},
   :width 450,
   :background "floralwhite",
   :title {:text "MA Plot"},
   :selection
   {"grid1"
    {:type "interval",
     :bind "scales",
     :translate
     "[mousedown[event.shiftKey], window:mouseup] > window:mousemove!",
     :encodings ["x" "y"],
     :zoom "wheel!",
     :resolve "global"},
    "brush"
    {:type "interval",
     :on
     "[mousedown[!event.shiftKey], window:mouseup] > window:mousemove!",
     :translate
     "[mousedown[!event.shiftKey], window:mouseup] > window:mousemove",
     :resolve "global",
     :mark {:fill "#333", :fillOpacity 0.125, :stroke "white"}}},
   :height 400}
  {:encoding
   {:x {:field "log2FoldChange", :type "quantitative"},
    :y {:field "-log10(pval)", :type "quantitative"},
    :color
    {:field "NM",
     :type "nominal",
     :scale {:range ["#e45756" "#54a24b" "#4c78a8"]}},
    :size
    {:condition {:selection {:not "brush"}, :value 40}, :value 400},
    :tooltip
    [{:field "Gene", :type "Nominal"}
     {:field "log2FoldChange", :type "quantitative"}
     {:field "-log10(pval)", :type "quantitative"}
     {:field "pvalue", :type "quantitative"}]},
   :mark {:type "circle", :size 40},
   :width 450,
   :background "floralwhite",
   :title {:text "Volcano Plot"},
   :selection
   {"grid2"
    {:type "interval",
     :bind "scales",
     :translate
     "[mousedown[event.shiftKey], window:mouseup] > window:mousemove!",
     :encodings ["x" "y"],
     :zoom "wheel!",
     :resolve "global"},
    "brush"
    {:type "interval",
     :on
     "[mousedown[!event.shiftKey], window:mouseup] > window:mousemove!",
     :translate
     "[mousedown[!event.shiftKey], window:mouseup] > window:mousemove",
     :resolve "global",
     :mark {:fill "#333", :fillOpacity 0.125, :stroke "white"}}},
   :height 400}],
 :config
 {:bar {:binSpacing 1, :discreteBandSize 5, :continuousBandSize 5}},
 :width 450,
 :background "floralwhite",
 :title
 {:text
  "RNASeq Exp 180109_NS500751_0066 DGE for aT4-024V10min vs aT4-024V30min",
  :offset 40},
 :height 400,
 :data {:values [... ]}}

Well, now that is quite a lot. When sent to a view, visualizes as follows. Note that this is a fully interactive visualization where each grid can be independently zoomed and panned and brush stroke highlighting in one view hightlights the covered points in the other view. Here the mouse is hovering over the upper left point in the volcano plot.

Hanami pic 6