Home

Awesome

<p align="center"> <img width=150 src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/logo.png"> </p> <h1 align="center">TensorSpace.js</h1> <p align="center"><b>Present Tensor in Space</b></p> <p align="center"> <strong>English</strong> | <a href="https://github.com/tensorspace-team/tensorspace/blob/master/README_zh.md"><strong>中文</strong></a> </p> <p align="center"> <a href="https://www.npmjs.com/package/tensorspace"><img src="https://img.shields.io/npm/v/tensorspace.svg" alt="npm version" height="18"></a> <a href="https://github.com/tensorspace-team/tensorspace/blob/master/LICENSE"><img src="https://img.shields.io/badge/license-Apache--2.0-green.svg" alt="license badge"></a> <a href="https://github.com/tensorflow/tfjs"><img src="https://img.shields.io/badge/dependencies-tfjs-brightgreen.svg" alt="dependencies badge"></a> <a href="https://github.com/mrdoob/three.js"><img src="https://img.shields.io/badge/dependencies-three.js-brightgreen.svg" alt="dependencies badge"></a> <a href="https://github.com/tweenjs/tween.js"><img src="https://img.shields.io/badge/dependencies-tween.js-brightgreen.svg" alt="dependencies badge"></a> <a href="https://travis-ci.org/tensorspace-team/tensorspace"><img src="https://travis-ci.org/tensorspace-team/tensorspace.svg?branch=master" alt="build"></a> <a href="https://gitter.im/tensorspacejs/Lobby#"><img src="https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg" alt="gitter"></a> </p>

TensorSpace is a neural network 3D visualization framework built using TensorFlow.js, Three.js and Tween.js. TensorSpace provides Keras-like APIs to build deep learning layers, load pre-trained models, and generate a 3D visualization in the browser. From TensorSpace, it is intuitive to learn what the model structure is, how the model is trained and how the model predicts the results based on the intermediate information. After preprocessing the model, TensorSpace supports to visualize pre-trained model from TensorFlow, Keras and TensorFlow.js.

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_lenet.gif"> </p> <p align="center"> <b>Fig. 1</b> - Interactive LeNet created by TensorSpace </p>

Table of Content

Motivation

TensorSpace is a neural network 3D visualization framework designed for not only showing the basic model structure, but also presenting the processes of internal feature abstractions, intermediate data manipulations and final inference generations.

By applying TensorSpace API, it is more intuitive to visualize and understand any pre-trained models built by TensorFlow, Keras, TensorFlow.js, etc. TensorSpace introduces a way for front end developers to be involved in the deep learning ecosystem. As an open source library, TensorSpace team welcomes any further development on visualization applications.

Getting Started

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/workflow.png"> </p> <p align="center"> <b>Fig. 2</b> - TensorSpace Workflow </p>

1. Install TensorSpace

Install in the Basic Case

Download dependencies build files TensorFlow.js (tf.min.js), Three.js (three.min.js), Tween.js (tween.min.js), TrackballControls (TrackballControls.js).

Download TensorSpace build file tensorspace.min.js from Github, NPM, TensorSpace official website or CDN:

<!-- Replace "VERSION" with the version you want to use. -->
<script src="https://cdn.jsdelivr.net/npm/tensorspace@VERSION/dist/tensorspace.min.js"></script>

Include all build files in web page.

<script src="tf.min.js"></script>
<script src="three.min.js"></script>
<script src="tween.min.js"></script>
<script src="TrackballControls.js"></script>
<script src="tensorspace.min.js"></script>

Install in the Progressive Framework

import * as TSP from 'tensorspace';

Checkout this Angular example for more information.

2. Preprocess the Pre-trained Model

Before applying TensorSpace to visualize the pre-trained model, there is an important pipeline - TensorSpace model preprocessing ( Checkout this article for more information about TensorSpace preprocessing ). We can use TensorSpace Converter to quickly complete the TensorSpace Preprocessing.

For example, if we have a tf.keras model in hand, we can use the following TensorSpace-Converter conversion script to convert a tf.keras model to the TensorSpace compatible format:

$ tensorspacejs_converter \
    --input_model_from="tensorflow" \
    --input_model_format="tf_keras" \
    --output_layer_names="padding_1,conv_1,maxpool_1,conv_2,maxpool_2,dense_1,dense_2,softmax" \
    ./PATH/TO/MODEL/tf_keras_model.h5 \
    ./PATH/TO/SAVE/DIR

Note:

<p align="center"> <img width="80%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/hello_converter.gif"> </p> <p align="center"> <b>Fig. 3</b> - TensorSpace-Converter Usage </p>

3. Using TensorSpace to Visualize the Model

If TensorSpace is installed successfully and the pre-trained deep learning model is preprocessed, let's create an interactive 3D TensorSpace model.

For convenience, we will use the the resources from this repository's HelloWorld directory, which includes preprocessed TensorSpace compatible LeNet model and sample input data ("5") as an example to illustrate this step. All source code can be found in helloworld.html.

First, we need to new a TensorSpace model instance:

let container = document.getElementById( "container" );
let model = new TSP.models.Sequential( container );

Next, based on the LeNet structure: Input + Padding2D + 2 X (Conv2D & Maxpooling) + 3 X (Dense), build the Topology of the TensorSpace model:

model.add( new TSP.layers.GreyscaleInput() );
model.add( new TSP.layers.Padding2d() );
model.add( new TSP.layers.Conv2d() );
model.add( new TSP.layers.Pooling2d() );
model.add( new TSP.layers.Conv2d() );
model.add( new TSP.layers.Pooling2d() );
model.add( new TSP.layers.Dense() );
model.add( new TSP.layers.Dense() );
model.add( new TSP.layers.Output1d({
    outputs: ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]
}) );

Last, we should load our preprocessed TensorSpace compatible model and use init() method to create the TensorSpace model:

model.load({
    type: "tensorflow",
    url: './PATH/TO/MODEL/model.json'
});
model.init(function(){
    console.log("Hello World from TensorSpace!");
});

We can get the following Fig. 3 model in the browser if everything looks good.

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/HelloWorld_empty_lenet.jpg"> </p> <p align="center"> <b>Fig. 4</b> - LeNet model without any input data </p>

We provide a extracted file which is a handwritten "5" as the input of our model: (online demo)

model.init(function() {
    model.predict( image_5 );
});

We put the predict( image_5 ) method in the callback function of init() to ensure the prediction is after the initialization complete.

Click the CodePen logo to try it in CodePen:   <a target="_blank" href="https://codepen.io/syt123450/pen/667a7943b0f23727790ca38c93389689"><img width=50 height=50 src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/codepen.png"></a>

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/HelloWorld_5.jpg"> </p> <p align="center"> <b>Fig. 5</b> - LeNet model with input data "5" </p>

Example

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_lenet.gif"> </p> <p align="center"> <b>Fig. 6</b> - Interactive LeNet created by TensorSpace </p>

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_alexnet.gif"> </p> <p align="center"> <b>Fig. 7</b> - Interactive AlexNet created by TensorSpace </p>

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_yolov2.gif"> </p> <p align="center"> <b>Fig. 8</b> - Interactive Yolov2-tiny created by TensorSpace </p>

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_resnet50.gif"> </p> <p align="center"> <b>Fig. 9</b> - Interactive ResNet-50 created by TensorSpace </p>

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_vgg.gif"> </p> <p align="center"> <b>Fig. 10</b> - Interactive Vgg16 created by TensorSpace </p>

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_acgan.gif"> </p> <p align="center"> <b>Fig. 11</b> - Interactive ACGAN created by TensorSpace </p>

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_mobilenetv1.gif"> </p> <p align="center"> <b>Fig. 12</b> - Interactive MobileNetv1 created by TensorSpace </p>

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_inceptionv3.gif"> </p> <p align="center"> <b>Fig. 13</b> - Interactive Inceptionv3 created by TensorSpace </p>

Visualize the LeNet Training Process with TensorSpace.js and TensorFlow.js.

➡ Live Demo

<p align="center"> <img width="100%" src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_lenet_training.gif"> </p> <p align="center"> <b>Fig. 14</b> - LeNet Training 3D Visualization </p>

View models locally

As some models above are extremely large, view them locally may be a good choice.

git clone https://github.com/tensorspace-team/tensorspace.git

Open "html" file in examples folder in local web server.

Documentation

Contributors

Thanks goes to these wonderful people (emoji key):

<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --> <!-- prettier-ignore -->
<img src="https://avatars2.githubusercontent.com/u/7977100?v=4" width="100px;"/><br /><sub><b>syt123450</b></sub><br />💻 🎨 📖 💡<img src="https://avatars3.githubusercontent.com/u/4524339?v=4" width="100px;"/><br /><sub><b>Chenhua Zhu</b></sub><br />💻 🎨 💡<img src="https://avatars0.githubusercontent.com/u/21956621?v=4" width="100px;"/><br /><sub><b>YaoXing Liu</b></sub><br />💻 🎨 💡<img src="https://avatars2.githubusercontent.com/u/19629037?v=4" width="100px;"/><br /><sub><b>Qi(Nora)</b></sub><br />💻 🎨<img src="https://avatars2.githubusercontent.com/u/97291?s=400&v=4" width="100px;"/><br /><sub><b>Dylan Schiemann</b></sub><br />📝<img src="https://avatars3.githubusercontent.com/u/25629006?s=400&v=4" width="100px;"/><br /><sub><b>BoTime</b></sub><br />💻 📖 💡<img src="https://avatars0.githubusercontent.com/u/9149028?s=400&v=4" width="100px;"/><br /><sub><b>Kamidi Preetham</b></sub><br />📖
<img src="https://avatars3.githubusercontent.com/u/333921?s=400&v=4" width="100px;"/><br /><sub><b>Wade Penistone</b></sub><br />📖
<!-- ALL-CONTRIBUTORS-LIST:END -->

Contact

If you have any issue or doubt, feel free to contact us by:

License

Apache License 2.0

<div id="next">Next Episode</div>

<p align="center"> <img width=300 src="https://raw.githack.com/tensorspace-team/tensorspace/master/assets/tensorspace_vr_logo.png"> </p> <h3 align="center">TensorSpace-VR</h1> <p align="center"><b>Present Neural Network in VR</b></p> <p align="center"> <img width=80% src="./assets/next_episode.gif"> </p> <p align="center"> <b>Fig. 15</b> - TensorSpace VR Demo </p>