Awesome
DISCONTINUATION OF PROJECT
This project will no longer be maintained by Intel. Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project. Intel no longer accepts patches to this project.
Deep Learning Tutorials on Apache Spark using BigDL
Step-by-step Deep Learning Tutorials on Apache Spark using BigDL. The tutorials are inspired by Apache Spark examples, the Theano Tutorials and the Tensorflow tutorials.
Topics
- RDD
- DataFrame
- SparkSQL
- StructureStreaming
- Forward and backward
- Linear Regression
- Introduction to MNIST
- Logistic Regression
- Feedforward Neural Network
- Convolutional Neural Network
- Recurrent Neural Network
- LSTM
- Bi-directional RNN
- Auto-encoder
- Visualizing Learning
Environment
- Python 3.5/3.6
- JDK 8
- Apache Spark >= 2.2.1
- Jupyter Notebook 4.1
- BigDL 0.7.0
- Setup env on Mac OS / Setup env on Linux
Start Jupyter Server
- Run
pip install BigDL==0.7.0 - Run
jupyter notebook --notebook-dir=./ --ip=0.0.0.0 --no-browser
Start Toree Kernel to Run Scala Notebooks
- Run
pip install BigDL==0.7.0 - Run
pip install https://dist.apache.org/repos/dist/release/incubator/toree/0.2.0-incubating/toree-pip/toree-0.2.0.tar.gz - Run
./toree_install.sh - Run
jupyter notebook --notebook-dir=./ --ip=0.0.0.0 --no-browser
Run Demo
- Open a browser - Suggest Chrome or Firefox or Safari
- Access notebook client at address http://localhost:8888, open the example ipynb files and execute.