Awesome
Terraform Provider Iterative + Jupyter + TensorBoard
Repository supporting blog: short version on dev.to & full version on hackernoon.com
Painlessly deploy an ML-ready Jupyter server and sync results with your preferred cloud compute provider.
To get started, clone this repo, then in the repo directory run:
export NGROK_TOKEN="..." # Sign up for free at https://ngrok.com
export TF_LOG_PROVIDER=INFO # (optional) Control verbosity
terraform init # Setup local dependencies
terraform apply # Create cloud resources & upload "shared" workdir
terraform refresh # Get Jupyter & TensorBoard URLs
# ...
# Have fun!
# (optional) Click "Quit" in Jupyter to shutdown the cloud machine
# ...
terraform destroy # Download "shared" workdir & terminate cloud resources
🛈 Note that it can take a couple of minutes after apply
for the machine to be ready.
🛈 Note that Terraform Provider Iterative (TPI) will automatically restart interrupted spot/preemptible instances (including restoring Jupyter's working directory). In such cases, run terraform refresh
again to obtain the new Jupyter & TensorBoard URLs.
Requirements
<!-- `git checkout generic` for Azure, GCP & K8s -->- Download
terraform
(free) - ngrok credentials (free)
- Cloud credentials (AWS, Azure, GCP, or Kubernetes). For example:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
Alternatives
Some that we are aware of: (Anything missing? Please do open a PR!)
- Google Colab: based on (but not identical to) Jupyter, has resource limits and limited config options (CPU, GPU, RAM, memory, timeouts)
- Binder: no GPU, no config options, for-profit use disallowed, has user guidelines
However there are a few distinct advantages to using terraform
over the alternatives:
- Lower cost: use your preferred cloud provider's existing pricing, including on-demand per-second billing and bulk discounts
- Auto-recovery: auto-backup
workdir
& auto-recover terminatedspot
instances - Custom spec: full control over hardware & software requirements via
main.tf
-- including machine types (CPU, GPU, RAM, storage) & images