Home

Awesome

This project are closed. New versions and much more are there: PiePline

Neural Pipeline

Neural networks training pipeline based on PyTorch. Designed to standardize training process and accelerate experiments.

Build Status Coverage Status Maintainability Gitter chat

Getting started:

Documentation

Documentation Status

See the examples

Neural Pipeline short overview:

import torch

from neural_pipeline.builtin.monitors.tensorboard import TensorboardMonitor
from neural_pipeline.monitoring import LogMonitor
from neural_pipeline import DataProducer, TrainConfig, TrainStage,\
    ValidationStage, Trainer, FileStructManager

from somethig import MyNet, MyDataset

fsm = FileStructManager(base_dir='data', is_continue=False)
model = MyNet().cuda()

train_dataset = DataProducer([MyDataset()], batch_size=4, num_workers=2)
validation_dataset = DataProducer([MyDataset()], batch_size=4, num_workers=2)

train_config = TrainConfig(model, [TrainStage(train_dataset),
                                   ValidationStage(validation_dataset)], torch.nn.NLLLoss(),
                           torch.optim.SGD(model.parameters(), lr=1e-4, momentum=0.5))

trainer = Trainer(train_config, fsm, torch.device('cuda:0')).set_epoch_num(50)
trainer.monitor_hub.add_monitor(TensorboardMonitor(fsm, is_continue=False))\
                   .add_monitor(LogMonitor(fsm))
trainer.train()

This example of training MyNet on MyDataset with vizualisation in Tensorflow and with metrics logging for further experiments comparison.

Installation:

PyPI version PyPI Downloads/Month PyPI Downloads

pip install neural-pipeline

For builtin module using install:

pip install tensorboardX matplotlib

Install latest version before it's published on PyPi

pip install -U git+https://github.com/toodef/neural-pipeline