Awesome
NRS_pytorch
official pytorch implementation of Neural Random Subapce (NRS)
paper is availabel at [arxiv]
Abstract
Random subspace is the pillar of random forests. We propose Neural Random Subspace (NRS), a novel deep learning based random subspace method. In contrast to previous forest methods, NRS enjoys the benefits of end-to-end, data-driven representation learning, as well as pervasive support from deep learning software and hardware platforms, hence achieving faster inference speed and higher accuracy. Furthermore, as a non-linear component to be encoded into Convolutional Neural Networks (CNNs), NRS learns non-linear feature representations in CNNs more efficiently than previous higher-order pooling methods, producing good results with negligible increase in parameters, floating point operations (FLOPs) and real running time. We achieve superior performance on 35 machine learning datasets when compared to random subspace, random forests and gradient boosting decision trees (GBDTs). Moreover, on both 2D image and 3D point cloud recognition tasks, integration of NRS with CNN architectures achieves consistent improvements with negligible extra cost.
Keywords: random subspace, ensemble learning, deep neural networks
Getting Started
Prerequisites
- python 3
- PyTorch (> 1.0)
- torchvision (> 0.2)
- Numpy
Train Examples
- Uci datasets: We use letter for example
CUDA_VISIBLE_DEVICES="0" python main.py \
--config-file configs/letter.yaml \
- CUB200: We used 4 GPUs to train CUB200.
CUDA_VISIBLE_DEVICES="0,1,2,3" python main.py \
--config-file configs/cub200resnet.yaml
- More configurations
CUDA_VISIBLE_DEVICES="0" python main.py \
--config-file configs/letter.yaml \
MODEL.META_ARCHITECTURE UciFCNet \ #if you want to use MLP, else default
MODEL.N_MUL 50 \ #if you want to use different values for nMul, the same for other hyper-parameters, e.g., nPer
SOLVER.NUM_EPOCHS 50 \ # if you want to specify the total epochs, the same for other settings, e.g., batch-size
OUTPUT_DIR "Results/uci/letter" \ #the output directory for log file and model file
Test Examples using Pretrained model
CUDA_VISIBLE_DEVICES="0,1,2,3" python main.py \
--config-file configs/cub200resnet.yaml \
MODEL.RESUME Results/ResNet_model_best.pth.tar \
TRAIN False
3D part code
Code for 3D recognition is availabel at [NRS_3D]
Citation
@article{NRS,
title={Neural random subspace},
author={Yun-Hao Cao and Jianxin Wu and Hanchen Wang and Joan Lasenby},
journal={Pattern Recognition},
volume={112},
pages={Article 107801},
year={2021},
publisher={Elsevier}
}