Awesome

ResNeXt-Tensorflow

Tensorflow implementation of ResNeXt using Cifar10

If you want to see the original author's code, please refer to this link

Requirements

Tensorflow 1.x
Python 3.x
tflearn (If you are easy to use global average pooling, you should install tflearn)

Issue

If not enough GPU memory, Please edit the code

with tf.Session() as sess : NO
with tf.Session(config=tf.ConfigProto(allow_soft_placement=True)) as sess : OK

Compare Architecture

ResNet

ResNeXt

I implemented (b)
(b) is split + transform(bottleneck) + concatenate + transition + merge

Idea

What is the "split" ?

  def split_layer(self, input_x, stride, layer_name):
      with tf.name_scope(layer_name) :
          layers_split = list()
          for i in range(cardinality) :
              splits = self.transform_layer(input_x, stride=stride, scope=layer_name + '_splitN_' + str(i))
              layers_split.append(splits)

          return Concatenation(layers_split)

Cardinality means how many times you want to split.

What is the "transform" ?

  def transform_layer(self, x, stride, scope):
      with tf.name_scope(scope) :
          x = conv_layer(x, filter=depth, kernel=[1,1], stride=stride, layer_name=scope+'_conv1')
          x = Batch_Normalization(x, training=self.training, scope=scope+'_batch1')
          x = Relu(x)

          x = conv_layer(x, filter=depth, kernel=[3,3], stride=1, layer_name=scope+'_conv2')
          x = Batch_Normalization(x, training=self.training, scope=scope+'_batch2')
          x = Relu(x)
          return x

What is the "transition" ?

  def transition_layer(self, x, out_dim, scope):
      with tf.name_scope(scope):
          x = conv_layer(x, filter=out_dim, kernel=[1,1], stride=1, layer_name=scope+'_conv1')
          x = Batch_Normalization(x, training=self.training, scope=scope+'_batch1')

          return x

Comapre Results (ResNet, DenseNet, ResNeXt)

compare

Related works

References

Classification Datasets Results

Author

Junho Kim