Awesome
Large-Margin Softmax Loss, Angular Softmax Loss, Additive Margin Softmax, ArcFaceLoss And FocalLoss In Tensorflow
This repository contains core codes of the reimplementation of the following papers in TensorFlow:
- Large-Margin Softmax Loss for Convolutional Neural Networks
- SphereFace: Deep Hypersphere Embedding for Face Recognition
- Additive Margin Softmax for Face Verification or CosFace: Large Margin Cosine Loss for Deep Face Recognition
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition
- Focal Loss for Dense Object Detection
If your goal is to reproduce the results in the original paper, please use the official codes:
- Large Margin Softmax Loss in ICML 2016
- Angular Softmax Loss in CVPR 2017
- Additive Margin Softmax
- ArcFace: Additive Angular Margin Loss
- Focal Loss in ICCV 2017
For using these Ops on your own machine:
-
copy the header file "cuda_config.h" from "your_python_path/site-packages/external/local_config_cuda/cuda/cuda/cuda_config.h" to "your_python_path/site-packages/tensorflow/include/tensorflow/stream_executor/cuda/cuda_config.h".
-
run the following script:
mkdir build
cd build && cmake ..
make
-
run "test_op.py" and check the numeric errors to test your install
-
follow the below codes snippet to integrate this Op into your own code:
- For Large Margin Softmax Loss:
op_module = tf.load_op_library(so_lib_path) large_margin_softmax = op_module.large_margin_softmax @ops.RegisterGradient("LargeMarginSoftmax") def _large_margin_softmax_grad(op, grad, _): '''The gradients for `LargeMarginSoftmax`. ''' inputs_features = op.inputs[0] inputs_weights = op.inputs[1] inputs_labels = op.inputs[2] cur_lambda = op.outputs[1] margin_order = op.get_attr('margin_order') grads = op_module.large_margin_softmax_grad(inputs_features, inputs_weights, inputs_labels, grad, cur_lambda[0], margin_order) return [grads[0], grads[1], None, None] var_weights = tf.Variable(initial_value, trainable=True, name='lsoftmax_weights') result = large_margin_softmax(features, var_weights, labels, global_step, 4, 1000., 0.000025, 35., 0.) loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=result[0]))
- For Angular Softmax Loss:
op_module = tf.load_op_library(so_lib_path) angular_softmax = op_module.angular_softmax @ops.RegisterGradient("AngularSoftmax") def _angular_softmax_grad(op, grad, _): '''The gradients for `AngularSoftmax`. ''' inputs_features = op.inputs[0] inputs_weights = op.inputs[1] inputs_labels = op.inputs[2] cur_lambda = op.outputs[1] margin_order = op.get_attr('margin_order') grads = op_module.angular_softmax_grad(inputs_features, inputs_weights, inputs_labels, grad, cur_lambda[0], margin_order) return [grads[0], grads[1], None, None] var_weights = tf.Variable(initial_value, trainable=True, name='asoftmax_weights') normed_var_weights = tf.nn.l2_normalize(var_weights, 1, 1e-10, name='weights_normed') result = angular_softmax(features, normed_var_weights, labels, global_step, 4, 1000., 0.000025, 35., 0.) loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=result[0]))
- For others just refer to this script.
All the codes was tested under TensorFlow 1.6, Python 3.5, Ubuntu 16.04 with CUDA 8.0. The outputs of these Ops in C++ had been compared with the original caffe codes' outputs, and the bias could be ignored. The gradients of this Op had been checked using tf.test.compute_gradient_error and tf.test.compute_gradient. While the others are implemented following the official implementation in Python Ops.
If you encountered some linkage problem when generating or loading *.so, you are highly recommended to read this section in the official tourial to make sure you were using the same C++ ABI version.
Any contributions to this repo is welcomed.
MIT License