Awesome

Simulated+Unsupervised (S+U) learning in TensorFlow

NYU Hand Dataset

Another TensorFlow implementation of Learning from Simulated and Unsupervised Images through Adversarial Training.

Thanks to TaeHoon Kim, I was able to run simGAN that generates refined synthetic eye dataset.
This is just another version of his code that can generate NYU hand datasets.

The structure of the refiner/discriminator networks are changed as it is described in the Apple paper.
The only code added in this version is ./data/hand_data.py.
Rest of the code runs in the same way as the original version.
To set up the environment(or to run UnityEyes dataset), please follow the instructions in this link.

###Notes -NYU hand dataset is preprocessed(e.g. background removed)
-Image size set to 128x128
-Buffer/Batch size was reduced due to memory issues
-Changed the size of the refiner/discriminator network

##Results

Given these synthetic images,

NYU_hand_synt_1 NYU_hand_synt_2 NYU_hand_synt_3 NYU_hand_synt_4 NYU_hand_synt_5 NYU_hand_synt_6

###Test 1

'lambda=0.1' with 'optimizer=sgd' after ~10k steps.

NYU_hand_ref_1 NYU_hand_ref_2 NYU_hand_ref_3 NYU_hand_ref_4 NYU_hand_ref_5 NYU_hand_ref_6

Discriminator Loss
scalar_d_result_1

Refiner Loss
scalar_r_result_1

###Test 2

'lambda=0.5' with 'optimizer=sgd' after ~10k steps.

NYU_hand_ref_7 NYU_hand_ref_8 NYU_hand_ref_9 NYU_hand_ref_10 NYU_hand_ref_11 NYU_hand_ref_12

scalar_result_2

###Test 3

'lambda=1.0' with 'optimizer=sgd' after ~10k steps.

NYU_hand_ref_13 NYU_hand_ref_14 NYU_hand_ref_15 NYU_hand_ref_16 NYU_hand_ref_17 NYU_hand_ref_18

scalar_result_3

##Summary -The result clearly shows that the refined images look more like the ones in the real dataset as the value of lambda gets smaller.
-The background of the refined images are darker. This is because some of the real image backgrounds were not properly removed while obtaining the arm hand segments. When the refiner tries to make refined synthetic images, it also changes the colour of the background to make it look like the ones in the real dataset.

Author

Seung Shin / @shinseung428