Home

Awesome

Open Source Love

Intro

This repository contains a CBIR (content-based image retrieval) system

Extract query image's feature, and retrieve similar ones from image database

<img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='https://github.com/pochih/CBIR/blob/img/CBIR.png' padding='5px' height="300px"></img>

<h6><a href='https://winstonhsu.info/2017f-mmai/'>Image src</a></h6>

Part1: Feature Extraction

In this system, I implement several popular image features:

all features are modulized

Feature Fusion

Some features are not robust enough, and turn to feature fusion

Dimension Reduction

The curse of dimensionality told that vectors in high dimension will sometimes lose distance property

Part2: Evaluation

CBIR system retrieves images based on feature similarity

Robustness of system is evaluated by MMAP (mean MAP), the evaluation formulas is refer to <a href='http://web.stanford.edu/class/cs276/handouts/EvaluationNew-handout-1-per.pdf' target="_blank">here</a>

Implementation of evaluation can found at evaluate.py

My database contains 25 classes, each class with 20 images, 500 images in total, depth=K will return top-K images from database

MethodcolordaisyedgegaborHOGvgg19resnet152
Mean MAP (depth=10)0.6140.4680.3010.3460.4500.9140.944

Part3: Image Retrieval (return top 5 of each method)

Let me show some results of the system

query1 - women dress

query <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query1-women_dress/query1.jpg' padding='5px' height="80px"></img>

color <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query1-women_dress/query1-color.jpg' padding='5px' height="80px"></img>

daisy <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query1-women_dress/query1-daisy.jpg' padding='5px' height="80px"></img>

edge <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query1-women_dress/query1-edge.jpg' padding='5px' height="80px"></img>

gabor <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query1-women_dress/query1-gabor.jpg' padding='5px' height="80px"></img>

HOG <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query1-women_dress/query1-hog.jpg' padding='5px' height="80px"></img>

VGG19 <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query1-women_dress/query1-vgg.jpg' padding='5px' height="80px"></img>

Resnet152 <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query1-women_dress/query1-res.jpg' padding='5px' height="80px"></img>

query2 - orange

query <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query2-orange/query2.jpg' padding='5px' height="80px"></img>

color <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query2-orange/query2-color.jpg' padding='5px' height="80px"></img>

daisy <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query2-orange/query2-daisy.jpg' padding='5px' height="80px"></img>

edge <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query2-orange/query2-edge.jpg' padding='5px' height="80px"></img>

gabor <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query2-orange/query2-gabor.jpg' padding='5px' height="80px"></img>

HOG <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query2-orange/query2-hog.jpg' padding='5px' height="80px"></img>

VGG19 <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query2-orange/query2-vgg.jpg' padding='5px' height="80px"></img>

Resnet152 <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query2-orange/query2-res.jpg' padding='5px' height="80px"></img>

query3 - NBA jersey

query <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query3-nba_jersey/query3.jpg' padding='5px' height="80px"></img>

color <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query3-nba_jersey/query3-color.jpg' padding='5px' height="80px"></img>

daisy <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query3-nba_jersey/query3-daisy.jpg' padding='5px' height="80px"></img>

edge <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query3-nba_jersey/query3-edge.jpg' padding='5px' height="80px"></img>

gabor <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query3-nba_jersey/query3-gabor.jpg' padding='5px' height="80px"></img>

HOG <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query3-nba_jersey/query3-hog.jpg' padding='5px' height="80px"></img>

VGG19 <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query3-nba_jersey/query3-vgg.jpg' padding='5px' height="80px"></img>

Resnet152 <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query3-nba_jersey/query3-res.jpg' padding='5px' height="80px"></img>

query4 - snack

query <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query4-snack/query4.jpg' padding='5px' height="80px"></img>

color <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query4-snack/query4-color.jpg' padding='5px' height="80px"></img>

daisy <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query4-snack/query4-daisy.jpg' padding='5px' height="80px"></img>

edge <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query4-snack/query4-edge.jpg' padding='5px' height="80px"></img>

gabor <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query4-snack/query4-gabor.jpg' padding='5px' height="80px"></img>

HOG <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query4-snack/query4-hog.jpg' padding='5px' height="80px"></img>

VGG19 <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query4-snack/query4-vgg.jpg' padding='5px' height="80px"></img>

Resnet152 <img align='center' style="border-color:gray;border-width:2px;border-style:dashed" src='result/retrieval_result/query4-snack/query4-res.jpg' padding='5px' height="80px"></img>

Part4: Usage of Repository

If you are interesting with the results, and want to try your own images,

Please refer to USAGE.md

The details are written inside.

Author

Po-Chih Huang / @pochih