Home

Awesome

Hypernymy Understanding Evaluation of Text-to-Image Models via WordNet Hierarchy

This repository contains the official implementation of the In-Subtree Probability and the Subtree Coverage Score, two metrics for measuring the hypernymy capabilities in text-to-image generation. Also, this repository contains the code of experiments from the corresponding paper.

<p align="center"> <img src="resources/sample_tree.png" width="80%" /> </p>

Installation

  1. Clone the repository.
  2. pip install -r requirements.txt
  3. pip install -e .
  4. If you want to evaluate GLIDE, install it using the tutorial from the official repository.

How to evaluate a model

For further details on how to run the scripts, please read the absl app flags.

Structure of the repository

Scripts

Notebooks

For further details on how to run the scripts, please read the absl app flags.