Home

Awesome

Grover

UPDATE, Sept 17 2019. We got into NeurIPS (camera ready coming soon!) and we've made Grover-Mega publicly available without you needing to fill out the form. You can download it using download_model.py.

(aka, code for Defending Against Neural Fake News)

Grover is a model for Neural Fake News -- both generation and detection. However, it probably can also be used for other generation tasks.

Visit our project page at rowanzellers.com/grover, the AI2 online demo, or read the full paper at arxiv.org/abs/1905.12616.

teaser

What's in this repo?

We are releasing the following:

Scroll down 👇 for some easy-to-use instructions for setting up Grover to generate news articles.

Setting up your environment

NOTE: If you just care about making your own RealNews dataset, you will need to set up your environment separately just for that, using an AWS machine (see realnews/.)

There are a few ways you can run Grover:

NOTE: You might be able to get things to work using different hardware. However, it might be a lot of work engineering wise and I don't recommend it if possible. Please don't contact me with requests like this, as there's not much help I can give you.

I used Python3.6 for everything. Usually I set it up using the following commands:

curl -o ~/miniconda.sh -O  https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh  && \
     chmod +x ~/miniconda.sh && \
     ~/miniconda.sh -b -p ~/conda && \
     rm ~/miniconda.sh && \
     ~/conda/bin/conda install -y python=3.6

Then pip install -r requirements-gpu.txt if you're installing on a GPU, or pip install requirements-tpu.txt for TPU.

Misc notes/tips:

Quickstart: setting up Grover for generation!

  1. Set up your environment. Here's the easy way, assuming anaconda is installed: conda create -y -n grover python=3.6 && source activate grover && pip install -r requirements-gpu.txt
  2. Download the model using python download_model.py base
  3. Now generate: PYTHONPATH=$(pwd) python sample/contextual_generate.py -model_config_fn lm/configs/base.json -model_ckpt models/base/model.ckpt -metadata_fn sample/april2019_set_mini.jsonl -out_fn april2019_set_mini_out.jsonl

Congrats! You can view the generations, conditioned on the domain/headline/date/authors, in april2019_set_mini_out.jsonl.

FAQ: What's the deal with the release of Grover?

Our core position is that it is important to release possibly-dangerous models to researchers. At the same time, we believe Grover-Mega isn't particularly useful to anyone who isn't doing research in this area, particularly as we have an online web demo available and the model is computationally expensive. We previously were a bit stricter and limited initial use of Grover-Mega to researchers. Now that several months have passed since we put the paper on arxiv, and since several other large-scale language models have been publicly released, we figured that there is little harm in fully releasing Grover-Mega.

Bibtex

@inproceedings{zellers2019grover,
    title={Defending Against Neural Fake News},
    author={Zellers, Rowan and Holtzman, Ari and Rashkin, Hannah and Bisk, Yonatan and Farhadi, Ali and Roesner, Franziska and Choi, Yejin},
    booktitle={Advances in Neural Information Processing Systems 32},
    year={2019}
}