Awesome

Gazeta dataset

Paper: Dataset for Automatic Summarization of Russian News

Download

Dropbox:

• v1, Full dataset: gazeta_jsonl.tar.gz
• v1, Train: gazeta_train.jsonl
• v1, Validation: gazeta_val.jsonl
• v1, Test: gazeta_test.jsonl
• v1, Spacy analyses: gazeta_v1_spacy.tar.gz
• v1, Raw version without any cleaning (not recommended): gazeta_raw.txt
• v1, Train with oracle summaries: gazeta_train_oracle.jsonl
• v1, Validation with oracle summaries: gazeta_val_oracle.jsonl
• v1, Test with oracle summaries: gazeta_test_oracle.jsonl
• v1, Preprocessed data for mBART: gazeta_data_mbart_600_160_v2.tar.gz

UPDATE:

• v2, Full dataset: gazeta_jsonl_v2.tar.gz
• v2, Spacy analyses: gazeta_v2_spacy.tar.gz
• v2, Preprocessed data for mBART: gazeta_v2_data_mbart_600_160_v2.tar.gz
• v2/v1 diff: gazeta_new.jsonl
• v2/v1 raw diff without any clenaning: gazeta_raw_new.jsonl

Other sources:

• GitHub: https://github.com/IlyaGusev/gazeta/releases/tag/2.0
• Kaggle: https://www.kaggle.com/phoenix120/gazeta-summaries
• Huggingface: https://huggingface.co/datasets/IlyaGusev/gazeta

Trained MBART model:

https://huggingface.co/IlyaGusev/mbart_ru_sum_gazeta

Additional notes

• Legal basis for distribution of the dataset:
  • Gazeta Terms of Use, paragraph 2.1.2.
  • Russian Civil Code 1274, part 1
  • All rights belong to "www.gazeta.ru". This dataset can be removed at the request of the copyright holder. Usage of this dataset is possible only on a non-commercial basis.
• Cleaning:
• Data analysis:
• Summarization methods:
• Other Russian summarization datasets:
  • Russian part of XL-Sum, parsed from www.bbc.com/russian, 77803 samples
  • Russian part of MLSUM, parsed from www.mk.ru, 27063 samples
  • Russian part of WikiLingua, parsed from WikiHow, 52928 samples

Contacts

• Telegram: @YallenGusev

Citation

@InProceedings{Gusev2020gazeta,
    author="Gusev, Ilya",
    title="Dataset for Automatic Summarization of Russian News",
    booktitle="Artificial Intelligence and Natural Language",
    year="2020",
    publisher="Springer International Publishing",
    address="Cham",
    pages="{122--134}",
    isbn="978-3-030-59082-6",
    doi={10.1007/978-3-030-59082-6_9}
}