Home

Awesome

ZEN 2.0

ZEN 2.0 is a pre-trained language model with Chinese and Arabic versions. ZEN 2.0 is based on the architecture of ZEN 1.0 with an update and adaptation from the following three aspects:

The structure of ZEN 2.0 is illustrated in the following figure. We elaborate the differences between ZEN 2.0 and ZEN 1.0 at here.  

ZEN_model

Citation

If you use or extend our work, please cite the following paper:

@article{Sinovation2021ZEN2,
  title="{ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders}",
  author={Yan Song, Tong Zhang, Yonggang Wang, Kai-Fu Lee},
  journal={arXiv preprint arXiv:2105.01279},
  year={2021},
}

Quick tour of pre-training and fine-tune using ZEN 2.0

The library comprises several example scripts for conducting Chinese/Arabic NLP tasks:

Examples of pre-training and fine-tune using ZEN 2.0.

Contact information

For help or issues using ZEN 2.0, please submit a GitHub issue.

For personal communication related to ZEN 2.0, please contact Yuanhe Tian (yhtian94@gmail.com).