[抽取式文本摘要 ACL2019]Searching for Effective Neural Extractive Summarization: What Works and What’s Next

Zhong et al., (2019)ACL.

 

模块

sentence encoder: sentence representations 选什么基本没区别

document encoder: contextualized sentence representations

decoder:  extract a subset of sentences

    Sequence Labeling (SeqLab) non auto-regressive decoder

    Pointer Network (Pointer)  auto-regressive decoder

pre-trained word representations 

    context independent word embeddings ：Word2Vec/GLOVE

    contextualized word embeddings ：ELMo/BERT

 

结论

1. 带有自回归decoder的模型（Pointer）更好。transformer不容易过拟合
2. 句子位置信息很重要（positional embedding）
3. 无监督预训练模型更好（BERT）
4. 强化学习可以进一步提升

   LSTM + PN + BERT（feature-based）+ RL