A Decoding Algorithm Based on Directed Acyclic Transformers for Length-Control Summarization

This repo contails code, model outputs, training/evaluation scripts, and additional results of our paper.

Replication

Environment setup

Our code based on the FairSeq library. Our code is written in PyTorch, and uses CUDA for GPU acceleration. To install the required packages, run the following command:

# Under `code` dir
bash env_setup.sh

Data

Our data is directly obtained from NAUS, here we only need the gigaref dataset. To download the data, run the following command:

gdown  https://drive.google.com/uc\?id\=1H0csbbv-1L5In9ebJjhk1GamrN8ysnbZ

After downloading the data, unzip it and put it under the datasets/data folder. Where you will have a datasets/data/gigaword_ref folder.

To preprocess the data, run the following command:

bash datasets/preprocess.sh

Where a binarized version of the data will be saved under the code/data-bin/gigaword_ref folder.

Training

The training is conduced under the code folder, where the trainer_with_config.sh file will convert a config file under code/config_summary to a FairSeq training command.

Training DAT model

# Under `code` dir
bash trainer_with_config.sh config_summary/train/dat.yaml

Training Seq-MAP model

NOTE: The Seq-MAP method is mainly implemented here.

# Under `code` dir
bash trainer_with_config.sh config_summary/reranker/dat_reranker_decoder_rouge_20_5.ymal

Note that you need change the finetune_from_model in the configure file according to the path of a trained DAT model.

Unfortunately, we cannot provide the trained DAT model without comprising annonymity. This is because of the size limit of Github.

Under code/config_summary, we also provide the configure files for training the baseline models, different settings of the Seq-MAP, and ablation studies.

Evaluation

# Under `code` dir
checkpoint_path=path_to_the_trained_model
result_path=path_to_save_the_results
ratio=0.25
decoding_algorithm=seq_map 
K_m=20
K_v=5

bash eval_rouge.sh $checkpoint_path $result_path $ratio $decoding_algorithm $K_m $K_v 

The supported decoding algorithms are: seq_map, seq_map_no_rerank, joint_map, ctc, at.

ratio is the length ratio of the summary to the source text. /