Espnet ASR-an4 Dataset-Inference with quantization

Espnet Installation

See here

An4 Dataset

1. an4 dataset can not be download from CMU offical website, but we can access it from this github repo
2. git clone https://github.com/kavuri/an4.git and delete .git dir from /an4. Then, compress /an4 to an4_sphere.tar.gz and put it into espnet/egs2/an4/asr1/downloads (you might need to create the /downloads dir)

ASR Inference Experiment with An4 Dataset

1. find a pretrained model in here with keywords like espnet, asr, en, etc.
2. cd espnet/egs2/an4/asr1 & ./run.sh --skip_train true --download_model <model-name>
   In which --skip_train true is to skip the trainning stage and --download_model <model-name> is to use the pretrained model you found in the last step.
   eg. ./run.sh --skip_train true --download_model espnet/Shinji_Watanabe_spgispeech_asr_train_asr_conformer6_n_fft512_hop_length256_raw_en_bpe5000_valid.acc.ave

Quantization

To enable model quantization

1. vim espnet/espnet2/bin/asr_inference.py & add following codes in line 89

   dynamic_quantize = True
   if dynamic_quantize:
       # q_config = {torch.nn.Linear,torch.nn.LSTM,torch.nn.GRU}
       quantize_dtype = torch.qint8
       asr_model = torch.quantization.quantize_dynamic(asr_model, None, dtype=quantize_dtype)
   

   which enables dynamic quantization among the asr model

2. you can also check the asr model in the runtime log by adding

   logging.info(f"ASR model: {asr_model}")