FireRedASR 模型部署及推理

1.源代码安装

git clone https://github.com/FireRedTeam/FireRedASR.git
cd FireRedASR
conda create --name fireredasr python=3.10
conda activate fireredasr
pip install -r requirements.txt

2.下载预训练模型，并将其放置在 pretrained_models 目录中

modelscope download --model pengzhendong/FireRedASR-AED-L


modelscope download --model pengzhendong/FireRedASR-LLM-L


modelscope download --model Qwen/Qwen2-7B-Instruct

下载后的目录如下：

# 下载后模型路径示例
pretrained_models/
├── FireRedASR-AED-L/
│   ├── model.pth.tar
│   ├── cmvn.ark
│   └── dict.txt
└── FireRedASR-LLM-L/
    ├── model.pth.tar
    ├── asr_encoder.pth.tar
    ├── cmvn.ark
    └── Qwen2-7B-Instruct/  # 按照上述步骤单独下载

3.模型推理

基本的 FireRedASR-AED 使用

from fireredasr.models.fireredasr import FireRedAsr
 
# 初始化模型
model = FireRedAsr.from_pretrained("aed", "pretrained_models/FireRedASR-AED-L")
 
# 准备输入
batch_uttid = ["sample_id"]
batch_wav_path = ["path/to/audio.wav"]
 
# 转录
results = model.transcribe(
    batch_uttid,
    batch_wav_path,
    {
        "use_gpu": 1,
        "beam_size": 3,
        "nbest": 1,
        "softmax_smoothing": 1.25,
        "aed_length_penalty": 0.6,
        "eos_penalty": 1.0
    }
)
 
# 访问结果
for result in results:
    print(f"语句 ID: {result['uttid']}")
    print(f"转录文本: {result['text']}")
    print(f"实时因子: {result['rtf']}")

基本的 FireRedASR-LLM 使用

from fireredasr.models.fireredasr import FireRedAsr
 
# 初始化模型
model = FireRedAsr.from_pretrained("llm", "pretrained_models/FireRedASR-LLM-L")
 
# 准备输入
batch_uttid = ["sample_id"]
batch_wav_path = ["path/to/audio.wav"]
 
# 转录
results = model.transcribe(
    batch_uttid,
    batch_wav_path,
    {
        "use_gpu": 1,
        "beam_size": 3,
        "repetition_penalty": 3.0,
        "llm_length_penalty": 1.0,
        "temperature": 1.0
    }
)
 
# 访问结果
for result in results:
    print(f"语句 ID: {result['uttid']}")
    print(f"转录文本: {result['text']}")
    print(f"实时因子: {result['rtf']}")

4.在推理过程中遇到的坑

File "/FireRedASR/llm_test.py", line 12, in <module>
    model = FireRedAsr.from_pretrained("llm", "pretrained_models/FireRedASR-LLM-L")
  File "/FireRedASR/fireredasr/models/fireredasr.py", line 31, in from_pretrained
    model, tokenizer = load_firered_llm_model_and_tokenizer(
  File "/FireRedASR/fireredasr/models/fireredasr.py", line 118, in load_firered_llm_model_and_tokenizer
    package = torch.load(model_path, map_location=lambda storage, loc: storage)
  File "/.conda/envs/fireredasr/lib/python3.10/site-packages/torch/serialization.py", line 1469, in load
    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
_pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint.
	(1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
	(2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
	WeightsUnpickler error: Unsupported global: GLOBAL argparse.Namespace was not an allowed global by default. Please use `torch.serialization.add_safe_globals([Namespace])` or the `torch.serialization.safe_globals([Namespace])` context manager to allowlist this global if you trust this class/function.

Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.

解决方法：

        使用 add_safe_globals
PyTorch 提供了 torch.serialization.add_safe_globals 函数，允许您显式地将特定类或函数添加到安全全局对象列表中，从而在保持 weights_only=True 的同时加载所需的类型16。

找到并编辑文件：打开 fireredasr.py 文件，在文件顶部或其他合适的位置添加导入和安全全局设置。

修改代码：在调用 torch.load 之前，添加 argparse.Namespace 到安全全局列表：

python
import argparse
import torch

# 在 torch.load 调用之前添加以下代码
torch.serialization.add_safe_globals([argparse.Namespace])

# 然后原有的 torch.load 调用可以保持不变（或显式设置 weights_only=True）
package = torch.load(model_path, map_location=lambda storage, loc: storage, weights_only=True)