t5-small模型部署

google-t5

https://huggingface.co/docs/transformers/model_doc/t5#inference
https://hf-mirror.com/google-t5/t5-small

本地加载

• 支持的语言：English、French、Romanian、German

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple transformer sentencepiece
pip install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu124

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("/data/models/t5-small")
model = T5ForConditionalGeneration.from_pretrained("/data/models/t5-small")

input_ids = tokenizer("translate English to French: La maison est merveilleuse.| The house is wonderful.|", return_tensors="pt").input_ids
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))