大模型推理环境安装过程中踩坑记录

1.找不到libsndfile

  File "/.conda/envs/gemma/lib/python3.11/site-packages/soundfile.py", line 181, in <module>
    import _soundfile_data  # ImportError if this doesn't exist
    ^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_soundfile_data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/.conda/envs/gemma/lib/python3.11/site-packages/soundfile.py", line 190, in <module>
    raise OSError('sndfile library not found using ctypes.util.find_library')
OSError: sndfile library not found using ctypes.util.find_library

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "gemma-app1.py", line 83, in predict
    inputs = processor.apply_chat_template(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/gemma/lib/python3.11/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/gemma/lib/python3.11/site-packages/transformers/processing_utils.py", line 1598, in apply_chat_template
    batch_audios.append(load_audio(fname, sampling_rate=mm_load_kwargs["sampling_rate"]))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/gemma/lib/python3.11/site-packages/transformers/audio_utils.py", line 62, in load_audio
    audio = librosa.load(audio, sr=sampling_rate)[0]
            ^^^^^^^^^^^^
  File "/.conda/envs/gemma/lib/python3.11/site-packages/lazy_loader/__init__.py", line 83, in __getattr__
    attr = getattr(submod, name)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/gemma/lib/python3.11/site-packages/lazy_loader/__init__.py", line 82, in __getattr__
    submod = importlib.import_module(submod_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.conda/envs/gemma/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/.conda/envs/gemma/lib/python3.11/site-packages/librosa/core/audio.py", line 10, in <module>
    import soundfile as sf
  File "/.conda/envs/gemma/lib/python3.11/site-packages/soundfile.py", line 212, in <module>
    _snd = _ffi.dlopen(_explicit_libname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: cannot load library 'libsndfile.so': libsndfile.so: cannot open shared object file: No such file or directory
Error generating response: cannot load library 'libsndfile.so': libsndfile.so: cannot open shared object file: No such file or directory

解决方法：

conda install -c conda-forge libsndfile

2.Input type (torch.cuda.FloatTensor) and weight type (CUDABFloat16Type) should be the same

原始代码如下：

inputs = processor.apply_chat_template(
            messages,
            add_generation_prompt=True,
            tokenize=True,
            return_dict=True,
            return_tensors="pt",
        ).to(device)

修改后：

inputs = processor.apply_chat_template(
            messages,
            add_generation_prompt=True,
            tokenize=True,
            return_dict=True,
            return_tensors="pt",
        ).to(device, dtype=torch.bfloat16)