【备忘】CogVLM2环境搭建2 —— 使用问题，triton无法编译

原创

已于 2024-10-09 18:52:23 修改 · 1.7k 阅读

15 ·

CC 4.0 BY-SA版权

文章标签：

#人工智能 #python

于 2024-07-12 14:43:53 首次发布

背景

如果想看搭建基础环境的，请参考第一篇，虽然环境跑起来了，但是第一通对话就不成功，显示如下问题

/usr/bin/ld: 找不到 -lcuda: 没有那个文件或目录
collect2: error: ld returned 1 exit status
Traceback (most recent call last):
  File "<string>", line 21, in rotary_kernel
KeyError: ('2-.-0-.-0-83ca8b715a9dc5f32dc1110973485f64-d6252949da17ceb5f3a278a70250af13-1af5134066c618146d2cd009138944a0-e177f815e3215f52f6b30ce0392b4ae7-3498c340fd4b6ee7805fd54b882a04f5-e1f133f98d04093da2078dfc51c36b72-b26258bf01f839199e39d64851821f26-d7c06e3b46e708006c15224aac7a1378-f585402118c8a136948ce0a49cfe122c', (torch.bfloat16, torch.bfloat16, torch.bfloat16, torch.bfloat16, None, 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32'), (128, False, False, False, False, 4), (True, True, True, True, (False,), (True, False), (False, False), (True, False), (True, False), (False, False), (False, False), (True, False), (True, False), (True, False), (False, True), (True, False), (True, False), (True, False), (False, True)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/think/CogVLM2-main/basic_demo/cli_demo.py", line 102, in <module>
    outputs = model.generate(**inputs, **gen_kwargs)
  File "/home/think/miniconda3/envs/cogvlm2/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/think/miniconda3/envs/cogvlm2/lib/python3.10/site-packages/transformers/generation/utils.py", line 1622, in generate
    result = self._sample(
  File "/home/think/miniconda3/envs/cogvlm2/lib/python3.10/site-packages/transformers/generation/utils.py", line 2791, in _sample
    outputs = self(
  File "/home/think/miniconda3/envs/cogvlm2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/think/miniconda3/envs/cogvlm2/lib/python3.10/site-packages/accelerate/hooks.py", line 169, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/home/think/.cache/hu

最低0.47元/天解锁文章