Qwen2-VL部署pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for

Qwen2-VL部署报错及解决办法

原创已于 2024-09-12 11:23:16 修改 · 3.7k 阅读

9 ·

CC 4.0 BY-SA版权

文章标签：

#人工智能 #数据库 #算法

于 2024-09-12 11:14:31 首次发布

部署阿里云团队的Qwen2-VL大模型发生报错如下：

模型为GPTQ-int8 计算卡为P40Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int8 · Hugging Facehttps://huggingface.co/Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int8

虚拟环境各py包如下：

(qwen2vl) PS C:\Users\Administrator> pip list
Package                       Version
----------------------------- ------------
accelerate                    0.34.2
aiofiles                      23.2.1
aiohappyeyeballs              2.4.0
aiohttp                       3.10.5
aiosignal                     1.3.1
altair                        5.4.1
annotated-types               0.7.0
anyio                         4.4.0
async-timeout                 4.0.3
attrs                         24.2.0
auto_gptq                     0.7.1+cu118
autoawq                       0.2.6
autoawq_kernels               0.0.7
av                            13.0.0
certifi                       2024.8.30
charset-normalizer            3.3.2
click                         8.1.7
colorama                      0.4.6
coloredlogs                   15.0.1
contourpy                     1.3.0
cycler                        0.12.1
datasets                      2.21.0
dill                          0.3.8
einops                        0.8.0
exceptiongroup                1.2.2
fastapi                       0.114.1
ffmpy                         0.4.0
filelock                      3.13.1
fonttools                     4.53.1
frozenlist                    1.4.1
fsspec                        2024.2.0
gekko                         1.2.1
gradio                        4.29.0
gradio_client                 0.16.1
h11                           0.14.0
httpcore                      1.0.5
httpx                         0.27.2
huggingface-hub               0.24.6
humanfriendly                 10.0
idna                          3.8
importlib_resources           6.4.5
intel-openmp                  2021.4.0
Jinja2                        3.1.3
jsonschema                    4.23.0
jsonschema-specifications     2023.12.1
kiwisolver                    1.4.7
markdown-it-py                3.0.0
MarkupSafe                    2.1.5
matplotlib                    3.9.2
mdurl                         0.1.2
mkl                           2021.4.0
mpmath                        1.3.0
multidict                     6.1.0
multiprocess                  0.70.16
narwhals                      1.6.4
networkx                      3.2.1
ninja                         1.11.1.1
numpy                         1.26.3
optimum                       1.22.0
orjson                        3.10.7
packaging                     24.1
pandas                        2.2.2
peft                          0.12.0
pillow                        10.2.0
pip                           24.2
protobuf                      5.28.0
psutil                        6.0.0
pyarrow                       17.0.0
pydantic                      2.9.1
pydantic_core                 2.23.3
pydub                         0.25.1
Pygments                      2.18.0
pyparsing                     3.1.4
pyreadline3                   3.4.3
python-dateutil               2.9.0.post0
python-multipart              0.0.9
pytz                          2024.2
PyYAML                        6.0.2
qwen-vl-utils                 0.0.2
referencing                   0.35.1
regex                         2024.7.24
requests                      2.32.3
rich                          13.8.1
rouge                         1.0.1
rpds-py                       0.20.0
ruff                          0.6.4
safetensors                   0.4.5
semantic-version              2.10.0
sentencepiece                 0.2.0
setuptools                    72.1.0
shellingham                   1.5.4
six                           1.16.0
sniffio                       1.3.1
starlette                     0.38.5
sympy                         1.12
tbb                           2021.13.1
tokenizers                    0.19.1
tomlkit                       0.12.0
torch                         2.3.1+cu118
torchaudio                    2.3.1+cu118
torchvision                   0.18.1+cu118
tqdm                          4.66.5
transformers                  4.45.0.dev0
transformers-stream-generator 0.0.4
typer                         0.12.5
typing_extensions             4.12.2
tzdata                        2024.1
urllib3                       2.2.2
uvicorn                       0.30.6
websockets                    11.0.3
wheel                         0.43.0
xxhash                        3.5.0
yarl                          1.11.1
zstandard                     0.23.0

发生报错具体如下:

(qwen2vl) PS D:\3\Qwen2-VL-main> python web_demo_mm.py
Unrecognized keys in `rope_scaling` for 'rope_type'='default': {'mrope_section'}
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\exllama.py:12: UserWarning: AutoAWQ could not load ExLlama kernels extension. Details: DLL load failed while importing exl_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load ExLlama kernels extension. Details: {ex}")
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\exllamav2.py:13: UserWarning: AutoAWQ could not load ExLlamaV2 kernels extension. Details: DLL load failed while importing exlv2_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load ExLlamaV2 kernels extension. Details: {ex}")
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\gemm.py:14: UserWarning: AutoAWQ could not load GEMM kernels extension. Details: DLL load failed while importing awq_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load GEMM kernels extension. Details: {ex}")
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\gemv.py:11: UserWarning: AutoAWQ could not load GEMV kernels extension. Details: DLL load failed while importing awq_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load GEMV kernels extension. Details: {ex}")
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\gemv_fast.py:10: UserWarning: AutoAWQ could not load GEMVFast kernels extension. Details: DLL load failed while importing awq_v2_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load GEMVFast kernels extension. Details: {ex}")
CUDA extension not installed.
CUDA extension not installed.
`Qwen2VLRotaryEmbedding` can now be fully parameterized by passing the model config through the `config` argument. All other arguments will be removed in v4.46
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\transformers\modeling_utils.py:4732: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
  warnings.warn(
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:06<00:00,  2.30s/it] 
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\type_adapter.py", line 270, in _init_core_attrs
    self._core_schema = _getattr_no_parents(self._type, '__pydantic_core_schema__')
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\type_adapter.py", line 112, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 406, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\fastapi\applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\middleware\errors.py", line 187, in __call__
    raise exc
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\middleware\errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\gradio\route_utils.py", line 713, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\gradio\route_utils.py", line 729, in simple_response
    await self.app(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\middleware\exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\starlette\routing.py", line 73, in app
    response = await f(request)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\fastapi\routing.py", line 291, in app
    solved_result = await solve_dependencies(
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\fastapi\dependencies\utils.py", line 639, in solve_dependencies
    ) = await request_body_to_args(  # body_params checked above
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\fastapi\dependencies\utils.py", line 813, in request_body_to_args
    fields_to_extract = get_cached_model_fields(first_field.type_)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\fastapi\_compat.py", line 657, in get_cached_model_fields
    return get_model_fields(model)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\fastapi\_compat.py", line 284, in get_model_fields
    return [
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\fastapi\_compat.py", line 285, in <listcomp>
    ModelField(field_info=field_info, name=name)
  File "<string>", line 6, in __init__
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\fastapi\_compat.py", line 110, in __post_init__
    self._type_adapter: TypeAdapter[Any] = TypeAdapter(
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\type_adapter.py", line 257, in __init__
    self._init_core_attrs(rebuild_mocks=False)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\type_adapter.py", line 135, in wrapped
    return func(self, *args, **kwargs)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\type_adapter.py", line 277, in _init_core_attrs
    self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\type_adapter.py", line 95, in _get_schema
    schema = gen.generate_schema(type_)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 655, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 908, in _generate_schema_inner
    return self._annotated_schema(obj)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 2028, in _annotated_schema
    schema = self._apply_annotations(source_type, annotations)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 2107, in _apply_annotations
    schema = get_inner_schema(source_type)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 2189, in new_handler
    schema = metadata_get_schema(source, get_inner_schema)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 2185, in <lambda>
    lambda source, handler: handler(source)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 2088, in inner_handler
    schema = self._generate_schema_inner(obj)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 929, in _generate_schema_inner
    return self.match_type(obj)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1029, in match_type
    return self._match_generic_type(obj, origin)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1058, in _match_generic_type
    return self._union_schema(obj)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1378, in _union_schema
    choices.append(self.generate_schema(arg))
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 655, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 929, in _generate_schema_inner
    return self.match_type(obj)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1038, in match_type
    return self._unknown_type_schema(obj)
  File "D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\pydantic\_internal\_generate_schema.py", line 558, in _unknown_type_schema
    raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.9/u/schema-for-unknown-type

参考github教程说明：PydanticSchemaGenerationError for class starlette.requests.Request · Issue #2783 · bmaltais/kohya_ss (github.com)https://github.com/bmaltais/kohya_ss/issues/2783

只需要更换一个gradio版本即可。

It's temporary solution but, you can modify the requirements.txt file to downgrade the gradio package to version 3.50.2.

gradio==3.50.2

pip install gradio==3.50.2

最终可以成功正常运行Qwen2-VL

(qwen2vl) PS D:\3\Qwen2-VL-main> python web_demo_mm.py
Unrecognized keys in `rope_scaling` for 'rope_type'='default': {'mrope_section'}
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\exllama.py:12: UserWarning: AutoAWQ could not load ExLlama kernels extension. Details: DLL load failed while importing exl_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load ExLlama kernels extension. Details: {ex}")
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\exllamav2.py:13: UserWarning: AutoAWQ could not load ExLlamaV2 kernels extension. Details: DLL load failed while importing exlv2_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load ExLlamaV2 kernels extension. Details: {ex}")
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\gemm.py:14: UserWarning: AutoAWQ could not load GEMM kernels extension. Details: DLL load failed while importing awq_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load GEMM kernels extension. Details: {ex}")
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\gemv.py:11: UserWarning: AutoAWQ could not load GEMV kernels extension. Details: DLL load failed while importing awq_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load GEMV kernels extension. Details: {ex}")
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\awq\modules\linear\gemv_fast.py:10: UserWarning: AutoAWQ could not load GEMVFast kernels extension. Details: DLL load failed while importing awq_v2_ext: 找不到指定的模块。
  warnings.warn(f"AutoAWQ could not load GEMVFast kernels extension. Details: {ex}")
CUDA extension not installed.
CUDA extension not installed.
`Qwen2VLRotaryEmbedding` can now be fully parameterized by passing the model config through the `config` argument. All other arguments will be removed in v4.46
D:\ProgramData\anaconda3\envs\qwen2vl\lib\site-packages\transformers\modeling_utils.py:4732: FutureWarning: `_is_quantized_training_enabled` is going to be deprecated in transformers 4.39.0. Please use `model.hf_quantizer.is_trainable` instead
  warnings.warn(
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:05<00:00,  1.86s/it]
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
IMPORTANT: You are using gradio version 3.50.2, however version 4.29.0 is available, please upgrade.
--------

关于提示词，需要尽可能的详细，

读取图像，解答这个题干和对应的题目，尝试给出对应的解答过程和答案给我。

12.8252GB约为当前所需的显存

但如果提示词不恰当（过于简略，则可能会导致Qwen2-VL大模型发生误判）最终还有可能会爆显存。