Intel 平台大模型加速库 OpenVINO 安装试用_openvino 大模型-优快云博客

本文链接：https://blog.youkuaiyun.com/seaneer/article/details/146169487

OpenVINO介绍

英特尔 ® 开放式视觉推理与神经网络优化器（OpenVINO™）工具套件是一款开源工具套件，它能够在保持准确性的同时，以更低的延迟和更高的吞吐量加速人工智能推理，减小模型占用空间，并优化硬件使用。它简化了人工智能开发，以及在计算机视觉、大语言模型（LLM）和生成式人工智能等领域中深度学习的集成。Openvino专门用于优化和部署人工智能推理的半开源的工具包，主要用于对深度推理做优化。Openvino内部集成了Opencv、TensorFlow模块，除此之外它还具有强大的Plugin开发框架，允许开发者在Openvino之上对推理过程做优化。Openvino整体框架为：Openvino前端→ Plugin中间层→ Backend后端。

具体来说，OpenVINO可将Embedding层和Attention计算分配到显卡，能提速约40%。

软件硬件需求

OS

Windows 11, 64-bit
Windows 10, 64-bit
Ubuntu 24.04 long-term support (LTS), 64-bit
Ubuntu 22.04 long-term support (LTS), 64-bit
Ubuntu 20.04 long-term support (LTS), 64-bit
CentOS 7
Red Hat Enterprise Linux (RHEL) 8 and 9, 64-bit

CPU

Intel® Core™ Ultra Series 1 and Series 2 (Windows only)
Intel® Xeon® 6 processor (preview)
Intel Atom® Processor X Series
Intel Atom® processor with Intel® SSE4.2 support
Intel® Pentium® processor N4200/5, N3350/5, N3450/5 with Intel® HD Graphics
6th - 14th generation Intel® Core™ processors
1st - 5th generation Intel® Xeon® Scalable Processors
ARM CPUs with armv7a and higher, ARM64 CPUs with arm64-v8a and higher, Apple® Mac with Apple silicon

GPU

Intel® Arc™ GPU Series
Intel® HD Graphics
Intel® UHD Graphics
Intel® Iris® Pro Graphics
Intel® Iris® Xe Graphics
Intel® Iris® Xe Max Graphics
Intel® Data Center GPU Flex Series
Intel® Data Center GPU Max Series

win11工具链需求

#Build environment components:
Microsoft Visual Studio 2019 or later
CMake 3.16 or higher
Python 3.9-3.12
Intel® HD Graphics Driver required for inference on GPU

安装方法

#创建python虚拟 环境，推荐版本3.10
conda create --n langchain python=3.10
conda activate langchain

# 在虚拟 环境中安装核心包
pip install openvino==2023.2.0

# 安装开发工具包（模型优化等）
pip install openvino-dev[onnx,tensorflow2]==2023.2.0

验证

先下载模型xml和bin文件：

blob:https://github.com/eacb03dd-ddc1-4c93-be07-0dbe4204b985

blob:https://github.com/d6576689-1be1-4e99-80d3-0b1d76e60c20

(langchain) D:\code\trae>python
Python 3.10.0 | packaged by conda-forge | (default, Nov 10 2021, 13:20:59) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import openvino as ov
>>> print(f"OpenVINO版本: {ov.__version__}")  # 应输出 2023.2.0
OpenVINO版本: 2023.2.0-13089-cfd42bd2cb0-HEAD
>>> core = ov.Core()
>>> devices = core.available_devices
>>> print(f"可用硬件设备: {devices}")
可用硬件设备: ['CPU', 'GPU']
>>>

>>> from openvino.runtime import Core
>>> import numpy as np
>>> core = Core()
>>> model = core.read_model("face-detection-adas-0001.xml") 
>>> compiled_model = core.compile_model(model, "CPU")
>>> input_shape = compiled_model.input(0).shape
>>> dummy_input = np.random.randn(*input_shape).astype(np.float32)
>>> results = compiled_model([dummy_input])[0]
>>> print("推理结果形状:", results.shape)  # 应匹配模型输出维度
推理结果形状: (1, 1, 200, 7)