【API调用gpt-4 (vision-preview)】基于微软的Azure OpenAI API

微软的Azure页面 ： https://learn.microsoft.com/zh-cn/azure/ai-services/openai/concepts/models
调用代码：https://learn.microsoft.com/zh-cn/azure/ai-services/openai/how-to/switching-endpoints
openai说明: https://platform.openai.com/docs/guides/vision

一、服务器区域选择与购买 (略)

不同区域的服务器开通不同模型 美国西部

二、上传本地图片解析

先安装openai

pip install -U openai

代码 + 自己api

api_key=“yourkey”
azure_endpoint=“xxxx/chat/completions?api-version=2023-07-01-preview”
api_version=“2023-12-01-preview”,

'''
https://platform.openai.com/docs/guides/vision
https://learn.microsoft.com/zh-cn/azure/ai-services/openai/concepts/models
https://learn.microsoft.com/zh-cn/azure/ai-services/openai/how-to/chatgpt?tabs=python&pivots=programming-language-chat-completions
https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision
'''

from openai import AzureOpenAI
api_key="yourkey"
import base64


azure_endpoint="xxxx/chat/completions?api-version=2023-07-01-preview"
client = AzureOpenAI(
    api_key=api_key,
    api_version="2023-12-01-preview",
    azure_endpoint=azure_endpoint
)

# Function to encode the image
def encode_image(image_path):
  with open(image_path, "rb") as image_file:
    return base64.b64encode(image_file.read()).decode('utf-8')
  


def request_base64_gpt4(image_path):
  base64_image=encode_image(image_path)
  response = client.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=[
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "这个是chibi的僵尸题材，生成prompt，以便用来进行text2img的模型训练，先输出中文描述，再输出对应的应为描述"},
          {
            "type": "image_url",
            "image_url": {
              "url": f"data:image/jpeg;base64,{base64_image}",
            },
          },
        ],
      }
    ],
    max_tokens=300,
  )
  print("response",response)
  print(response.choices[0])

if  __name__ == "__main__":
  request_base64_gpt4("test.png")

输入图片

返回值

这是一个以chibi风格画的僵尸题材插图。画面中的僵尸角色是一只卡通化的狐狸，它有着白紫相间的毛发，头上戴着一个大蝴蝶结，眼睛是闪亮的蓝色。它身穿一件粉蓝色的和服，和服上有粉色的花朵装饰。它的手臂下垂，手掌朝上，似乎在展示一个暗紫色的瓶子，瓶子上系着一个粉色的蝴蝶结。背景是深紫色，上方有一些红色的液体滴落

filter_results={‘hate’: {‘filtered’: False, ‘severity’: ‘safe’}, ‘self_harm’: {‘filtered’: False, ‘severity’: ‘safe’}, ‘sexual’: {‘filtered’: False, ‘severity’: ‘safe’}, ‘violence’: {‘filtered’: False, ‘severity’: ‘safe’}})], created=1709793219, model=‘gpt-4’, object=‘chat.completion’, system_fingerprint=None, usage=CompletionUsage(completion_tokens=300, prompt_tokens=820,
total_tokens=1120), prompt_filter_results=[{‘prompt_index’: 0, ‘content_filter_results’: {‘hate’: {‘filtered’: False, ‘severity’: ‘safe’}, ‘self_harm’: {‘filtered’: False, ‘severity’: ‘safe’}, ‘sexual’: {‘filtered’: False, ‘severity’: ‘safe’}, ‘violence’: {‘filtered’: False, ‘severity’: ‘safe’}}}])
Choice(finish_reason=‘length’, index=0, logprobs=None, message=ChatCompletionMessage(content=‘中文描述：这是一个以chibi风格画的僵尸题材插图。画面中的僵尸角色是一只卡通化的狐狸，它有着白紫
相间的毛发，头上戴着一个大蝴蝶结，眼睛是闪亮的蓝色。它身穿一件粉蓝色的和服，和服上有粉色的花朵装饰。它的手臂下垂，手掌朝上，似乎在展示一个暗紫色的瓶子，瓶子上系着一个粉色的蝴蝶结。背景是深
紫色，上方有一些红色的液体滴落。\n\n英文描述：This is a chibi-style zombie-themed illustration. The zombie character in the picture is a cartoonized fox with white and purple fur and a big bow on its head, with shiny blue eyes. It is wearing a light blue kimono with pink flower decorations. Its arms are drooping, palms facing up, seemingly showing off a dark purple bottle tied with a pink bow. The background is dark’, role=‘assistant’, function_call=None, tool_calls=None), content_filter_results={‘hate’: {‘filtered’: False, ‘severity’: ‘safe’}, ‘self_harm’:
{‘filtered’: False, ‘severity’: ‘safe’}, ‘sexual’: {‘filtered’: False, ‘severity’: ‘safe’}, ‘violence’: {‘filtered’: False, ‘severity’: ‘safe’}})

参考代码，GPT4识别图片，并中文回复

prompt=“What’s in this image? 并使用中文回答”
需要解析的远程图片

完整代码

from openai import AzureOpenAI
api_key="your_key"
azure_endpoint="your_model_url"
client = AzureOpenAI(
    api_key=api_key,
    api_version="2023-12-01-preview",
    azure_endpoint=azure_endpoint
)

response = client.chat.completions.create(
  model="gpt-4-vision-preview",
  messages=[
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "What’s in this image? 并使用中文回答"},
        {
          "type": "image_url",
          "image_url": {
            "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
          },
        },
      ],
    }
  ],
  max_tokens=300,
)

print(response.choices[0])

回应

这张图片是一个木制的步道穿过一片绿色的草地，远处有一些树木，天空是蓝色的，有一些白云。

Choice(finish_reason=None, index=0, logprobs=None, message=ChatCompletionMessage(
content='这张图片是一个木制的步道穿过一片绿色的草地，远处有一些树木，天空是蓝色的，有一些白云。', role='assistant', function_call=None, tool_calls=None), 
finish_details={'type': 'stop', 'stop': '<|fim_suffix|>'}, 
content_filter_results={'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}})