IOPaint API接口详解：RESTful接口调用与二次开发指南-优快云博客

IOPaint API接口详解：RESTful接口调用与二次开发指南

【免费下载链接】IOPaint 项目地址: https://gitcode.com/GitHub_Trending/io/IOPaint

概述

IOPaint是一个基于SOTA AI模型的免费开源图像修复（Inpainting）和扩展（Outpainting）工具。它提供了完整的RESTful API接口，支持多种AI模型和插件，为开发者提供了强大的图像处理能力。本文将深入解析IOPaint的API接口设计、使用方法以及二次开发指南。

API架构设计

IOPaint采用FastAPI框架构建RESTful API，支持WebSocket实时通信，整体架构如下：

mermaid

核心API接口详解

1. 服务配置接口

获取服务器配置

GET /api/v1/server-config

响应示例：

{
  "plugins": [
    {
      "name": "interactive_seg",
      "support_gen_image": false,
      "support_gen_mask": true
    }
  ],
  "modelInfos": [
    {
      "name": "lama",
      "path": "lama",
      "model_type": "inpaint",
      "need_prompt": false
    }
  ],
  "removeBGModel": "briaai/RMBG-1.4",
  "removeBGModels": ["briaai/RMBG-1.4", "u2net", "u2netp"],
  "enableFileManager": true,
  "enableAutoSaving": true,
  "samplers": ["DPM++ 2M", "DPM++ 2M Karras", "Euler", "DDIM"]
}

2. 模型管理接口

获取当前模型信息

GET /api/v1/model

切换模型

POST /api/v1/model
Content-Type: application/json

{
  "name": "runwayml/stable-diffusion-inpainting"
}

3. 图像修复核心接口

执行图像修复

POST /api/v1/inpaint
Content-Type: application/json

{
  "image": "base64编码的图像数据",
  "mask": "base64编码的掩码数据",
  "prompt": "a beautiful sunset",
  "negative_prompt": "blurry, low quality",
  "sd_strength": 0.8,
  "sd_steps": 30,
  "sd_guidance_scale": 7.5,
  "sd_sampler": "DPM++ 2M",
  "sd_seed": 42
}

请求参数详解：

参数	类型	说明	默认值
`image`	string	Base64编码的输入图像	必填
`mask`	string	Base64编码的掩码图像	必填
`prompt`	string	正向提示词	""
`negative_prompt`	string	负向提示词	""
`sd_strength`	float	修复强度(0.0-1.0)	1.0
`sd_steps`	int	扩散步数	50
`sd_guidance_scale`	float	引导尺度	7.5
`sd_sampler`	string	采样器	"UniPC"
`sd_seed`	int	随机种子	42

4. 插件接口

运行插件生成图像

POST /api/v1/run_plugin_gen_image
Content-Type: application/json

{
  "name": "realesrgan",
  "image": "base64编码的图像数据",
  "scale": 2.0
}

运行插件生成掩码

POST /api/v1/run_plugin_gen_mask
Content-Type: application/json

{
  "name": "interactive_seg",
  "image": "base64编码的图像数据",
  "clicks": [[100, 200, 1], [150, 250, 0]]
}

数据模型定义

InpaintRequest 模型

class InpaintRequest(BaseModel):
    image: Optional[str] = Field(None, description="base64 encoded image")
    mask: Optional[str] = Field(None, description="base64 encoded mask")
    prompt: str = Field("", description="Prompt for diffusion models.")
    negative_prompt: str = Field("", description="Negative prompt for diffusion models.")
    sd_strength: float = Field(1.0, description="Strength for diffusion models")
    sd_steps: int = Field(50, description="Steps for diffusion models")
    sd_guidance_scale: float = Field(7.5, description="Guidance scale for diffusion models")
    sd_sampler: str = Field("UniPC", description="Sampler for diffusion model")
    sd_seed: int = Field(42, description="Seed for diffusion model")
    # ... 其他参数

支持的采样器类型

class SDSampler(str, Enum):
    dpm_plus_plus_2m = "DPM++ 2M"
    dpm_plus_plus_2m_karras = "DPM++ 2M Karras"
    dpm_plus_plus_2m_sde = "DPM++ 2M SDE"
    euler = "Euler"
    euler_a = "Euler a"
    ddim = "DDIM"
    lcm = "LCM"

客户端调用示例

Python调用示例

import requests
import base64
from PIL import Image
import io

class IOPaintClient:
    def __init__(self, base_url="http://localhost:8080"):
        self.base_url = base_url
        self.api_url = f"{base_url}/api/v1"
    
    def image_to_base64(self, image_path):
        with open(image_path, "rb") as image_file:
            return base64.b64encode(image_file.read()).decode('utf-8')
    
    def inpaint(self, image_path, mask_path, prompt="", **kwargs):
        image_base64 = self.image_to_base64(image_path)
        mask_base64 = self.image_to_base64(mask_path)
        
        payload = {
            "image": image_base64,
            "mask": mask_base64,
            "prompt": prompt,
            **kwargs
        }
        
        response = requests.post(f"{self.api_url}/inpaint", json=payload)
        if response.status_code == 200:
            return Image.open(io.BytesIO(response.content))
        else:
            raise Exception(f"API调用失败: {response.status_code}")

# 使用示例
client = IOPaintClient()
result_image = client.inpaint(
    "input.jpg", 
    "mask.png",
    prompt="a beautiful landscape",
    sd_strength=0.8,
    sd_steps=30
)
result_image.save("output.jpg")

JavaScript/TypeScript调用示例

import axios from 'axios';

class IOPaintClient {
    constructor(baseURL = 'http://localhost:8080') {
        this.apiURL = `${baseURL}/api/v1`;
    }

    async imageToBase64(file) {
        return new Promise((resolve, reject) => {
            const reader = new FileReader();
            reader.onload = () => resolve(reader.result.split(',')[1]);
            reader.onerror = reject;
            reader.readAsDataURL(file);
        });
    }

    async inpaint(imageFile, maskFile, prompt = '', options = {}) {
        const imageBase64 = await this.imageToBase64(imageFile);
        const maskBase64 = await this.imageToBase64(maskFile);

        const payload = {
            image: imageBase64,
            mask: maskBase64,
            prompt,
            ...options
        };

        try {
            const response = await axios.post(`${this.apiURL}/inpaint`, payload, {
                responseType: 'blob'
            });
            return URL.createObjectURL(response.data);
        } catch (error) {
            console.error('API调用失败:', error);
            throw error;
        }
    }
}

// 使用示例
const client = new IOPaintClient();
const imageInput = document.getElementById('image-input');
const maskInput = document.getElementById('mask-input');

const processImage = async () => {
    const resultUrl = await client.inpaint(
        imageInput.files[0],
        maskInput.files[0],
        'a beautiful sunset',
        {
            sd_strength: 0.8,
            sd_steps: 30,
            sd_guidance_scale: 7.5
        }
    );
    
    const resultImg = document.getElementById('result-image');
    resultImg.src = resultUrl;
};

高级功能接口

ControlNet支持

# 启用ControlNet
payload = {
    "image": image_base64,
    "mask": mask_base64,
    "prompt": "a cat sitting on a chair",
    "enable_controlnet": True,
    "controlnet_method": "lllyasviel/control_v11p_sd15_canny",
    "controlnet_conditioning_scale": 0.4
}

BrushNet支持

# 启用BrushNet
payload = {
    "image": image_base64,
    "mask": mask_base64,
    "enable_brushnet": True,
    "brushnet_method": "lllyasviel/control_v11p_sd15_scribble",
    "brushnet_conditioning_scale": 1.0
}

PowerPaint V2支持

# 启用PowerPaint V2
payload = {
    "image": image_base64,
    "mask": mask_base64,
    "enable_powerpaint_v2": True,
    "powerpaint_task": "object-remove",
    "fitting_degree": 0.8
}

错误处理与状态码

IOPaint API使用标准HTTP状态码：

状态码	说明	处理建议
200	请求成功	正常处理响应数据
400	请求参数错误	检查参数格式和内容
404	资源未找到	检查接口路径和资源ID
422	处理失败	查看错误详情，通常是模型或插件问题
500	服务器内部错误	查看服务器日志获取详细信息

错误响应示例：

{
  "error": "HTTPException",
  "detail": "Image size and mask size not match.",
  "errors": "Image size(512,512) and mask size(256,256) not match."
}

性能优化建议

1. 批量处理优化

# 使用批量处理接口
def batch_process(images_dir, masks_dir, output_dir):
    for image_file in os.listdir(images_dir):
        image_path = os.path.join(images_dir, image_file)
        mask_path = os.path.join(masks_dir, image_file)
        
        if os.path.exists(mask_path):
            result = client.inpaint(image_path, mask_path)
            result.save(os.path.join(output_dir, image_file))

2. 内存优化配置

# 启动时配置低内存模式
iopaint start --model=lama --device=cuda --low-mem --cpu-offload

3. 模型预热

# 预先加载常用模型
def preload_models():
    models_to_preload = ["lama", "runwayml/stable-diffusion-inpainting"]
    for model in models_to_preload:
        requests.post(f"{API_URL}/model", json={"name": model})

二次开发指南

自定义插件开发

from iopaint.plugins.base_plugin import BasePlugin
from PIL import Image
import numpy as np

class CustomPlugin(BasePlugin):
    name = "custom_plugin"
    support_gen_image = True
    support_gen_mask = False
    
    def __init__(self, device, **kwargs):
        super().__init__(device, **kwargs)
        # 初始化自定义模型
    
    def gen_image(self, image_np: np.ndarray, config) -> np.ndarray:
        # 实现图像处理逻辑
        processed_image = self.process_image(image_np)
        return processed_image
    
    def process_image(self, image_np):
        # 自定义图像处理逻辑
        return image_np

# 注册插件
def register_custom_plugin():
    from iopaint.plugins import register_plugin
    register_plugin("custom_plugin", CustomPlugin)

API扩展开发

from fastapi import APIRouter, UploadFile
from iopaint.api import Api

class CustomApi(Api):
    def __init__(self, app, config):
        super().__init__(app, config)
        self.setup_custom_routes()
    
    def setup_custom_routes(self):
        router = APIRouter()
        
        @router.post("/api/v1/custom_endpoint")
        async def custom_endpoint(file: UploadFile):
            # 自定义端点逻辑
            return {"message": "Custom endpoint processed"}
        
        self.app.include_router(router)

部署与运维

Docker部署

FROM pytorch/pytorch:2.1.2-cuda11.8-cudnn8-runtime

WORKDIR /app
COPY . .

RUN pip install -r requirements.txt

EXPOSE 8080
CMD ["python", "main.py", "start", "--model", "lama", "--port", "8080"]

性能监控

# 添加性能监控中间件
import time
from fastapi import Request

@app.middleware("http")
async def add_process_time_header(request: Request, call_next):
    start_time = time.time()
    response = await call_next(request)
    process_time = time.time() - start_time
    response.headers["X-Process-Time"] = str(process_time)
    return response

最佳实践

1. 图像预处理

def preprocess_image(image_path, target_size=512):
    image = Image.open(image_path)
    # 保持宽高比调整大小
    image.thumbnail((target_size, target_size), Image.Resampling.LANCZOS)
    return image

2. 掩码优化

def optimize_mask(mask_path):
    mask = Image.open(mask_path).convert('L')
    # 二值化处理
    mask = mask.point(lambda x: 255 if x > 127 else 0)
    return mask

3. 结果后处理

def postprocess_result(result_image, original_image):
    # 融合处理结果和原图
    result = Image.blend(original_image, result_image, alpha=0.9)
    return result

总结

IOPaint提供了强大而灵活的API接口，支持多种AI模型和插件，为图像修复和处理任务提供了完整的解决方案。通过本文的详细解析，开发者可以：

快速集成：使用提供的客户端示例快速集成到现有系统中
深度定制：基于API接口进行二次开发和功能扩展
性能优化：根据业务需求进行性能调优和资源管理
错误处理：完善的错误处理机制保证系统稳定性

IOPaint的API设计遵循RESTful原则，接口清晰易懂，响应格式规范，是构建图像处理应用的理想选择。无论是简单的物体去除还是复杂的图像生成任务，IOPaint都能提供专业级的解决方案。

下一步建议：

阅读官方文档了解最新功能更新
参与社区讨论获取技术支持
贡献代码推动项目发展
关注模型库更新获取更好的处理效果

【免费下载链接】IOPaint 项目地址: https://gitcode.com/GitHub_Trending/io/IOPaint

创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考