Weaviate部署完全指南：从单机到K8s集群-优快云博客

Weaviate部署完全指南：从单机到K8s集群

【免费下载链接】weaviate Weaviate is an open source vector database that stores both objects and vectors, allowing for combining vector search with structured filtering with the fault-tolerance and scalability of a cloud-native database, all accessible through GraphQL, REST, and various language clients. 项目地址: https://gitcode.com/GitHub_Trending/we/weaviate

概述

Weaviate是一个开源的向量数据库（Vector Database），它能够同时存储对象和向量，支持向量搜索与结构化过滤的结合。本文将为您提供从单机部署到Kubernetes集群部署的完整指南，涵盖各种部署场景和最佳实践。

部署方式对比

部署方式	适用场景	优点	缺点
Docker单机	开发测试、小型项目	简单快速、资源占用少	不适合生产环境
Docker Compose	本地开发、演示环境	多服务编排、配置灵活	单点故障风险
Kubernetes	生产环境、高可用	高可用性、自动扩缩容	配置复杂、资源需求高
Helm Chart	Kubernetes生产部署	标准化部署、版本管理	需要K8s知识

单机部署

Docker快速启动

# 拉取最新镜像
docker pull semitechnologies/weaviate:latest

# 启动Weaviate容器
docker run -d \
  -p 8080:8080 \
  -p 50051:50051 \
  --name weaviate \
  -e PERSISTENCE_DATA_PATH="/var/lib/weaviate" \
  -e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED="true" \
  -e QUERY_DEFAULTS_LIMIT="25" \
  -e DEFAULT_VECTORIZER_MODULE="none" \
  semitechnologies/weaviate:latest

Docker Compose部署

创建docker-compose.yml文件：

version: '3.4'
services:
  weaviate:
    image: semitechnologies/weaviate:latest
    ports:
      - "8080:8080"
      - "50051:50051"
    environment:
      - PERSISTENCE_DATA_PATH="/var/lib/weaviate"
      - AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED="true"
      - QUERY_DEFAULTS_LIMIT="25"
      - DEFAULT_VECTORIZER_MODULE="none"
      - CLUSTER_HOSTNAME="node1"
    volumes:
      - weaviate_data:/var/lib/weaviate
    restart: unless-stopped

volumes:
  weaviate_data:

启动服务：

docker-compose up -d

生产环境部署

Kubernetes集群部署

创建命名空间

apiVersion: v1
kind: Namespace
metadata:
  name: weaviate

部署配置

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: weaviate
  namespace: weaviate
spec:
  serviceName: weaviate
  replicas: 3
  selector:
    matchLabels:
      app: weaviate
  template:
    metadata:
      labels:
        app: weaviate
    spec:
      containers:
      - name: weaviate
        image: semitechnologies/weaviate:latest
        ports:
        - containerPort: 8080
        - containerPort: 50051
        env:
        - name: PERSISTENCE_DATA_PATH
          value: "/var/lib/weaviate"
        - name: AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED
          value: "true"
        - name: CLUSTER_HOSTNAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        volumeMounts:
        - name: data
          mountPath: /var/lib/weaviate
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "standard"
      resources:
        requests:
          storage: 10Gi

创建服务

apiVersion: v1
kind: Service
metadata:
  name: weaviate
  namespace: weaviate
spec:
  selector:
    app: weaviate
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: grpc
    port: 50051
    targetPort: 50051
  type: ClusterIP

高可用配置

集群配置示例

apiVersion: v1
kind: ConfigMap
metadata:
  name: weaviate-config
  namespace: weaviate
data:
  config.yaml: |
    persistence:
      data_path: /var/lib/weaviate
    cluster:
      enabled: true
      hostname: $(HOSTNAME)
      join: weaviate-0.weaviate.weaviate.svc.cluster.local
    authentication:
      anonymous_access:
        enabled: true

监控与运维

健康检查配置

livenessProbe:
  httpGet:
    path: /v1/.well-known/ready
    port: 8080
  initialDelaySeconds: 30
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /v1/.well-known/ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5

资源限制

resources:
  requests:
    memory: "2Gi"
    cpu: "1"
  limits:
    memory: "4Gi"
    cpu: "2"

数据持久化策略

mermaid

安全配置

TLS证书配置

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: weaviate-ingress
  namespace: weaviate
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - weaviate.example.com
    secretName: weaviate-tls
  rules:
  - host: weaviate.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: weaviate
            port:
              number: 8080

认证配置

env:
- name: AUTHENTICATION_APIKEY_ENABLED
  value: "true"
- name: AUTHENTICATION_APIKEY_ALLOWED_KEYS
  value: "your-secret-api-key"
- name: AUTHENTICATION_APIKEY_USERS
  value: "user1,user2"

备份与恢复

数据备份策略

# 创建备份
kubectl exec -n weaviate weaviate-0 -- \
  curl -X POST http://localhost:8080/v1/backups/filesystem \
  -H "Content-Type: application/json" \
  -d '{"id":"backup-001","include":{"classes":["Article","Image"]}}'

# 恢复备份
kubectl exec -n weaviate weaviate-0 -- \
  curl -X POST http://localhost:8080/v1/backups/filesystem/backup-001/restore \
  -H "Content-Type: application/json"

性能优化

内存配置优化

env:
- name: MEMORY_LIMIT
  value: "4G"
- name: DISK_CACHE_SIZE
  value: "2G"
- name: VECTOR_CACHE_MAX_OBJECTS
  value: "1000000"

查询优化配置

env:
- name: QUERY_MAXIMUM_RESULTS
  value: "10000"
- name: QUERY_DEFAULTS_LIMIT
  value: "100"
- name: QUERY_NESTED_CROSS_REFERENCE_LIMIT
  value: "100"

故障排除

常见问题解决

问题现象	可能原因	解决方案
启动失败	端口冲突	检查端口8080和50051是否被占用
内存不足	资源限制	增加内存限制或调整配置
节点无法加入集群	网络配置	检查网络策略和DNS解析
数据丢失	存储问题	检查持久化卷配置

日志查看

# 查看Pod日志
kubectl logs -n weaviate weaviate-0 -f

# 查看集群状态
kubectl exec -n weaviate weaviate-0 -- \
  curl http://localhost:8080/v1/nodes

总结

Weaviate提供了灵活的部署选项，从简单的单机部署到复杂的Kubernetes集群部署。选择适合您业务需求的部署方式，并遵循本文中的最佳实践，可以确保系统的稳定性、性能和可维护性。

记住定期备份数据、监控系统性能，并根据业务增长及时调整资源配置。通过合理的部署架构和运维策略，Weaviate能够为您的AI应用提供可靠的向量数据库服务。

创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考