Data Formulator集成测试：端到端测试与CI/CD流水线
项目地址: https://gitcode.com/GitHub_Trending/da/data-formulator 痛点：AI数据可视化工具的测试挑战
你还在为复杂的AI驱动数据可视化工具编写繁琐的端到端测试而头疼吗？Data Formulator作为一个结合了前端交互、后端AI处理和数据库操作的复杂系统，传统的单元测试难以覆盖完整的用户工作流。本文将为你揭秘如何构建完整的集成测试体系，并建立高效的CI/CD流水线。
读完本文你将获得：
- Data Formulator端到端测试架构设计
- 多环境测试策略与最佳实践
- 完整的CI/CD流水线配置方案
- 测试数据管理与Mock策略
- 性能测试与负载测试方案
Data Formulator架构概述
在深入测试策略之前，让我们先了解Data Formulator的核心架构：
技术栈矩阵
| 组件 | 技术 | 测试挑战 |
|---|---|---|
| 前端 | React + TypeScript + Vite | 交互测试、状态管理 |
| 后端 | Flask + Python 3.11+ | API测试、AI集成 |
| AI处理 | LiteLLM + 多模型支持 | LLM响应Mock |
| 数据库 | DuckDB + 多数据源 | 数据一致性测试 |
| 实时通信 | Server-Sent Events | 异步测试 |
端到端测试架构设计
测试金字塔策略
测试目录结构规划
tests/
├── unit/
│ ├── frontend/
│ ├── backend/
│ └── agents/
├── integration/
│ ├── api/
│ ├── database/
│ └── ai_agents/
├── e2e/
│ ├── cypress/
│ └── playwright/
├── performance/
└── fixtures/
├── test_data/
└── mocks/
核心测试场景设计
1. 数据加载与处理测试
# tests/integration/test_data_loading.py
import pytest
from data_formulator.data_loader import ExternalDataLoader
class TestDataLoading:
@pytest.mark.parametrize("file_format", ["csv", "json", "parquet"])
def test_file_loading(self, file_format):
"""测试不同文件格式的数据加载"""
loader = ExternalDataLoader()
test_file = f"test_data/sample.{file_format}"
# 加载测试数据
result = loader.load_file(test_file)
assert result.success
assert result.dataframe is not None
assert len(result.dataframe) > 0
def test_database_connection(self):
"""测试数据库连接功能"""
# 使用测试数据库配置
config = {
"db_type": "sqlite",
"database": ":memory:"
}
loader = ExternalDataLoader()
connection = loader.connect_to_database(config)
assert connection is not None
assert connection.test_connection()
2. AI Agent集成测试
# tests/integration/test_ai_agents.py
from unittest.mock import patch
from data_formulator.agents import AgentManager
class TestAIAgents:
@patch('data_formulator.agents.LLMClient.generate_response')
def test_sql_generation_agent(self, mock_llm):
"""测试SQL生成Agent的完整工作流"""
# 设置Mock响应
mock_llm.return_value = {
"sql": "SELECT category, SUM(sales) FROM sales_data GROUP BY category",
"explanation": "按类别分组计算销售总额"
}
agent = AgentManager.get_agent('sql_data_transform')
test_data = [{"category": "A", "sales": 100}, {"category": "B", "sales": 200}]
result = agent.process(test_data, "按类别汇总销售额")
assert result.success
assert "SELECT" in result.generated_code
assert result.explanation is not None
3. 端到端用户工作流测试
# tests/e2e/test_user_workflow.py
import time
from playwright.sync_api import expect
def test_complete_visualization_workflow(page):
"""完整的可视化创建工作流测试"""
# 1. 打开应用
page.goto("http://localhost:5000")
# 2. 上传测试数据
page.get_by_label("上传数据").click()
with page.expect_file_chooser() as fc_info:
page.get_by_text("选择文件").click()
file_chooser = fc_info.value
file_chooser.set_files("tests/fixtures/sample_data.csv")
# 3. 配置可视化
page.get_by_text("销售额").drag_to(page.get_by_text("Y轴"))
page.get_by_text("日期").drag_to(page.get_by_text("X轴"))
# 4. 使用AI生成衍生字段
page.get_by_placeholder("输入衍生字段...").fill("月度增长率")
page.get_by_text("Formulate").click()
# 5. 验证结果
expect(page.get_by_text("可视化生成成功")).to_be_visible()
expect(page.locator(".vega-chart")).to_be_visible()
CI/CD流水线配置
GitHub Actions工作流配置
# .github/workflows/ci-cd.yml
name: Data Formulator CI/CD
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.9', '3.10', '3.11']
node-version: ['18.x', '20.x']
services:
postgres:
image: postgres:13
env:
POSTGRES_USER: testuser
POSTGRES_PASSWORD: testpass
POSTGRES_DB: testdb
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'yarn'
- name: Install backend dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pytest pytest-cov pytest-playwright
- name: Install frontend dependencies
run: yarn install
- name: Run backend tests
run: |
python -m pytest tests/unit/backend/ tests/integration/ --cov=py-src/data_formulator --cov-report=xml
- name: Run frontend tests
run: yarn test --coverage
- name: Run E2E tests
run: |
playwright install
python -m pytest tests/e2e/ --headed
- name: Upload coverage reports
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
flags: unittests
build-and-deploy:
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20.x'
- name: Build frontend
run: yarn build
- name: Build Python package
run: |
pip install build
python -m build
- name: Create release
uses: softprops/action-gh-release@v1
with:
files: dist/*.whl
generate_release_notes: true
测试数据管理策略
测试数据集设计
# tests/fixtures/test_datasets.yaml
datasets:
- name: "sales_data"
description: "模拟销售数据用于测试"
columns:
- name: "date"
type: "date"
values: "2024-01-01 to 2024-12-31"
- name: "category"
type: "string"
values: ["Electronics", "Clothing", "Food", "Books"]
- name: "sales"
type: "float"
range: [100.0, 10000.0]
- name: "quantity"
type: "integer"
range: [1, 100]
- name: "user_behavior"
description: "用户行为分析测试数据"
columns:
- name: "user_id"
type: "string"
pattern: "user_[0-9]{5}"
- name: "action"
type: "string"
values: ["click", "view", "purchase", "logout"]
- name: "timestamp"
type: "datetime"
range: ["2024-01-01 00:00:00", "2024-01-31 23:59:59"]
Mock服务配置
# tests/conftest.py
import pytest
from unittest.mock import Mock, patch
from data_formulator import create_app
@pytest.fixture
def mock_llm_client():
"""创建Mock的LLM客户端"""
with patch('data_formulator.agents.LLMClient') as mock:
mock_instance = Mock()
mock.return_value = mock_instance
# 设置默认响应
mock_instance.generate_response.return_value = {
"sql": "SELECT * FROM test_data LIMIT 10",
"explanation": "测试查询",
"success": True
}
yield mock_instance
@pytest.fixture
def test_app(mock_llm_client):
"""创建测试用的Flask应用"""
app = create_app({
'TESTING': True,
'SQLALCHEMY_DATABASE_URI': 'sqlite:///:memory:',
'LLM_PROVIDER': 'mock'
})
yield app
@pytest.fixture
def client(test_app):
"""测试客户端"""
with test_app.test_client() as client:
yield client
性能测试与监控
负载测试方案
# tests/performance/test_load_performance.py
import locust
from locust import HttpUser, task, between
class DataFormulatorUser(HttpUser):
wait_time = between(1, 5)
@task(3)
def test_data_loading(self):
"""测试数据加载性能"""
files = {'file': ('test_data.csv', open('tests/fixtures/sample_data.csv', 'rb'))}
self.client.post("/api/upload", files=files)
@task(5)
def test_visualization_generation(self):
"""测试可视化生成性能"""
payload = {
"data": [{"x": i, "y": i*2} for i in range(100)],
"encoding": {"x": "x", "y": "y"},
"chart_type": "line"
}
self.client.post("/api/generate-chart", json=payload)
@task(2)
def test_ai_agent_processing(self):
"""测试AI Agent处理性能"""
payload = {
"data": [{"category": "A", "value": 100}],
"prompt": "计算每个类别的总和"
}
self.client.post("/api/ai/process", json=payload)
性能指标监控
| 指标 | 目标值 | 监控频率 | 告警阈值 |
|---|---|---|---|
| API响应时间 | <200ms | 实时 | >500ms |
| 数据加载时间 | <1s | 每次请求 | >3s |
| AI处理时间 | <5s | 每次请求 | >10s |
| 内存使用 | <512MB | 每分钟 | >1GB |
| 并发用户数 | 50+ | 实时 | <10 |
测试环境管理
多环境配置策略
# config/test-environment.yaml
environments:
local:
database: sqlite:///test.db
llm_provider: mock
features:
- testing
- development
ci:
database: postgresql://testuser:testpass@localhost:5432/testdb
llm_provider: mock
features:
- testing
- ci
staging:
database: ${STAGING_DB_URL}
llm_provider: openai
model: gpt-3.5-turbo
features:
- staging
- pre-production
总结与最佳实践
通过本文的完整测试方案，你可以为Data Formulator构建强大的质量保障体系：
关键收获
- 分层测试策略：单元测试覆盖核心逻辑，集成测试验证组件协作，端到端测试确保用户体验
- Mock智能策略：合理使用Mock替代外部依赖，提高测试稳定性和速度
- 数据驱动测试：使用多样化的测试数据集覆盖各种边界情况
- 性能监控：建立完整的性能基准和监控体系
实施路线图
后续优化方向
- 测试数据生成自动化：开发智能测试数据生成工具
- AI测试增强：针对LLM响应的多样性设计更智能的测试策略
- 安全测试集成：加入安全扫描和漏洞检测
- 跨浏览器测试：确保前端在各种浏览器环境下的兼容性
通过这套完整的测试体系，Data Formulator项目可以确保在快速迭代的同时保持高质量的代码标准，为用户提供稳定可靠的数据可视化体验。
创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
