8-MongoDB时间序列集合深度解析:loT与监控场景的数据存储优化

MongoDB 时间序列集合深度解析:IoT 与监控场景的数据存储优化

专为时间序列数据优化的 MongoDB 特性,大幅提升 IoT、监控、金融数据的存储和查询性能

📚 文章概述

时间序列数据是现代社会最重要的数据类型之一,从 IoT 设备的传感器数据到金融市场的交易记录,从系统监控指标到用户行为日志,时间序列数据无处不在。MongoDB 5.0 引入的时间序列集合(Time Series Collections)功能,专门针对这类数据进行了深度优化,能够提供比传统集合更高效的存储和查询性能。

本文将深入解析 MongoDB 时间序列集合的技术原理、应用场景和企业级实践,通过 IoT 监控系统、金融交易系统、用户行为分析等实际案例,帮助读者掌握这一重要特性,并在实际项目中发挥其最大价值。

文章特色:

  • 技术深度:从底层原理到应用实践的全方位解析
  • 实战导向:IoT、监控、金融三大核心应用场景
  • 性能优化:企业级性能调优和最佳实践
  • 代码示例:30+ 可运行的实际代码案例
  • 企业视角:生产环境的部署和运维指导

🎯 学习目标

通过本文学习,您将掌握:

  1. 时间序列集合的核心概念和技术原理
  2. 时间序列集合的创建、配置和优化方法
  3. IoT 设备监控系统的完整实现方案
  4. 金融交易数据的高效存储和查询策略
  5. 企业级性能优化和监控实践
  6. 时间序列数据的分片和集群部署方案

📖 目录结构

  1. 时间序列集合基础概念
  2. 创建和配置时间序列集合
  3. 时间序列数据操作
  4. IoT 设备监控系统实战
  5. 金融交易数据系统实战
  6. 性能优化与调优
  7. 企业级部署与运维
  8. 时间序列集合最佳实践

1. 时间序列集合基础概念

1.1 什么是时间序列数据

时间序列数据是指按时间顺序排列的数据点序列,每个数据点都包含一个时间戳和对应的数值。这类数据具有以下特征:

  • 时间有序性:数据按时间顺序产生和存储
  • 高频写入:通常有大量的写入操作
  • 低频更新:数据一旦写入很少修改
  • 时间范围查询:经常需要按时间范围查询数据
  • 数据压缩:相似时间点的数据具有相似性
// 典型的 IoT 传感器数据示例
{
  "_id": ObjectId("..."),
  "timestamp": ISODate("2024-01-15T10:30:00Z"),
  "deviceId": "sensor-001",
  "location": "building-a-floor-1",
  "temperature": 23.5,
  "humidity": 65.2,
  "pressure": 1013.25,
  "batteryLevel": 85
}

1.2 传统集合 vs 时间序列集合

传统集合的局限性

使用传统集合存储时间序列数据会面临以下问题:

// 传统集合存储时间序列数据的问题
db.sensor_data.insertMany([
  {
    "_id": ObjectId("..."),
    "timestamp": ISODate("2024-01-15T10:30:00Z"),
    "deviceId": "sensor-001",
    "temperature": 23.5
  },
  {
    "_id": ObjectId("..."),
    "timestamp": ISODate("2024-01-15T10:31:00Z"),
    "deviceId": "sensor-001", 
    "temperature": 23.7
  }
  // ... 大量相似数据
]);

主要问题:

  • 存储效率低:重复的字段名和结构信息占用大量空间
  • 查询性能差:缺乏针对时间序列的索引优化
  • 写入性能差:随机插入导致频繁的索引重建
  • 压缩效果差:无法利用时间序列数据的相似性进行压缩
时间序列集合的优势

时间序列集合专门针对时间序列数据进行了优化:

// 时间序列集合的优势
db.createCollection("sensor_data", {
  timeseries: {
    timeField: "timestamp",
    metaField: "metadata",
    granularity: "minutes"
  }
});

核心优势:

  • 存储优化:自动压缩相似的时间序列数据
  • 查询优化:针对时间范围查询进行索引优化
  • 写入优化:批量插入和顺序写入优化
  • 自动分桶:按时间自动组织数据到桶中

1.3 时间序列集合的工作原理

数据分桶机制

时间序列集合使用分桶(Bucketing)机制来组织数据:

// 时间序列集合的数据分桶原理
{
  "_id": ObjectId("..."),
  "control": {
    "version": 1,
    "min": {
      "_id": ObjectId("..."),
      "timestamp": ISODate("2024-01-15T10:30:00Z"),
      "metadata": {"deviceId": "sensor-001"}
    },
    "max": {
      "_id": ObjectId("..."),
      "timestamp": ISODate("2024-01-15T10:35:00Z"),
      "metadata": {"deviceId": "sensor-001"}
    }
  },
  "data": {
    "timestamp": [
      ISODate("2024-01-15T10:30:00Z"),
      ISODate("2024-01-15T10:31:00Z"),
      ISODate("2024-01-15T10:32:00Z"),
      ISODate("2024-01-15T10:33:00Z"),
      ISODate("2024-01-15T10:34:00Z"),
      ISODate("2024-01-15T10:35:00Z")
    ],
    "temperature": [23.5, 23.7, 23.9, 24.1, 24.0, 23.8],
    "humidity": [65.2, 65.1, 65.3, 65.5, 65.4, 65.2]
  }
}
自动压缩机制

时间序列集合会自动对相似数据进行压缩:

// 压缩前后的数据对比
// 压缩前:每个数据点一个文档
[
  {"timestamp": "2024-01-15T10:30:00Z", "temperature": 23.5},
  {"timestamp": "2024-01-15T10:31:00Z", "temperature": 23.7},
  {"timestamp": "2024-01-15T10:32:00Z", "temperature": 23.9}
]

// 压缩后:时间序列数据组织在一个文档中
{
  "timestamp": ["2024-01-15T10:30:00Z", "2024-01-15T10:31:00Z", "2024-01-15T10:32:00Z"],
  "temperature": [23.5, 23.7, 23.9]
}

2. 创建和配置时间序列集合

2.1 基础创建语法

最简单的创建方式
// 创建基础的时间序列集合
db.createCollection("sensor_readings", {
  timeseries: {
    timeField: "timestamp"
  }
});

// 验证集合创建
db.runCommand({listCollections: 1, filter: {name: "sensor_readings"}});
完整配置参数
// 创建完整配置的时间序列集合
db.createCollection("iot_metrics", {
  timeseries: {
    timeField: "timestamp",        // 必需:时间字段
    metaField: "device_info",      // 可选:元数据字段
    granularity: "seconds"         // 可选:数据粒度
  },
  expireAfterSeconds: 2592000,     // 可选:TTL索引(30天)
  clusteredIndex: {                // 可选:聚集索引
    key: {_id: 1},
    unique: true
  }
});

2.2 关键配置参数详解

timeField(时间字段)
// timeField 是必需参数,指定时间戳字段
db.createCollection("stock_prices", {
  timeseries: {
    timeField: "trade_time"  // 指定时间戳字段
  }
});

// 插入数据示例
db.stock_prices.insertOne({
  "trade_time": new Date(),  // 时间字段
  "symbol": "AAPL",
  "price": 150.25,
  "volume": 1000
});
metaField(元数据字段)
// metaField 用于存储设备或实体的标识信息
db.createCollection("weather_data", {
  timeseries: {
    timeField: "measurement_time",
    metaField: "station_info"  // 元数据字段
  }
});

// 插入数据示例
db.weather_data.insertOne({
  "measurement_time": new Date(),
  "station_info": {           // 元数据
    "station_id": "WS001",
    "location": "Beijing",
    "altitude": 50
  },
  "temperature": 22.5,
  "humidity": 60,
  "pressure": 1013.25
});
granularity(数据粒度)
// granularity 影响分桶策略和查询性能
db.createCollection("high_frequency_data", {
  timeseries: {
    timeField: "timestamp",
    metaField: "device",
    granularity: "seconds"    // 秒级数据
  }
});

db.createCollection("daily_reports", {
  timeseries: {
    timeField: "date",
    metaField: "region",
    granularity: "hours"      // 小时级数据
  }
});

// 支持的粒度值
// "seconds" - 秒级数据(默认)
// "minutes" - 分钟级数据  
// "hours" - 小时级数据

2.3 TTL 索引配置

自动数据过期
// 配置数据自动过期
db.createCollection("log_entries", {
  timeseries: {
    timeField: "log_time"
  },
  expireAfterSeconds: 86400  // 24小时后自动删除
});

// 或者使用 TTL 索引
db.log_entries.createIndex(
  { "log_time": 1 },
  { expireAfterSeconds: 86400 }
);
条件过期策略
// 基于条件的 TTL 配置
db.createCollection("user_activity", {
  timeseries: {
    timeField: "activity_time",
    metaField: "user_info"
  }
});

// 为不同用户类型设置不同的过期时间
db.user_activity.createIndex(
  { "activity_time": 1, "user_info.userType": 1 },
  { 
    expireAfterSeconds: 2592000,  // 普通用户30天
    partialFilterExpression: { "user_info.userType": "regular" }
  }
);

db.user_activity.createIndex(
  { "activity_time": 1, "user_info.userType": 1 },
  { 
    expireAfterSeconds: 7776000,  // VIP用户90天
    partialFilterExpression: { "user_info.userType": "vip" }
  }
);

2.4 聚集索引配置

聚集索引的优势
// 创建带聚集索引的时间序列集合
db.createCollection("performance_metrics", {
  timeseries: {
    timeField: "timestamp",
    metaField: "server_info"
  },
  clusteredIndex: {
    key: {_id: 1},
    unique: true,
    name: "clustered_index"
  }
});

// 聚集索引的优势:
// 1. 减少索引维护开销
// 2. 提高查询性能
// 3. 优化存储空间使用
聚集索引最佳实践
// 为不同场景选择合适的聚集索引
// 场景1:按时间顺序查询为主
db.createCollection("time_series_1", {
  timeseries: {
    timeField: "ts"
  },
  clusteredIndex: {
    key: {_id: 1},  // 使用默认的_id聚集
    unique: true
  }
});

// 场景2:按设备ID分组查询
db.createCollection("device_metrics", {
  timeseries: {
    timeField: "timestamp",
    metaField: "device"
  },
  clusteredIndex: {
    key: {"device.deviceId": 1, "timestamp": 1},  // 复合聚集索引
    unique: false
  }
});

3. 时间序列数据操作

3.1 数据插入操作

单条数据插入
// 插入单条时间序列数据
db.sensor_data.insertOne({
  "timestamp": new Date(),
  "deviceId": "temp_sensor_001",
  "location": "building_a_floor_1",
  "temperature": 23.5,
  "humidity": 65.2,
  "batteryLevel": 85
});

// 验证插入结果
db.sensor_data.findOne({deviceId: "temp_sensor_001"});
批量数据插入
// 批量插入时间序列数据
const sensorData = [];
const baseTime = new Date();

for (let i = 0; i < 100; i++) {
  sensorData.push({
    "timestamp": new Date(baseTime.getTime() + i * 60000), // 每分钟一个数据点
    "deviceId": `sensor_${String(i % 10).padStart(3, '0')}`,
    "temperature": 20 + Math.random() * 10,
    "humidity": 50 + Math.random() * 30,
    "pressure": 1000 + Math.random() * 50
  });
}

// 执行批量插入
const result = db.sensor_data.insertMany(sensorData);
print(`插入了 ${result.insertedIds.length} 条记录`);
有序批量插入
// 有序批量插入(推荐方式)
db.sensor_data.insertMany([
  {
    "timestamp": new Date("2024-01-15T10:00:00Z"),
    "deviceId": "sensor_001",
    "temperature": 23.1
  },
  {
    "timestamp": new Date("2024-01-15T10:01:00Z"),
    "deviceId": "sensor_001", 
    "temperature": 23.3
  },
  {
    "timestamp": new Date("2024-01-15T10:02:00Z"),
    "deviceId": "sensor_001",
    "temperature": 23.5
  }
], { ordered: true }); // 有序插入,提高性能

3.2 数据查询操作

时间范围查询
// 查询指定时间范围的数据
const startTime = new Date("2024-01-15T10:00:00Z");
const endTime = new Date("2024-01-15T11:00:00Z");

db.sensor_data.find({
  "timestamp": {
    $gte: startTime,
    $lte: endTime
  }
}).sort({"timestamp": 1});

// 使用聚合管道进行时间范围查询
db.sensor_data.aggregate([
  {
    $match: {
      "timestamp": {
        $gte: startTime,
        $lte: endTime
      }
    }
  },
  {
    $group: {
      "_id": "$deviceId",
      "avgTemperature": {$avg: "$temperature"},
      "maxTemperature": {$max: "$temperature"},
      "minTemperature": {$min: "$temperature"},
      "count": {$sum: 1}
    }
  }
]);
元数据过滤查询
// 基于元数据的查询
db.sensor_data.find({
  "deviceId": "sensor_001",
  "timestamp": {
    $gte: new Date("2024-01-15T00:00:00Z"),
    $lt: new Date("2024-01-16T00:00:00Z")
  }
});

// 复合条件查询
db.sensor_data.find({
  $and: [
    {"deviceId": {$in: ["sensor_001", "sensor_002"]}},
    {"temperature": {$gt: 25}},
    {"timestamp": {$gte: new Date("2024-01-15T10:00:00Z")}}
  ]
}).sort({"timestamp": -1});

3.3 数据聚合操作

时间窗口聚合
// 按时间窗口进行数据聚合
db.sensor_data.aggregate([
  {
    $match: {
      "timestamp": {
        $gte: new Date("2024-01-15T00:00:00Z"),
        $lt: new Date("2024-01-16T00:00:00Z")
      }
    }
  },
  {
    $group: {
      "_id": {
        "hour": {$hour: "$timestamp"},
        "deviceId": "$deviceId"
      },
      "avgTemperature": {$avg: "$temperature"},
      "maxTemperature": {$max: "$temperature"},
      "minTemperature": {$min: "$temperature"},
      "count": {$sum: 1}
    }
  },
  {
    $sort: {"_id.hour": 1, "_id.deviceId": 1}
  }
]);
数据降采样
// 数据降采样:将高频数据聚合为低频数据
db.high_freq_data.aggregate([
  {
    $match: {
      "timestamp": {
        $gte: new Date("2024-01-15T00:00:00Z"),
        $lt: new Date("2024-01-16T00:00:00Z")
      }
    }
  },
  {
    $group: {
      "_id": {
        "deviceId": "$deviceId",
        "year": {$year: "$timestamp"},
        "month": {$month: "$timestamp"},
        "day": {$dayOfMonth: "$timestamp"},
        "hour": {$hour: "$timestamp"}
      },
      "avgValue": {$avg: "$value"},
      "maxValue": {$max: "$value"},
      "minValue": {$min: "$value"},
      "sampleCount": {$sum: 1}
    }
  },
  {
    $project: {
      "_id": 0,
      "deviceId": "$_id.deviceId",
      "timestamp": {
        $dateFromParts: {
          "year": "$_id.year",
          "month": "$_id.month",
          "day": "$_id.day",
          "hour": "$_id.hour"
        }
      },
      "avgValue": 1,
      "maxValue": 1,
      "minValue": 1,
      "sampleCount": 1
    }
  }
]);

3.4 数据更新和删除

数据更新操作
// 更新时间序列数据(谨慎使用)
db.sensor_data.updateOne(
  {
    "deviceId": "sensor_001",
    "timestamp": new Date("2024-01-15T10:00:00Z")
  },
  {
    $set: {
      "temperature": 23.8,
      "lastModified": new Date()
    }
  }
);

// 批量更新元数据
db.sensor_data.updateMany(
  {"deviceId": "old_sensor_001"},
  {
    $set: {
      "deviceId": "new_sensor_001",
      "updatedAt": new Date()
    }
  }
);
数据删除操作
// 删除指定时间范围的数据
db.sensor_data.deleteMany({
  "timestamp": {
    $lt: new Date("2024-01-01T00:00:00Z")
  }
});

// 删除指定设备的数据
db.sensor_data.deleteMany({
  "deviceId": "decommissioned_sensor_001"
});

// 使用 TTL 自动删除过期数据
db.sensor_data.createIndex(
  {"timestamp": 1},
  {expireAfterSeconds: 2592000} // 30天后自动删除
);

4. IoT 设备监控系统实战

4.1 系统架构设计

整体架构
// IoT 监控系统架构
/*
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   IoT 设备      │───▶│  数据采集层      │───▶│  MongoDB 集群    │
│   (传感器)      │    │  (数据网关)      │    │  (时间序列集合)  │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                │                        │
                                ▼                        ▼
                       ┌─────────────────┐    ┌─────────────────┐
                       │  消息队列        │    │  实时分析引擎    │
                       │  (Apache Kafka) │    │  (流处理)       │
                       └─────────────────┘    └─────────────────┘
                                │                        │
                                ▼                        ▼
                       ┌─────────────────┐    ┌─────────────────┐
                       │  告警系统        │    │  可视化界面      │
                       │  (阈值监控)      │    │  (Grafana)     │
                       └─────────────────┘    └─────────────────┘
*/
数据模型设计
// IoT 设备元数据集合
db.createCollection("iot_devices", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["deviceId", "deviceType", "location", "status"],
      properties: {
        deviceId: {bsonType: "string"},
        deviceType: {bsonType: "string"},
        location: {
          bsonType: "object",
          properties: {
            building: {bsonType: "string"},
            floor: {bsonType: "int"},
            room: {bsonType: "string"},
            coordinates: {
              bsonType: "array",
              items: {bsonType: "double"}
            }
          }
        },
        status: {enum: ["active", "inactive", "maintenance"]},
        specifications: {bsonType: "object"},
        createdAt: {bsonType: "date"},
        updatedAt: {bsonType: "date"}
      }
    }
  }
});

// 设备监控数据时间序列集合
db.createCollection("device_metrics", {
  timeseries: {
    timeField: "timestamp",
    metaField: "device",
    granularity: "seconds"
  },
  expireAfterSeconds: 7776000 // 90天过期
});

4.2 数据采集实现

设备注册
// 设备注册函数
function registerIoTDevice(deviceInfo) {
  const device = {
    deviceId: deviceInfo.deviceId,
    deviceType: deviceInfo.deviceType,
    location: {
      building: deviceInfo.building,
      floor: deviceInfo.floor,
      room: deviceInfo.room,
      coordinates: deviceInfo.coordinates
    },
    status: "active",
    specifications: {
      manufacturer: deviceInfo.manufacturer,
      model: deviceInfo.model,
      firmwareVersion: deviceInfo.firmwareVersion,
      capabilities: deviceInfo.capabilities
    },
    createdAt: new Date(),
    updatedAt: new Date()
  };
  
  return db.iot_devices.insertOne(device);
}

// 注册示例设备
registerIoTDevice({
  deviceId: "temp_humidity_001",
  deviceType: "environmental_sensor",
  building: "Building_A",
  floor: 1,
  room: "Office_101",
  coordinates: [116.3974, 39.9093],
  manufacturer: "SensoTech",
  model: "ST-2000",
  firmwareVersion: "2.1.0",
  capabilities: ["temperature", "humidity", "pressure"]
});
数据采集服务
// 模拟数据采集服务
class IoTDataCollector {
  constructor(deviceId) {
    this.deviceId = deviceId;
    this.isCollecting = false;
  }
  
  // 开始数据采集
  startCollection(interval = 60000) { // 默认1分钟采集一次
    if (this.isCollecting) return;
    
    this.isCollecting = true;
    this.interval = setInterval(() => {
      this.collectData();
    }, interval);
    
    console.log(`开始采集设备 ${this.deviceId} 的数据`);
  }
  
  // 停止数据采集
  stopCollection() {
    if (!this.isCollecting) return;
    
    clearInterval(this.interval);
    this.isCollecting = false;
    console.log(`停止采集设备 ${this.deviceId} 的数据`);
  }
  
  // 采集数据
  collectData() {
    const sensorData = {
      timestamp: new Date(),
      device: {
        deviceId: this.deviceId,
        type: "environmental_sensor"
      },
      temperature: this.generateTemperature(),
      humidity: this.generateHumidity(),
      pressure: this.generatePressure(),
      batteryLevel: this.generateBatteryLevel(),
      signalStrength: this.generateSignalStrength()
    };
    
    // 插入时间序列数据
    db.device_metrics.insertOne(sensorData);
    
    // 检查告警条件
    this.checkAlerts(sensorData);
  }
  
  // 生成模拟温度数据
  generateTemperature() {
    const baseTemp = 23;
    const variation = (Math.random() - 0.5) * 4;
    return Math.round((baseTemp + variation) * 10) / 10;
  }
  
  // 生成模拟湿度数据
  generateHumidity() {
    const baseHumidity = 65;
    const variation = (Math.random() - 0.5) * 20;
    return Math.max(0, Math.min(100, Math.round((baseHumidity + variation) * 10) / 10));
  }
  
  // 生成模拟气压数据
  generatePressure() {
    const basePressure = 1013.25;
    const variation = (Math.random() - 0.5) * 10;
    return Math.round((basePressure + variation) * 100) / 100;
  }
    // 生成模拟气压数据
  generatePressure() {
    const basePressure = 1013.25;
    const variation = (Math.random() - 0.5) * 10;
    return Math.round((basePressure + variation) * 100) / 100;
  }
  
  // 生成模拟电池电量
  generateBatteryLevel() {
    const baseLevel = 85;
    const variation = (Math.random() - 0.5) * 10;
    return Math.max(0, Math.min(100, Math.round((baseLevel + variation) * 10) / 10));
  }
  
  // 生成模拟信号强度
  generateSignalStrength() {
    const baseStrength = -60;
    const variation = (Math.random() - 0.5) * 20;
    return Math.round((baseStrength + variation) * 10) / 10;
  }
  
  // 检查告警条件
  checkAlerts(data) {
    const alerts = [];
    
    // 温度告警
    if (data.temperature > 30 || data.temperature < 15) {
      alerts.push({
        type: "temperature_alert",
        severity: data.temperature > 35 || data.temperature < 10 ? "critical" : "warning",
        value: data.temperature,
        threshold: data.temperature > 30 ? 30 : 15,
        message: `温度异常: ${data.temperature}°C`
      });
    }
    
    // 湿度告警
    if (data.humidity > 80 || data.humidity < 30) {
      alerts.push({
        type: "humidity_alert",
        severity: data.humidity > 90 || data.humidity < 20 ? "critical" : "warning",
        value: data.humidity,
        threshold: data.humidity > 80 ? 80 : 30,
        message: `湿度异常: ${data.humidity}%`
      });
    }
    
    // 电池电量告警
    if (data.batteryLevel < 20) {
      alerts.push({
        type: "battery_alert",
        severity: data.batteryLevel < 10 ? "critical" : "warning",
        value: data.batteryLevel,
        threshold: 20,
        message: `电池电量低: ${data.batteryLevel}%`
      });
    }
    
    // 信号强度告警
    if (data.signalStrength < -80) {
      alerts.push({
        type: "signal_alert",
        severity: data.signalStrength < -90 ? "critical" : "warning",
        value: data.signalStrength,
        threshold: -80,
        message: `信号强度弱: ${data.signalStrength}dBm`
      });
    }
    
    // 如果有告警,记录到告警集合
    if (alerts.length > 0) {
      alerts.forEach(alert => {
        db.device_alerts.insertOne({
          ...alert,
          deviceId: this.deviceId,
          timestamp: data.timestamp,
          acknowledged: false,
          createdAt: new Date()
        });
      });
    }
  }
}

// 使用示例
const collector = new IoTDataCollector("temp_humidity_001");
collector.startCollection(60000); // 每分钟采集一次
批量数据插入优化
// 批量数据插入优化类
class BatchDataInserter {
  constructor(collection, batchSize = 1000) {
    this.collection = collection;
    this.batchSize = batchSize;
    this.buffer = [];
    this.flushInterval = 5000; // 5秒刷新一次
    this.startFlushTimer();
  }
  
  // 添加数据到缓冲区
  addData(data) {
    this.buffer.push(data);
    
    // 如果缓冲区满了,立即刷新
    if (this.buffer.length >= this.batchSize) {
      this.flush();
    }
  }
  
  // 刷新缓冲区数据到数据库
  flush() {
    if (this.buffer.length === 0) return;
    
    try {
      // 按时间戳排序,提高插入效率
      this.buffer.sort((a, b) => new Date(a.timestamp) - new Date(b.timestamp));
      
      // 批量插入
      const result = this.collection.insertMany(this.buffer, { ordered: false });
      console.log(`批量插入了 ${result.insertedIds.length} 条记录`);
      
      // 清空缓冲区
      this.buffer = [];
    } catch (error) {
      console.error("批量插入失败:", error);
      // 可以选择重试或记录错误
    }
  }
  
  // 启动定时刷新
  startFlushTimer() {
    setInterval(() => {
      this.flush();
    }, this.flushInterval);
  }
  
  // 关闭时确保所有数据都被刷新
  close() {
    this.flush();
  }
}

// 使用批量插入器
const batchInserter = new BatchDataInserter(db.device_metrics, 500);

// 模拟多设备数据采集
function simulateMultiDeviceCollection() {
  const devices = ["sensor_001", "sensor_002", "sensor_003", "sensor_004", "sensor_005"];
  
  devices.forEach(deviceId => {
    const collector = new IoTDataCollector(deviceId);
    
    // 每10秒采集一次数据
    setInterval(() => {
      const data = {
        timestamp: new Date(),
        device: { deviceId: deviceId, type: "environmental_sensor" },
        temperature: 20 + Math.random() * 10,
        humidity: 50 + Math.random() * 30,
        pressure: 1000 + Math.random() * 50,
        batteryLevel: 80 + Math.random() * 20,
        signalStrength: -50 - Math.random() * 30
      };
      
      // 添加到批量插入器
      batchInserter.addData(data);
    }, 10000);
  });
}

// 启动多设备数据采集
simulateMultiDeviceCollection();

4.3 实时监控和告警系统

告警规则配置
// 告警规则集合
db.createCollection("alert_rules", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["ruleId", "deviceType", "metric", "condition", "threshold"],
      properties: {
        ruleId: {bsonType: "string"},
        deviceType: {bsonType: "string"},
        metric: {bsonType: "string"},
        condition: {enum: ["gt", "lt", "eq", "gte", "lte"]},
        threshold: {bsonType: "double"},
        severity: {enum: ["info", "warning", "critical"]},
        enabled: {bsonType: "bool"},
        cooldown: {bsonType: "int"}, // 冷却时间(秒)
        notificationChannels: {bsonType: "array"},
        createdAt: {bsonType: "date"},
        updatedAt: {bsonType: "date"}
      }
    }
  }
});

// 插入告警规则
db.alert_rules.insertMany([
  {
    ruleId: "temp_high_001",
    deviceType: "environmental_sensor",
    metric: "temperature",
    condition: "gt",
    threshold: 30,
    severity: "warning",
    enabled: true,
    cooldown: 300, // 5分钟冷却时间
    notificationChannels: ["email", "sms"],
    createdAt: new Date(),
    updatedAt: new Date()
  },
  {
    ruleId: "temp_critical_001",
    deviceType: "environmental_sensor",
    metric: "temperature",
    condition: "gt",
    threshold: 35,
    severity: "critical",
    enabled: true,
    cooldown: 60, // 1分钟冷却时间
    notificationChannels: ["email", "sms", "webhook"],
    createdAt: new Date(),
    updatedAt: new Date()
  },
  {
    ruleId: "battery_low_001",
    deviceType: "environmental_sensor",
    metric: "batteryLevel",
    condition: "lt",
    threshold: 20,
    severity: "warning",
    enabled: true,
    cooldown: 600, // 10分钟冷却时间
    notificationChannels: ["email"],
    createdAt: new Date(),
    updatedAt: new Date()
  }
]);
实时告警处理引擎
// 实时告警处理引擎
class AlertProcessor {
  constructor() {
    this.alertRules = new Map();
    this.alertHistory = new Map(); // 记录告警历史,用于冷却时间控制
    this.loadAlertRules();
  }
  
  // 加载告警规则
  async loadAlertRules() {
    const rules = await db.alert_rules.find({enabled: true}).toArray();
    rules.forEach(rule => {
      this.alertRules.set(rule.ruleId, rule);
    });
    console.log(`加载了 ${rules.length} 条告警规则`);
  }
  
  // 处理设备数据,检查告警条件
  async processDeviceData(data) {
    const deviceType = data.device.type;
    const deviceId = data.device.deviceId;
    
    // 遍历所有相关的告警规则
    for (const [ruleId, rule] of this.alertRules) {
      if (rule.deviceType !== deviceType) continue;
      
      const metricValue = data[rule.metric];
      if (metricValue === undefined) continue;
      
      // 检查告警条件
      const shouldAlert = this.checkCondition(metricValue, rule.condition, rule.threshold);
      
      if (shouldAlert) {
        // 检查冷却时间
        if (this.isInCooldown(ruleId, deviceId, rule.cooldown)) {
          continue;
        }
        
        // 触发告警
        await this.triggerAlert(rule, deviceId, metricValue, data.timestamp);
        
        // 记录告警历史
        this.recordAlertHistory(ruleId, deviceId);
      }
    }
  }
  
  // 检查告警条件
  checkCondition(value, condition, threshold) {
    switch (condition) {
      case "gt": return value > threshold;
      case "lt": return value < threshold;
      case "eq": return value === threshold;
      case "gte": return value >= threshold;
      case "lte": return value <= threshold;
      default: return false;
    }
  }
  
  // 检查是否在冷却时间内
  isInCooldown(ruleId, deviceId, cooldown) {
    const key = `${ruleId}_${deviceId}`;
    const lastAlert = this.alertHistory.get(key);
    
    if (!lastAlert) return false;
    
    const timeSinceLastAlert = (Date.now() - lastAlert) / 1000;
    return timeSinceLastAlert < cooldown;
  }
  
  // 触发告警
  async triggerAlert(rule, deviceId, value, timestamp) {
    const alert = {
      ruleId: rule.ruleId,
      deviceId: deviceId,
      metric: rule.metric,
      value: value,
      threshold: rule.threshold,
      condition: rule.condition,
      severity: rule.severity,
      message: this.generateAlertMessage(rule, deviceId, value),
      timestamp: timestamp,
      acknowledged: false,
      createdAt: new Date(),
      notificationChannels: rule.notificationChannels
    };
    
    // 保存告警记录
    await db.device_alerts.insertOne(alert);
    
    // 发送通知
    await this.sendNotifications(alert);
    
    console.log(`告警触发: ${alert.message}`);
  }
  
  // 生成告警消息
  generateAlertMessage(rule, deviceId, value) {
    const conditionText = {
      "gt": "大于",
      "lt": "小于",
      "eq": "等于",
      "gte": "大于等于",
      "lte": "小于等于"
    };
    
    return `设备 ${deviceId}${rule.metric}${value} ${conditionText[rule.condition]} 阈值 ${rule.threshold}`;
  }
  
  // 发送通知
  async sendNotifications(alert) {
    for (const channel of alert.notificationChannels) {
      switch (channel) {
        case "email":
          await this.sendEmailNotification(alert);
          break;
        case "sms":
          await this.sendSMSNotification(alert);
          break;
        case "webhook":
          await this.sendWebhookNotification(alert);
          break;
      }
    }
  }
  
  // 发送邮件通知
  async sendEmailNotification(alert) {
    // 这里集成邮件服务
    console.log(`发送邮件通知: ${alert.message}`);
  }
  
  // 发送短信通知
  async sendSMSNotification(alert) {
    // 这里集成短信服务
    console.log(`发送短信通知: ${alert.message}`);
  }
  
  // 发送Webhook通知
  async sendWebhookNotification(alert) {
    // 这里集成Webhook服务
    console.log(`发送Webhook通知: ${alert.message}`);
  }
  
  // 记录告警历史
  recordAlertHistory(ruleId, deviceId) {
    const key = `${ruleId}_${deviceId}`;
    this.alertHistory.set(key, Date.now());
  }
}

// 创建告警处理器实例
const alertProcessor = new AlertProcessor();

// 修改数据采集器,集成告警处理
class EnhancedIoTDataCollector extends IoTDataCollector {
  async collectData() {
    const sensorData = {
      timestamp: new Date(),
      device: {
        deviceId: this.deviceId,
        type: "environmental_sensor"
      },
      temperature: this.generateTemperature(),
      humidity: this.generateHumidity(),
      pressure: this.generatePressure(),
      batteryLevel: this.generateBatteryLevel(),
      signalStrength: this.generateSignalStrength()
    };
    
    // 插入时间序列数据
    await db.device_metrics.insertOne(sensorData);
    
    // 处理告警
    await alertProcessor.processDeviceData(sensorData);
  }
}

4.4 数据查询和可视化

实时数据查询
// 实时数据查询类
class RealTimeDataQuery {
  constructor() {
    this.cache = new Map();
    this.cacheTimeout = 30000; // 30秒缓存
  }
  
  // 获取设备最新数据
  async getLatestDeviceData(deviceId, limit = 10) {
    const cacheKey = `latest_${deviceId}_${limit}`;
    const cached = this.getCachedData(cacheKey);
    if (cached) return cached;
    
    const data = await db.device_metrics.find(
      {"device.deviceId": deviceId}
    ).sort({"timestamp": -1}).limit(limit).toArray();
    
    this.setCachedData(cacheKey, data);
    return data;
  }
  
  // 获取时间范围内的数据
  async getDataByTimeRange(deviceId, startTime, endTime) {
    const query = {
      "device.deviceId": deviceId,
      "timestamp": {
        $gte: startTime,
        $lte: endTime
      }
    };
    
    return await db.device_metrics.find(query).sort({"timestamp": 1}).toArray();
  }
  
  // 获取设备统计信息
  async getDeviceStatistics(deviceId, timeRange = 24) {
    const endTime = new Date();
    const startTime = new Date(endTime.getTime() - timeRange * 60 * 60 * 1000);
    
    const pipeline = [
      {
        $match: {
          "device.deviceId": deviceId,
          "timestamp": {
            $gte: startTime,
            $lte: endTime
          }
        }
      },
      {
        $group: {
          "_id": null,
          "avgTemperature": {$avg: "$temperature"},
          "maxTemperature": {$max: "$temperature"},
          "minTemperature": {$min: "$temperature"},
          "avgHumidity": {$avg: "$humidity"},
          "maxHumidity": {$max: "$humidity"},
          "minHumidity": {$min: "$humidity"},
          "avgBatteryLevel": {$avg: "$batteryLevel"},
          "minBatteryLevel": {$min: "$batteryLevel"},
          "dataCount": {$sum: 1},
          "lastUpdate": {$max: "$timestamp"}
        }
      }
    ];
    
    const result = await db.device_metrics.aggregate(pipeline).toArray();
    return result[0] || {};
  }
  
  // 获取所有设备状态概览
  async getAllDevicesOverview() {
    const pipeline = [
      {
        $group: {
          "_id": "$device.deviceId",
          "deviceType": {$first: "$device.type"},
          "lastDataTime": {$max: "$timestamp"},
          "lastTemperature": {$last: "$temperature"},
          "lastHumidity": {$last: "$humidity"},
          "lastBatteryLevel": {$last: "$batteryLevel"},
          "dataCount": {$sum: 1}
        }
      },
      {
        $lookup: {
          from: "iot_devices",
          localField: "_id",
          foreignField: "deviceId",
          as: "deviceInfo"
        }
      },
      {
        $unwind: {path: "$deviceInfo", preserveNullAndEmptyArrays: true}
      },
      {
        $project: {
          "deviceId": "$_id",
          "deviceType": 1,
          "location": "$deviceInfo.location",
          "status": "$deviceInfo.status",
          "lastDataTime": 1,
          "lastTemperature": 1,
          "lastHumidity": 1,
          "lastBatteryLevel": 1,
          "dataCount": 1,
          "isOnline": {
            $cond: {
              if: {$gt: [{$subtract: [new Date(), "$lastDataTime"]}, 300000]}, // 5分钟
              then: false,
              else: true
            }
          }
        }
      }
    ];
    
    return await db.device_metrics.aggregate(pipeline).toArray();
  }
  
  // 缓存管理
  getCachedData(key) {
    const cached = this.cache.get(key);
    if (!cached) return null;
    
    if (Date.now() - cached.timestamp > this.cacheTimeout) {
      this.cache.delete(key);
      return null;
    }
    
    return cached.data;
  }
  
  setCachedData(key, data) {
    this.cache.set(key, {
      data: data,
      timestamp: Date.now()
    });
  }
}

// 使用示例
const dataQuery = new RealTimeDataQuery();

// 获取设备最新数据
dataQuery.getLatestDeviceData("temp_humidity_001", 5).then(data => {
  console.log("最新数据:", data);
});

// 获取设备统计信息
dataQuery.getDeviceStatistics("temp_humidity_001", 24).then(stats => {
  console.log("24小时统计:", stats);
});

// 获取所有设备概览
dataQuery.getAllDevicesOverview().then(overview => {
  console.log("设备概览:", overview);
});

5. 金融交易数据系统实战

5.1 金融数据模型设计

交易数据时间序列集合
// 创建金融交易数据时间序列集合
db.createCollection("trading_data", {
  timeseries: {
    timeField: "trade_timestamp",
    metaField: "instrument_info",
    granularity: "seconds"
  },
  expireAfterSeconds: 31536000 // 1年过期
});

// 创建市场数据集合
db.createCollection("market_data", {
  timeseries: {
    timeField: "market_timestamp",
    metaField: "symbol_info",
    granularity: "seconds"
  },
  expireAfterSeconds: 7776000 // 90天过期
});

// 创建用户交易记录集合
db.createCollection("user_trades", {
  timeseries: {
    timeField: "trade_time",
    metaField: "user_info",
    granularity: "minutes"
  },
  expireAfterSeconds: 63072000 // 2年过期
});
金融数据插入示例
// 股票交易数据
db.trading_data.insertOne({
  trade_timestamp: new Date(),
  instrument_info: {
    symbol: "AAPL",
    exchange: "NASDAQ",
    instrument_type: "stock",
    sector: "technology"
  },
  price: 150.25,
  volume: 1000,
  trade_type: "buy",
  order_id: "ORD-20240115-001",
  trader_id: "TRD-001"
});

// 外汇交易数据
db.trading_data.insertOne({
  trade_timestamp: new Date(),
  instrument_info: {
    symbol: "EURUSD",
    exchange: "FOREX",
    instrument_type: "currency_pair",
    base_currency: "EUR",
    quote_currency: "USD"
  },
  price: 1.0856,
  volume: 10000,
  trade_type: "sell",
  order_id: "ORD-20240115-002",
  trader_id: "TRD-002"
});

// 加密货币交易数据
db.trading_data.insertOne({
  trade_timestamp: new Date(),
  instrument_info: {
    symbol: "BTCUSD",
    exchange: "BINANCE",
    instrument_type: "cryptocurrency",
    base_currency: "BTC",
    quote_currency: "USD"
  },
  price: 42000.50,
  volume: 0.1,
  trade_type: "buy",
  order_id: "ORD-20240115-003",
  trader_id: "TRD-003"
});

5.2 实时市场数据处理

市场数据处理器
// 实时市场数据处理器
class MarketDataProcessor {
  constructor() {
    this.priceCache = new Map();
    this.volumeCache = new Map();
    this.updateInterval = 1000; // 1秒更新一次
    this.startProcessing();
  }
  
  // 开始处理市场数据
  startProcessing() {
    setInterval(() => {
      this.processMarketData();
    }, this.updateInterval);
  }
  
  // 处理市场数据
  async processMarketData() {
    const symbols = ["AAPL", "GOOGL", "MSFT", "TSLA", "AMZN"];
    
    for (const symbol of symbols) {
      await this.updateSymbolData(symbol);
    }
  }
  
  // 更新单个交易品种的数据
  async updateSymbolData(symbol) {
    // 获取最新价格数据
    const latestPrice = await this.getLatestPrice(symbol);
    if (!latestPrice) return;
    
    // 计算价格变化
    const priceChange = await this.calculatePriceChange(symbol, latestPrice.price);
    
    // 计算成交量
    const volume = await this.calculateVolume(symbol);
    
    // 计算技术指标
    const technicalIndicators = await this.calculateTechnicalIndicators(symbol);
    
    // 更新市场数据
    await db.market_data.insertOne({
      market_timestamp: new Date(),
      symbol_info: {
        symbol: symbol,
        exchange: "NASDAQ",
        instrument_type: "stock"
      },
      price: latestPrice.price,
      volume: volume,
      price_change: priceChange,
      price_change_percent: (priceChange / (latestPrice.price - priceChange)) * 100,
      technical_indicators: technicalIndicators,
      market_cap: latestPrice.price * volume * 1000, // 假设流通股数
      last_updated: new Date()
    });
    
    // 更新缓存
    this.priceCache.set(symbol, latestPrice.price);
    this.volumeCache.set(symbol, volume);
  }
  
  // 获取最新价格
  async getLatestPrice(symbol) {
    const result = await db.trading_data.findOne(
      {"instrument_info.symbol": symbol},
      {sort: {"trade_timestamp": -1}}
    );
    return result ? {price: result.price, timestamp: result.trade_timestamp} : null;
  }
  
  // 计算价格变化
  async calculatePriceChange(symbol, currentPrice) {
    const previousPrice = this.priceCache.get(symbol);
    if (!previousPrice) return 0;
    
    return currentPrice - previousPrice;
  }
  
  // 计算成交量
  async calculateVolume(symbol) {
    const endTime = new Date();
    const startTime = new Date(endTime.getTime() - 60000); // 过去1分钟
    
    const pipeline = [
      {
        $match: {
          "instrument_info.symbol": symbol,
          "trade_timestamp": {
            $gte: startTime,
            $lte: endTime
          }
        }
      },
      {
        $group: {
          "_id": null,
          "total_volume": {$sum: "$volume"}
        }
      }
    ];
    
    const result = await db.trading_data.aggregate(pipeline).toArray();
    return result[0]?.total_volume || 0;
  }
  
  // 计算技术指标
  async calculateTechnicalIndicators(symbol) {
    const endTime = new Date();
    const startTime = new Date(endTime.getTime() - 20 * 60000); // 过去20分钟
    
    // 获取历史价格数据
    const priceData = await db.market_data.find(
      {
        "symbol_info.symbol": symbol,
        "market_timestamp": {
          $gte: startTime,
          $lte: endTime
        }
      },
      {sort: {"market_timestamp": 1}}
    ).toArray();
    
    if (priceData.length < 2) return {};
    
    const prices = priceData.map(d => d.price);
    
    // 计算简单移动平均线
        // 计算简单移动平均线
    const sma5 = this.calculateSMA(prices, 5);
    const sma20 = this.calculateSMA(prices, 20);
    
    // 计算RSI
    const rsi = this.calculateRSI(prices);
    
    // 计算布林带
    const bollingerBands = this.calculateBollingerBands(prices, 20, 2);
    
    return {
      sma5: sma5,
      sma20: sma20,
      rsi: rsi,
      bollinger_upper: bollingerBands.upper,
      bollinger_middle: bollingerBands.middle,
      bollinger_lower: bollingerBands.lower
    };
  }
  
  // 计算简单移动平均线
  calculateSMA(prices, period) {
    if (prices.length < period) return null;
    const sum = prices.slice(-period).reduce((a, b) => a + b, 0);
    return sum / period;
  }
  
  // 计算RSI
  calculateRSI(prices, period = 14) {
    if (prices.length < period + 1) return null;
    
    let gains = 0;
    let losses = 0;
    
    for (let i = 1; i <= period; i++) {
      const change = prices[i] - prices[i - 1];
      if (change > 0) gains += change;
      else losses -= change;
    }
    
    const avgGain = gains / period;
    const avgLoss = losses / period;
    
    if (avgLoss === 0) return 100;
    
    const rs = avgGain / avgLoss;
    return 100 - (100 / (1 + rs));
  }
  
  // 计算布林带
  calculateBollingerBands(prices, period = 20, multiplier = 2) {
    if (prices.length < period) return {upper: null, middle: null, lower: null};
    
    const sma = this.calculateSMA(prices, period);
    const recentPrices = prices.slice(-period);
    
    const variance = recentPrices.reduce((sum, price) => {
      return sum + Math.pow(price - sma, 2);
    }, 0) / period;
    
    const stdDev = Math.sqrt(variance);
    
    return {
      upper: sma + (multiplier * stdDev),
      middle: sma,
      lower: sma - (multiplier * stdDev)
    };
  }
}

// 创建市场数据处理器实例
const marketProcessor = new MarketDataProcessor();

5.3 高频交易数据分析

交易模式识别
// 交易模式识别引擎
class TradingPatternAnalyzer {
  constructor() {
    this.patterns = new Map();
    this.loadPatternDefinitions();
  }
  
  // 加载模式定义
  loadPatternDefinitions() {
    this.patterns.set('head_and_shoulders', {
      name: '头肩形态',
      description: '经典的反转形态',
      conditions: this.headAndShouldersConditions,
      confidence: 0.8
    });
    
    this.patterns.set('double_top', {
      name: '双顶形态',
      description: '看跌反转形态',
      conditions: this.doubleTopConditions,
      confidence: 0.75
    });
    
    this.patterns.set('triangle', {
      name: '三角形形态',
      description: '整理形态',
      conditions: this.triangleConditions,
      confidence: 0.7
    });
  }
  
  // 分析交易模式
  async analyzePatterns(symbol, timeRange = 24) {
    const endTime = new Date();
    const startTime = new Date(endTime.getTime() - timeRange * 60 * 60 * 1000);
    
    // 获取价格数据
    const priceData = await db.market_data.find(
      {
        "symbol_info.symbol": symbol,
        "market_timestamp": {
          $gte: startTime,
          $lte: endTime
        }
      },
      {sort: {"market_timestamp": 1}}
    ).toArray();
    
    if (priceData.length < 50) return []; // 数据不足
    
    const patterns = [];
    
    // 检查各种模式
    for (const [patternId, pattern] of this.patterns) {
      const detected = await pattern.conditions(priceData);
      if (detected) {
        patterns.push({
          pattern_id: patternId,
          pattern_name: pattern.name,
          description: pattern.description,
          confidence: pattern.confidence,
          detected_at: new Date(),
          symbol: symbol
        });
      }
    }
    
    return patterns;
  }
  
  // 头肩形态检测条件
  headAndShouldersConditions(priceData) {
    if (priceData.length < 20) return false;
    
    const prices = priceData.map(d => d.price);
    const highs = this.findLocalHighs(prices);
    
    if (highs.length < 3) return false;
    
    const [left_shoulder, head, right_shoulder] = highs.slice(-3);
    
    // 检查头肩形态条件
    return head > left_shoulder && 
           head > right_shoulder && 
           Math.abs(left_shoulder - right_shoulder) < head * 0.05;
  }
  
  // 双顶形态检测条件
  doubleTopConditions(priceData) {
    if (priceData.length < 15) return false;
    
    const prices = priceData.map(d => d.price);
    const highs = this.findLocalHighs(prices);
    
    if (highs.length < 2) return false;
    
    const [first_top, second_top] = highs.slice(-2);
    
    // 检查双顶形态条件
    return Math.abs(first_top - second_top) < first_top * 0.03;
  }
  
  // 三角形形态检测条件
  triangleConditions(priceData) {
    if (priceData.length < 30) return false;
    
    const prices = priceData.map(d => d.price);
    const highs = this.findLocalHighs(prices);
    const lows = this.findLocalLows(prices);
    
    if (highs.length < 3 || lows.length < 3) return false;
    
    // 检查收敛趋势
    const recentHighs = highs.slice(-3);
    const recentLows = lows.slice(-3);
    
    const highTrend = this.calculateTrend(recentHighs);
    const lowTrend = this.calculateTrend(recentLows);
    
    // 一个上升,一个下降,形成三角形
    return (highTrend > 0 && lowTrend < 0) || (highTrend < 0 && lowTrend > 0);
  }
  
  // 寻找局部高点
  findLocalHighs(prices, window = 5) {
    const highs = [];
    for (let i = window; i < prices.length - window; i++) {
      let isHigh = true;
      for (let j = i - window; j <= i + window; j++) {
        if (j !== i && prices[j] >= prices[i]) {
          isHigh = false;
          break;
        }
      }
      if (isHigh) highs.push(prices[i]);
    }
    return highs;
  }
  
  // 寻找局部低点
  findLocalLows(prices, window = 5) {
    const lows = [];
    for (let i = window; i < prices.length - window; i++) {
      let isLow = true;
      for (let j = i - window; j <= i + window; j++) {
        if (j !== i && prices[j] <= prices[i]) {
          isLow = false;
          break;
        }
      }
      if (isLow) lows.push(prices[i]);
    }
    return lows;
  }
  
  // 计算趋势
  calculateTrend(values) {
    if (values.length < 2) return 0;
    const first = values[0];
    const last = values[values.length - 1];
    return (last - first) / first;
  }
}

// 使用示例
const patternAnalyzer = new TradingPatternAnalyzer();

// 分析AAPL的交易模式
patternAnalyzer.analyzePatterns("AAPL", 24).then(patterns => {
  console.log("检测到的交易模式:", patterns);
});

5.4 风险管理和合规监控

实时风险监控
// 实时风险监控系统
class RiskMonitor {
  constructor() {
    this.riskRules = new Map();
    this.alertThresholds = new Map();
    this.loadRiskRules();
  }
  
  // 加载风险规则
  loadRiskRules() {
    // 价格波动风险
    this.riskRules.set('price_volatility', {
      name: '价格波动风险',
      check: this.checkPriceVolatility,
      severity: 'high'
    });
    
    // 成交量异常风险
    this.riskRules.set('volume_anomaly', {
      name: '成交量异常风险',
      check: this.checkVolumeAnomaly,
      severity: 'medium'
    });
    
    // 流动性风险
    this.riskRules.set('liquidity_risk', {
      name: '流动性风险',
      check: this.checkLiquidityRisk,
      severity: 'high'
    });
  }
  
  // 监控交易风险
  async monitorTradeRisk(tradeData) {
    const risks = [];
    
    for (const [ruleId, rule] of this.riskRules) {
      const risk = await rule.check(tradeData);
      if (risk) {
        risks.push({
          rule_id: ruleId,
          rule_name: rule.name,
          severity: rule.severity,
          details: risk,
          detected_at: new Date()
        });
      }
    }
    
    return risks;
  }
  
  // 检查价格波动风险
  async checkPriceVolatility(tradeData) {
    const symbol = tradeData.instrument_info.symbol;
    const currentPrice = tradeData.price;
    
    // 获取过去1小时的价格数据
    const endTime = new Date();
    const startTime = new Date(endTime.getTime() - 60 * 60 * 1000);
    
    const priceData = await db.market_data.find(
      {
        "symbol_info.symbol": symbol,
        "market_timestamp": {
          $gte: startTime,
          $lte: endTime
        }
      },
      {sort: {"market_timestamp": 1}}
    ).toArray();
    
    if (priceData.length < 10) return null;
    
    const prices = priceData.map(d => d.price);
    const returns = [];
    
    // 计算收益率
    for (let i = 1; i < prices.length; i++) {
      returns.push((prices[i] - prices[i-1]) / prices[i-1]);
    }
    
    // 计算波动率
    const mean = returns.reduce((a, b) => a + b, 0) / returns.length;
    const variance = returns.reduce((sum, ret) => sum + Math.pow(ret - mean, 2), 0) / returns.length;
    const volatility = Math.sqrt(variance);
    
    // 年化波动率
    const annualizedVolatility = volatility * Math.sqrt(365 * 24 * 60);
    
    // 检查是否超过阈值
    if (annualizedVolatility > 0.5) { // 50%年化波动率
      return {
        volatility: annualizedVolatility,
        threshold: 0.5,
        message: `价格波动率过高: ${(annualizedVolatility * 100).toFixed(2)}%`
      };
    }
    
    return null;
  }
  
  // 检查成交量异常风险
  async checkVolumeAnomaly(tradeData) {
    const symbol = tradeData.instrument_info.symbol;
    const currentVolume = tradeData.volume;
    
    // 获取过去7天的平均成交量
    const endTime = new Date();
    const startTime = new Date(endTime.getTime() - 7 * 24 * 60 * 60 * 1000);
    
    const pipeline = [
      {
        $match: {
          "instrument_info.symbol": symbol,
          "trade_timestamp": {
            $gte: startTime,
            $lte: endTime
          }
        }
      },
      {
        $group: {
          "_id": {
            "year": {$year: "$trade_timestamp"},
            "month": {$month: "$trade_timestamp"},
            "day": {$dayOfMonth: "$trade_timestamp"},
            "hour": {$hour: "$trade_timestamp"}
          },
          "avg_volume": {$avg: "$volume"}
        }
      },
      {
        $group: {
          "_id": null,
          "overall_avg_volume": {$avg: "$avg_volume"}
        }
      }
    ];
    
    const result = await db.trading_data.aggregate(pipeline).toArray();
    const avgVolume = result[0]?.overall_avg_volume || 0;
    
    // 检查成交量是否异常
    if (avgVolume > 0 && currentVolume > avgVolume * 5) {
      return {
        current_volume: currentVolume,
        average_volume: avgVolume,
        ratio: currentVolume / avgVolume,
        message: `成交量异常: 当前成交量是平均值的 ${(currentVolume / avgVolume).toFixed(2)}`
      };
    }
    
    return null;
  }
  
  // 检查流动性风险
  async checkLiquidityRisk(tradeData) {
    const symbol = tradeData.instrument_info.symbol;
    const tradeSize = tradeData.volume;
    
    // 获取最近的市场深度数据
    const recentData = await db.market_data.findOne(
      {"symbol_info.symbol": symbol},
      {sort: {"market_timestamp": -1}}
    );
    
    if (!recentData || !recentData.market_cap) return null;
    
    // 计算交易对市场的影响
    const marketImpact = tradeSize / (recentData.market_cap / 1000); // 假设流通股数
    
    // 检查流动性风险
    if (marketImpact > 0.01) { // 1%的市场影响
      return {
        trade_size: tradeSize,
        market_cap: recentData.market_cap,
        market_impact: marketImpact,
        message: `交易对市场影响过大: ${(marketImpact * 100).toFixed(2)}%`
      };
    }
    
    return null;
  }
}

// 创建风险监控器实例
const riskMonitor = new RiskMonitor();

// 监控交易风险示例
async function monitorTradeRisk(tradeData) {
  const risks = await riskMonitor.monitorTradeRisk(tradeData);
  
  if (risks.length > 0) {
    console.log("检测到风险:", risks);
    
    // 记录风险事件
    await db.risk_events.insertMany(risks.map(risk => ({
      ...risk,
      trade_data: tradeData
    })));
    
    // 发送告警通知
    await sendRiskAlert(risks);
  }
  
  return risks;
}

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值