Starward项目中原神抽卡记录重复问题的分析与解决
痛点：抽卡记录重复困扰
还在为Starward中原神抽卡记录重复而烦恼吗？每次导入或获取抽卡记录时，重复的数据不仅占用存储空间，还会影响统计分析的准确性。本文将深入分析Starward项目中抽卡记录重复问题的根源，并提供完整的解决方案。
读完本文，你将获得：
- ✅ 抽卡记录重复问题的根本原因分析
- ✅ Starward项目中的重复数据处理机制
- ✅ 完整的防重复解决方案和最佳实践
- ✅ 代码层面的优化建议
问题根源分析
1. 数据源重复获取
在原神抽卡记录获取过程中，存在多个可能导致重复的场景：
2. 数据库插入机制
Starward使用SQLite数据库存储抽卡记录，核心的插入逻辑如下：
// GenshinGachaService.cs 中的插入方法
protected override int InsertGachaLogItems(List<GachaLogItem> items)
{
using var dapper = DatabaseService.CreateConnection();
using var t = dapper.BeginTransaction();
var affect = dapper.Execute("""
INSERT OR REPLACE INTO GenshinGachaItem
(Uid, Id, Name, Time, ItemId, ItemType, RankType, GachaType, Count, Lang)
VALUES (@Uid, @Id, @Name, @Time, @ItemId, @ItemType, @RankType, @GachaType, @Count, @Lang);
""", items, t);
t.Commit();
UpdateGachaItemId();
return affect;
}
关键点在于使用了 INSERT OR REPLACE 语句，这依赖于 Id 字段的唯一性来避免重复。
重复问题具体表现
场景1：分页请求边界重复
// GachaLogClient.cs 中的分页逻辑
protected async Task<List<T>> GetGachaLogByTypeAsync<T>(string prefix, IGachaType gachaType,
long endId = 0, IProgress<(IGachaType GachaType, int Page)>? progress = null,
CancellationToken cancellationToken = default) where T : GachaLogItem
{
var param = new GachaLogQuery(gachaType, 1, 20, 0);
var result = new List<T>();
while (true)
{
progress?.Report((gachaType, param.Page));
var list = await GetGachaLogByQueryAsync<T>(prefix, param, cancellationToken);
result.AddRange(list);
if (list.Count == 20 && list.Last().Id > endId)
{
param.Page++;
param.EndId = list.Last().Id; // 这里可能产生边界重复
}
else
{
break;
}
}
return result;
}
场景2：UIGF文件重复导入
// GenshinGachaService.cs 中的导入逻辑
public override long ImportGachaLog(string file)
{
var str = File.ReadAllText(file);
var obj = JsonSerializer.Deserialize<UIGF3File<UIGFGenshinGachaItem>>(str);
if (obj != null)
{
// ... 数据处理 ...
var count = InsertGachaLogItems(obj.List.ToList<GachaLogItem>());
return obj.Info.Uid;
}
return 0;
}
解决方案与最佳实践
1. 数据库层面优化
唯一索引确保数据完整性
-- 建议的数据库表结构优化
CREATE TABLE IF NOT EXISTS GenshinGachaItem (
Uid INTEGER NOT NULL,
Id INTEGER NOT NULL,
Name TEXT NOT NULL,
Time DATETIME NOT NULL,
ItemId INTEGER NOT NULL,
ItemType TEXT NOT NULL,
RankType INTEGER NOT NULL,
GachaType INTEGER NOT NULL,
Count INTEGER NOT NULL DEFAULT 1,
Lang TEXT,
PRIMARY KEY (Uid, Id) -- 复合主键防止重复
);
CREATE INDEX IF NOT EXISTS idx_genshin_gacha_time ON GenshinGachaItem(Time);
CREATE INDEX IF NOT EXISTS idx_genshin_gacha_uid ON GenshinGachaItem(Uid);
2. 应用层去重逻辑
内存中去重处理
// 增强的插入方法，添加内存去重逻辑
protected override int InsertGachaLogItems(List<GachaLogItem> items)
{
// 内存中去重处理
var distinctItems = items
.GroupBy(x => new { x.Uid, x.Id })
.Select(g => g.First())
.ToList();
using var dapper = DatabaseService.CreateConnection();
using var t = dapper.BeginTransaction();
var affect = dapper.Execute("""
INSERT OR REPLACE INTO GenshinGachaItem
(Uid, Id, Name, Time, ItemId, ItemType, RankType, GachaType, Count, Lang)
VALUES (@Uid, @Id, @Name, @Time, @ItemId, @ItemType, @RankType, @GachaType, @Count, @Lang);
""", distinctItems, t);
t.Commit();
UpdateGachaItemId();
return affect;
}
3. 分页请求优化
避免边界记录重复
// 改进的分页获取逻辑
protected async Task<List<T>> GetGachaLogByTypeAsync<T>(string prefix, IGachaType gachaType,
long lastId = 0, IProgress<(IGachaType GachaType, int Page)>? progress = null,
CancellationToken cancellationToken = default) where T : GachaLogItem
{
var param = new GachaLogQuery(gachaType, 1, 20, lastId);
var result = new List<T>();
var hasMore = true;
while (hasMore && !cancellationToken.IsCancellationRequested)
{
progress?.Report((gachaType, param.Page));
var list = await GetGachaLogByQueryAsync<T>(prefix, param, cancellationToken);
if (list.Count > 0)
{
// 过滤掉可能重复的最后一条记录
var filteredList = list.Where(item => item.Id > lastId).ToList();
result.AddRange(filteredList);
lastId = list.Max(item => item.Id);
param.Page++;
param.EndId = lastId;
hasMore = list.Count == 20;
}
else
{
hasMore = false;
}
}
return result;
}
完整防重复处理流程
实践建议与注意事项
1. 定期清理重复数据
// 重复数据清理方法
public virtual int CleanDuplicateGachaRecords(long uid)
{
using var dapper = DatabaseService.CreateConnection();
return dapper.Execute("""
DELETE FROM GenshinGachaItem
WHERE rowid NOT IN (
SELECT MIN(rowid)
FROM GenshinGachaItem
WHERE Uid = @uid
GROUP BY Uid, Id
) AND Uid = @uid;
""", new { uid });
}
2. 数据验证机制
// 数据完整性验证
public virtual bool ValidateGachaDataConsistency(long uid)
{
using var dapper = DatabaseService.CreateConnection();
var duplicateCount = dapper.QueryFirstOrDefault<int>("""
SELECT COUNT(*) - COUNT(DISTINCT Id)
FROM GenshinGachaItem
WHERE Uid = @uid;
""", new { uid });
var timeOrderValid = dapper.QueryFirstOrDefault<int>("""
SELECT COUNT(*) FROM (
SELECT Id, Time,
LAG(Time) OVER (ORDER BY Id) as PrevTime
FROM GenshinGachaItem
WHERE Uid = @uid
) WHERE Time < PrevTime;
""", new { uid });
return duplicateCount == 0 && timeOrderValid == 0;
}
性能优化考虑
批量处理优化
// 批量插入性能优化
private const int BatchSize = 100;
protected override int InsertGachaLogItems(List<GachaLogItem> items)
{
var distinctItems = items
.GroupBy(x => new { x.Uid, x.Id })
.Select(g => g.First())
.ToList();
using var dapper = DatabaseService.CreateConnection();
using var t = dapper.BeginTransaction();
var totalAffected = 0;
for (int i = 0; i < distinctItems.Count; i += BatchSize)
{
var batch = distinctItems.Skip(i).Take(BatchSize).ToList();
var affected = dapper.Execute("""
INSERT OR REPLACE INTO GenshinGachaItem
(Uid, Id, Name, Time, ItemId, ItemType, RankType, GachaType, Count, Lang)
VALUES (@Uid, @Id, @Name, @Time, @ItemId, @ItemType, @RankType, @GachaType, @Count, @Lang);
""", batch, t);
totalAffected += affected;
}
t.Commit();
UpdateGachaItemId();
return totalAffected;
}
总结与展望
通过本文的分析和解决方案，Starward项目中的原神抽卡记录重复问题可以得到有效解决。关键点在于：
- 理解重复根源：分页边界、重复导入、缓存解析
- 多层次防护：数据库约束、内存去重、请求优化
- 持续监控：定期清理、数据验证、性能优化
未来可以进一步考虑：
- 实时同步机制避免重复获取
- 智能去重算法提升准确性
- 用户操作历史记录防止重复导入
通过系统性的解决方案，Starward能够为用户提供更加准确、可靠的抽卡记录管理体验。
创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
