SQL去重

本文介绍使用SQL语句处理数据表中的重复记录,包括查找及删除重复项的方法,并提供基于单个字段或多字段的实例。
用SQL语句,删除掉重复项只保留一条

在几千条记录里,存在着些相同的记录,如何能用SQL语句,删除掉重复的呢
1、查找表中多余的重复记录,重复记录是根据单个字段(peopleId)来判断
select * from people
where peopleId in (select peopleId from people group by peopleId having count(peopleId) > 1)

2、删除表中多余的重复记录,重复记录是根据单个字段(peopleId)来判断,只留有rowid最小的记录
delete from people
where   peopleName in (select peopleName    from people group by peopleName      having count(peopleName) > 1)
and   peopleId not in (select min(peopleId) from people group by peopleName     having count(peopleName)>1)

3、查找表中多余的重复记录(多个字段)
select * from vitae a
where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)

4、删除表中多余的重复记录(多个字段),只留有rowid最小的记录
delete from vitae a
where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)
and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)

5、查找表中多余的重复记录(多个字段),不包含rowid最小的记录
select * from vitae a
where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)
and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)  

6.消除一个字段的左边的第一位:

update tableName set [Title]=Right([Title],(len([Title])-1)) where Title like '村%'

7.消除一个字段的右边的第一位:

update tableName set [Title]=left([Title],(len([Title])-1)) where Title like '%村'

8.假删除表中多余的重复记录(多个字段),不包含rowid最小的记录
update vitae set ispass=-1
where peopleId in (select peopleId from vitae group by peopleId

SQL实现数据有以下几种常见方法: 1. **使用`DISTINCT`关键字**:`DISTINCT`放在`SELECT`后边,会对后面所有的字段的值统一进行。例如,如果`distinct`后面有两个字段,那么`1,1`和`1,2`这两条记录不是复值。示例代码如下: ```sql SELECT DISTINCT column1, column2 FROM your_table; ``` 在后保存为新表时,可使用如下语句: ```sql CREATE TABLE new_table AS SELECT * FROM ( SELECT DISTINCT old_table.id, old_table.buyTime FROM old_table ) a; ``` 这里以`CREATE TABLE before_12 AS SELECT * FROM ( SELECT DISTINCT all_before_12.id, all_before_12.buyTime FROM all_before_12 ) a;`为例,将`all_before_12`表后的数据保存到`before_12`表中 [^1][^3][^4]。 2. **使用`GROUP BY`子句**:可以通过`GROUP BY`列出指定字段的所有唯一值(后的记录,`NULL`也是值)。例如,列出`task_id`的所有唯一值: ```sql SELECT task_id FROM Task GROUP BY task_id; ``` 若要统计后的任务总数,可以使用子查询: ```sql SELECT COUNT(task_id) task_num FROM (SELECT task_id FROM Task GROUP BY task_id) tmp; ``` [^4] 3. **使用`ROW_NUMBER()`函数(结合CTE)**:在SQL Server中,可以利用CTE和`ROW_NUMBER()`函数来查找和删除复数据。`ROW_NUMBER()`函数会为结果集中的每一行分配一个唯一的行号,通过分区和排序可以标记出复行。示例代码如下(假设要删除`your_table`中`column1`和`column2`复的记录): ```sql WITH CTE AS ( SELECT *, ROW_NUMBER() OVER (PARTITION BY column1, column2 ORDER BY (SELECT NULL)) AS rn FROM your_table ) DELETE FROM CTE WHERE rn > 1; ``` [^2] 4. **使用`EXISTS`子句**:可以使用`EXISTS`子句进行复数据删除。通过子查询判断是否存在复记录,若存在则进行相应操作。示例代码如下(假设要删除`your_table`中`column1`和`column2`复的记录): ```sql DELETE FROM your_table t1 WHERE EXISTS ( SELECT 1 FROM your_table t2 WHERE t1.column1 = t2.column1 AND t1.column2 = t2.column2 AND t1.id > t2.id ); ``` [^2]
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值