Select statement to find duplicates on certain fields Select statement to find duplicates on certain fields sql-server sql-server

Select statement to find duplicates on certain fields


To get the list of fields for which there are multiple records, you can use..

select field1,field2,field3, count(*)  from table_name  group by field1,field2,field3  having count(*) > 1

Check this link for more information on how to delete the rows.

http://support.microsoft.com/kb/139444

There should be a criterion for deciding how you define "first rows" before you use the approach in the link above. Based on that you'll need to use an order by clause and a sub query if needed. If you can post some sample data, it would really help.


You mention "the first one", so I assume that you have some kind of ordering on your data. Let's assume that your data is ordered by some field ID.

This SQL should get you the duplicate entries except for the first one. It basically selects all rows for which another row with (a) the same fields and (b) a lower ID exists. Performance won't be great, but it might solve your problem.

SELECT A.ID, A.field1, A.field2, A.field3  FROM myTable A WHERE EXISTS (SELECT B.ID                 FROM myTable B                WHERE B.field1 = A.field1                  AND B.field2 = A.field2                  AND B.field3 = A.field3                  AND B.ID < A.ID)


This is a fun solution with SQL Server 2005 that I like. I'm going to assume that by "for every record except for the first one", you mean that there is another "id" column that we can use to identify which row is "first".

SELECT id    , field1    , field2    , field3FROM(    SELECT id        , field1        , field2        , field3        , RANK() OVER (PARTITION BY field1, field2, field3 ORDER BY id ASC) AS [rank]    FROM table_name) aWHERE [rank] > 1