About Remove Duplicates From CSV
This page opens with trimming, duplicate removal and Ignore case switched on, on a sample table that has two kinds of duplicate: a row repeated exactly, and a row that repeats once spaces and capitals are ignored. It is free, needs no sign-up and runs in your browser, so nothing you enter is uploaded.
The sample has 14 rows and 12 remain, with 2 duplicates removed. A row is a duplicate only if every column matches, and the first occurrence is kept. Trim spaces first, otherwise " Alice" and "Alice" count as different.
How to use Remove Duplicates From CSV
- Open Paste or upload data to load your file, or start with the sample.
- Switch on the steps you need: whitespace, case, types, missing values, duplicates, find and replace.
- To look for outliers, choose a number column and a method, and adjust the threshold if you want.
- Read Before and after for the row counts and what changed, and check the preview below it.
- Copy the cleaned CSV or download it.
Features
- Trim and collapse whitespace.
- Remove duplicate rows, optionally ignoring case.
- Treat NA, N/A, null, nan and a lone dash as missing, then keep, drop, or fill with a value, the mean, the median, the most common value or the value above.
- Fix types: currency and thousands separators, bracket negatives and yes/no, while keeping ids with leading zeros.
- Standardise text to lower, upper or title case.
- Find and replace in one column or all, with whole-cell, case and regular expression options.
- Outlier detection with IQR, z-score or modified z-score, highlighted in the preview, with an option to remove them.
- A before and after summary with counts of every change.
Worked examples
Sample table gives 14 rows to 12
2 duplicate rows removed.
Privacy: how it works
- Your table is read and cleaned by code running in this browser tab. Nothing is uploaded, and your original file is never modified.
Tips
- Trim whitespace first. A trailing space is the usual reason two equal-looking values count as different.
- Turn on "treat NA as missing" before filling, so the placeholders are counted as gaps.
- Filling with the mean pulls values towards the centre and understates spread. The median is safer when outliers exist.
- Download a cleaned copy and keep the original. This tool never edits the source file.
Frequently asked questions
Is it really free?
Yes. There is no price, no trial and no credit card. Ulkaa Tool does the work with your own device, so there is no server bill to pass on to you.
Do I need an account?
No. There is no sign-up and no login. Open the page and use it. An Ulkaa account is not needed for anything on this tool.
Does it run in my browser?
Yes. Your data are processed by code running in your browser tab, on your own device.
Is my data uploaded?
No. Your data are never sent to Ulkaa or to anyone else. You can check this yourself by watching the Network tab in your browser developer tools while you use the tool.
Does it remove duplicates in one column only?
It compares whole rows. To list unique values of one column, use SELECT DISTINCT in the SQL on CSV tool.
More Data Cleaner pages
- Data CleanerTidy a messy table and download it as a clean CSV, with a before and after summary.
- Fill Missing Values in a CSVReplace gaps with a sensible value, and see how many were filled.
- Trim Whitespace in a CSVRemove stray spaces that make equal values look different.
- Change Case in a CSVMake text consistent: Title Case, lower case or UPPER CASE.
- Fix Data Types in a CSVConvert currency text and thousands separators into plain numbers.
- Find and Replace in a CSVReplace values across every column or just one.
- Outlier Detection in a CSVFlag the values that sit far from the rest of a column.
- IQR Outlier CalculatorFind values outside Q1 - 1.5 x IQR and Q3 + 1.5 x IQR.
- Z-Score Outlier DetectionFlag values more than a set number of standard deviations from the mean.
- Modified Z-Score OutliersA robust outlier rule based on the median and the median absolute deviation.