🗑️ Remove Duplicates
TextSorter Remove Duplicates is a free online text cleaning utility that allows you to delete repeated lines and keep only unique values instantly
No duplicates removed yet.
Click "Remove Duplicates" to start.
What Each Button Actually Removes, Reorders or Reveals
The short version: two buttons remove duplicates and keep your original order, one button removes duplicates and then alphabetizes what is left, and one button removes nothing at all, it only shows you what is repeated.
john@email.com
jane@email.com
John@Email.com
bob@email.com
jane@email.comjohn@email.com
jane@email.com
John@Email.com
bob@email.com| Button | Case handling | Keeps original order | Hands off to a background thread past 5,000 lines |
|---|---|---|---|
| Remove Duplicates | Case-sensitive exact match | Yes | Yes |
| Ignore Case | Case-insensitive match | Yes | Yes |
| Sort + Dedup | Case-sensitive exact match | No, alphabetized | No |
| Show Dupes | Case-sensitive exact match | Removes nothing | No |
Every one of those four buttons runs the same first step before it compares a single line: the pasted text is split on line breaks, each line has its leading and trailing whitespace trimmed, and any line that is now empty is dropped from the list entirely. What survives that pass is what the rest of the button's logic works on. That first step is identical for Remove Duplicates and Ignore Case, so the only real difference between them is the single comparison rule underneath, not the trimming or blank-line handling around it.
Only "Remove Duplicates" and "Ignore Case" send their deleted lines to the "Removed Duplicates" panel on the right. "Sort + Dedup" also fills that panel, since it deduplicates before it sorts. "Show Dupes" never touches it, because it is not removing anything, it is only rearranging what stays in the main editor.
Large lists behave differently depending on the button. Past 5,000 lines, "Remove Duplicates" and "Ignore Case" hand off to a background Web Worker so the tab stays responsive. "Sort + Dedup" and "Show Dupes" have no such handoff and always run directly on the page, so an unusually large paste can make the tab pause briefly when you click either of those two.
Case Sensitive by Default: When Apple and apple Count as Two
Remove Duplicates treats capitalization as part of the text. Click Ignore Case instead and apple, Apple and APPLE collapse into whichever spelling was pasted first.
john@email.com
jane@email.com
John@Email.com
bob@email.comjohn@email.com
jane@email.com
bob@email.comBoth columns start from the same pasted list: john@email.com, jane@email.com, John@Email.com, bob@email.com, jane@email.com. Plain "Remove Duplicates" only catches the exact repeat of jane@email.com on the last line, since it compares lines exactly as written. John@Email.com survives right next to john@email.com, because to a case-sensitive comparison, capital letters and lowercase letters are simply different characters.
That default is correct for data where case genuinely carries meaning: product SKUs, case-sensitive usernames, and codes where ABC123 and abc123 might refer to different records entirely. It is the wrong default for an email list, since john@email.com and John@Email.com point at the same inbox, and sending twice to the same address is exactly what deduplicating was supposed to prevent.
Ignore Case is a button, not a checkbox. Clicking it lowercases every line for comparison purposes only, keeps the first spelling it saw for each unique value, and runs the whole removal in that one click. It does not flip a persistent setting that changes what the plain Remove Duplicates button does the next time you press it. There is no partial option either, no way to ignore case for part of a line, or only outside a domain name. The choice is binary: compare every line exactly as typed, or fold every line to lowercase before comparing.
Sort + Dedup has no case-insensitive version of itself. It always compares lines exactly, the same as the plain Remove Duplicates button. If you need both a case-insensitive result and an alphabetized one, click Ignore Case first to fold the case variants together, then click Sort + Dedup to alphabetize what is left. Running Sort + Dedup alone on mixed-case data will keep every case variant it does not recognize as identical.
Why Trimmed Spaces Fix Some Duplicates and Not Others
Every comparison trims the start and end of each line first. A stray leading or trailing space never causes a false non-match. A doubled space in the middle of a line still counts as different text.
john@email.com
john@email.comjohn@email.comBoth lines above look identical, and to your eyes they are. The second one was pasted with an invisible trailing space, the kind that comes along for free when you copy a value out of a spreadsheet cell or a PDF table. Before this tool compares anything, it strips leading and trailing whitespace from every line, so that trailing space never gets the chance to make two otherwise identical lines look different. The result is a single surviving line, exactly as it should be.
What trimming does not do is touch whitespace inside a line. New York and New York, the second with two spaces between the words instead of one, are trimmed the same way at their edges but still differ in the middle, so none of the four buttons treats them as duplicates. Collapsing internal whitespace automatically would risk merging two genuinely different multi-word entries, so this tool leaves it alone on purpose. Tabs are trimmed the same way spaces are: a line that starts with a tab character instead of a space is stripped just as cleanly, since the trimming step removes every kind of whitespace from both ends of a line, not only the space character specifically.
Inconsistent internal spacing survives every button on this page. If your data has entries like New York and New York sitting side by side, deduplicating will not catch it, because trimming only ever touches the two ends of a line. Normalize internal spacing by hand or with a find-and-replace pass first, then run the removal again.
Which Copy Survives, and Whether the List Stays in Order
Remove Duplicates and Ignore Case both keep every surviving line exactly where it was. Sort + Dedup keeps the same first occurrence, then alphabetizes the result, so the original order is gone.
Banana
Apple
CherryApple
Banana
CherryPaste Banana, Apple, Banana, Cherry and both buttons agree on which copy of Banana to keep: the first one, encountered while scanning down the list from top to bottom. Where they disagree is what happens to the survivors afterward. Remove Duplicates leaves Banana, Apple, Cherry in that exact order, since nothing about the removal step reorders anything, it only takes lines out. Sort + Dedup takes the same three survivors and alphabetizes them, landing on Apple, Banana, Cherry instead.
Keeping the original order matters whenever the sequence of your data carries information of its own: a chronological log, a list where the first entry is the canonical record and later ones are accidental re-pastes, or any dataset where re-sorting would erase something about priority or timing. Reach for Sort + Dedup specifically when you want a clean alphabetized reference list and the order the data arrived in was never meaningful to begin with. That first-occurrence rule is fixed on every button here, too: nothing on this page keeps the last occurrence of a repeated line instead of the first, so there is no setting to flip if what you actually wanted was the most recent copy.
The Removed Duplicates panel follows scan order too. Whether you clicked Remove Duplicates, Ignore Case or Sort + Dedup, the lines that show up in the right-hand panel are listed in the order they were encountered while scanning, not grouped by which surviving line they duplicated and not alphabetized.
Why a Misspelled Duplicate Slips Through Untouched
Every comparison here is exact. John Smith and Jon Smith, or Acme Inc. and Acme Inc, are two different lines to this tool, not a duplicate and a typo of each other.
John Smith
Jon Smith
Acme Inc.
Acme IncJohn Smith
Jon Smith
Acme Inc.
Acme IncClick Remove Duplicates on that list and nothing changes. All four lines are kept, and the tool tells you so with a "No duplicates found!" message, because every comparison it makes is a character-by-character match on the trimmed, and optionally lowercased, line. It has no concept of "close enough." A one-letter typo, a missing trailing period, or a hyphen swapped for a space all produce two lines that are simply unrelated as far as any button here is concerned.
That is a deliberate limitation, not an oversight. Deciding how similar two lines have to be before they count as the same thing, usually measured with something like edit distance, is a fundamentally different kind of algorithm from exact comparison, and it makes judgment calls that vary by dataset. Getting it wrong in the other direction, merging two records that only look alike, can quietly destroy real data, which is a worse outcome than leaving an obvious near-duplicate sitting in the list for a human to catch.
A practical way to handle messy data: run Show Dupes first to see how many exact repeats already exist, standardize the obvious formatting differences by hand or with a find-and-replace pass, consistent capitalization, consistent punctuation, consistent abbreviations, and then run Remove Duplicates again. Cleanup that turns near-duplicates into exact matches is what lets this tool catch them.
What Show Dupes Actually Displays, and in What Order
Show Dupes lists every value that appears more than once, once each, without deleting anything. If your list mixes plain numbers with words, the numbers come first in ascending order, not in the order you pasted them.
100
5
zebra
100
5
zebra5
100
zebra"Show Dupes" counts how many times each trimmed line appears, keeps only the values that appear more than once, and replaces the editor contents with that list, one entry per duplicated value rather than one entry per repeat. A line pasted five times shows up once in the result, since the point is to show which values repeat, not how many copies exist. Nothing in your original text is changed or removed by this button.
The number ordering in the example above is not a sorting feature. It falls out of how the JavaScript object counting your lines orders its own keys: entries that look like plain non-negative whole numbers, such as 5 and 100, are always listed first in ascending numeric order, ahead of every other duplicated value, which then appears in the order it was first encountered. That only shows up when your duplicated data mixes bare numeric lines with text lines. A list of duplicated names or words with no plain numbers in it comes back in first-encountered order, with no quirk to notice. The same numeric-first behavior shows up no matter how many times a value repeats or how long the list is, since it comes from how the counting object orders its keys rather than from anything about the size of your data.
Show Dupes never touches the Removed Duplicates panel on the right. That panel keeps showing whatever it displayed last, or "Waiting..." if you have not run Remove Duplicates, Ignore Case or Sort + Dedup yet, since Show Dupes does not remove anything and never writes to it.
Six Ways People Actually Use This Tool
Scrubbing an email list before a campaign
Paste a combined list from multiple signup sources and click Ignore Case rather than the plain button. The same address typed with different capitalization is still the same inbox, and sending twice to the same person is exactly the outcome you are trying to avoid.
Reviewing before you commit to deleting anything
Click Show Dupes first on any list you are not fully sure about. Seeing exactly which values repeat, without losing your original data, gives you a chance to catch a case where an apparent duplicate is actually meaningful before you run the removal itself.
Building a clean, alphabetized master list
When you are consolidating several source lists into one reference document and the original order does not matter, Sort + Dedup does both jobs in one click: unique values only, arranged alphabetically, ready to use as a lookup list.
Cleaning a column exported from a spreadsheet
Spreadsheet exports frequently carry trailing spaces from cell padding, along with inconsistent capitalization from manual data entry. Since every comparison here trims whitespace automatically, and Ignore Case handles the capitalization half, pasting an exported column straight in usually catches both problems in one pass.
Keeping a chronological or first-entry-wins record clean
When the order of your data matters, a log, or a list where the first entry is the original record, click plain Remove Duplicates or Ignore Case rather than Sort + Dedup. Both keep every surviving line exactly where it was, so the sequence you started with is still readable afterward, and neither one will quietly rearrange a list you were counting on staying in place.
Working with very large exports
Remove Duplicates and Ignore Case both hand off to a background thread once a list passes 5,000 lines, so the tab keeps responding while a large file processes. Sort + Dedup and Show Dupes run directly on the page regardless of size. Either way, nothing you paste leaves your device: there is no upload step, no processing queue, and no account to create.
Frequently Asked Questions
Related Text Tools
Need to do more with your data? Try these complementary tools:
🔒 100% Private & Secure
Your data privacy is our priority. All duplicate removal happens locally in your browser-your text is never uploaded to any server. We can't see, store, or access your data. Use this tool with complete confidence for sensitive information like email lists, customer data, or confidential documents.