Deduplicate Lines (Remove Duplicates from Text)

Strip duplicate lines from any text — keep the first occurrence and remove later duplicates. Case-sensitive option, optional trim before comparing, optional sort. Useful for cleaning lists, log files, exported data.

How to use the Deduplicate Lines (Remove Duplicates from Text)

Paste your lines and choose how matching works — case-sensitive or not, trimmed or literal. Optionally sort the result, or invert it to keep only the lines that repeated. The output and a count of how many lines were removed update as you go.

Removing duplicate lines from a list

Lists collect duplicates: merged exports, concatenated logs, a column of emails pasted from a spreadsheet. Past a few dozen lines, spotting and removing the repeats by hand is impractical — and whether Apple and apple  should count as the same line depends entirely on the job.

This removes duplicate lines with control over what “duplicate” means: case-sensitive or not, comparing trimmed or literal text. It keeps the first occurrence by default, can sort the deduplicated result, or invert to show only the lines that repeated. To reorder lines without removing anything, use sort lines; to compare two separate lists for what they share or differ on, compare two lists.

Common use cases

• List cleanup — collapse a list of emails or IDs to unique values.
• Log dedupe — remove repeated lines from concatenated log files.
• Finding repeats — invert to list exactly which lines occurred more than once.
• Set building — turn a messy column into a distinct set.
• Merge cleanup — dedupe after combining two files into one.