convert

Remove Duplicate Lines

Delete repeated lines from a list while keeping the first occurrence and the original order.

Runs on your deviceBatch readyPaste textNew

Paste or drop

Paste text below, or drop a file instead.

0 chars · 0 lines
…or drop a file
Drag and drop, or choose files
TXTMDCSVTSVLOGJSONXML+10

Choose an output

Delete repeated lines from a list while keeping the first occurrence and the original order.

Fine-tune

Optional — sensible defaults are already set.

About Remove Duplicate Lines

Delete repeated lines from a list and keep only the first time each one appears. The original order is kept, which matters when the order of your list means something, unlike sort-based de-duplication. The summary tells you how many lines were removed and how many remain.

When it helps

  • De-duplicating an email list or customer list before an import.
  • Cleaning up a keyword list collected from several tools.
  • Removing repeated entries from log output or a scraped list of URLs.
  • Merging two lists by pasting one under the other and keeping each item once.

Tips

  • 'Trim spaces before comparing' is on by default, so lines that differ only by leading or trailing spaces count as duplicates. The kept lines are also trimmed.
  • Turn on 'Ignore case' to treat 'Apple' and 'apple' as the same entry. The first spelling seen is the one kept.
  • Only whole lines are compared. To de-duplicate items in a comma-separated list, split it into lines first.

Good to know

Quick answers about remove duplicate lines in your browser.

No. Lines stay in their original order and only later repeats are dropped. Run the sort lines tool afterwards if you also want the list sorted.

This tool compares entire lines, so two CSV rows only match if every column is identical. Copy just that column out of your spreadsheet if you want to de-duplicate by one field.

Yes. There is no sign-up, no watermark and no limit on how many files you process.

No. Files are read and processed locally in your browser, so they never leave your device.

Related tools

DEVURL EncodePercent-encode the contents of a text file for safe URLs.DEVURL DecodeDecode percent-encoded text back to readable text.TXTNewWord CounterCount words, characters, sentences and paragraphs, and estimate reading and speaking time, with a top-words list.TXTNewWord Frequency CounterList every word in your text with how often it appears, ranked and exported as a CSV spreadsheet.TXTNewCase ConverterConvert text to UPPERCASE, lowercase, Title Case, Sentence case, camelCase, snake_case, kebab-case and more.TXTNewFind and ReplaceFind and replace words or patterns in text, with optional regular expressions, case matching and whole-word search.TXTNewText Compare (Diff)Compare two texts line by line and get a unified diff showing exactly what was added, removed or changed.TXTNewReverse TextReverse text backwards character by character, flip the order of lines or words, or mirror each line.TXTNewFancy Text GeneratorTurn plain text into bold, italic, script, gothic, bubble, small caps or upside-down Unicode text you can paste anywhere.

Part of the Text Tools collection.