Remove duplicate lines

Last checked November 2, 2026 by the TextEditor.com team

Which copy stays?

By default the tool keeps the first copy of each line in its place and drops the later ones. The order of your list does not change. Excel, Notepad++ and VS Code do the same thing.

When the copies match exactly, keeping the last one only moves the line further down. The choice matters when copies differ a little, for example when you compare one column. Say people signed up twice with the same email address. The last row may hold their new city, so "Last copy" keeps the newest row.

"Only lines with no copy" removes every line that shows up more than once, the first copy too. "Only repeated lines" does the opposite. It lists each repeated line once, so you see what was doubled. To sort the result by number or by length, use sort lines.

Why some copies are not found

Two lines only count as copies when every character matches. Some differences are hard to see. A space at the end of a line is invisible. "Milk" and "milk" differ in one letter. Text from a web page often has a no-break space instead of a normal one, and some sites add a zero width space that does not show at all.

The letter é can be stored two ways: as one character, or as e plus an accent mark. Both look the same. Tick "Invisible characters" and the tool treats both forms as one letter. It also skips zero width characters and reads odd spaces as normal spaces.

Files from Windows end each line with two characters, CRLF. Linux and Mac files use one, LF. Join two such files and half the lines carry a hidden carriage return. This tool reads both kinds of line end the same way.

When a looser setting would find more copies, the note under the result says how many. To clean the text itself, use the text cleaner. To see exactly which hidden characters are in a line, paste it into the invisible character tool.

Remove duplicates in Excel and Google Sheets

In Excel, select the column, then go to Data, Data Tools, Remove Duplicates. Pick the columns to compare and click OK. Excel keeps the first copy and deletes the other rows. Press Ctrl+Z if you change your mind.

In our test with Excel for Microsoft 365, Remove Duplicates and the UNIQUE function both ignored case, so "Apple" and "apple" were one value. A space at the end still made a value different. Excel also compares what a cell shows, so one date in two formats counts as two values. See Microsoft's guide to removing duplicate values for the details.

In Google Sheets, select the data and go to Data, Data cleanup, Remove duplicates. Google's help page says cells still count as copies when they differ only in letter case, formatting or formulas. To also catch copies that differ only in spaces, run Data cleanup, Trim whitespace first. Checked November 2026.

Notepad++, VS Code and Word

Notepad++ has Edit, Line Operations, Remove Duplicate Lines. It keeps the first copy of each line anywhere in the file, or in the selected lines. Remove Consecutive Duplicate Lines only removes a copy that sits right below the line it repeats.

In VS Code, press Ctrl+Shift+P and run "Delete Duplicate Lines". With no selection it works on the whole file. It keeps the first copy and compares lines exactly, so case and spaces count.

Word has no command to remove duplicate lines. Home, Sort puts the paragraphs in order, so copies end up next to each other and you can delete them by hand. A wildcard find and replace can also do it, but in our test it removed only one copy per run and paid attention to case. It is quicker to paste the list here and paste the result back into Word.

Checked in the Notepad++ user manual, the VS Code source code and Word for Microsoft 365, November 2026.

Remove duplicate lines on the command line

On Linux and macOS, sort -u sorts the lines and keeps one of each. uniq works another way: it only merges copies that sit right next to each other, so people sort first. Both lose the order of the list. The awk line below keeps the order, because it prints a line only the first time it sees it.

A file saved on Windows still has a carriage return at the end of each line. Then sort and uniq see "apple" with and without that character as two different lines, so convert the line ends first. The uniq manual lists more options, like -i to ignore case. Checked with GNU coreutils 8.32 and Windows PowerShell 5.1, November 2026.

Count how often each line appears

The Repeated tab lists every line that has a copy, with how many times it shows up and on which lines. The most common lines come first.

Tick "Show how often" to put the count in front of each line of the result, with a tab after it. Pasted into a spreadsheet, the counts land in a column of their own. Set the order to "most common first" and a list of search terms or error messages becomes a top list.

Duplicates in one column of a table

Rows of a CSV file can differ in one value and still be the same person or product. Open More options, set Compare to "one column" and give its number. The whole row stays or goes, as in Excel. Tick "Keep the first line on top" so the row of column names is left alone.

The tool finds the separator by itself: a tab, a comma, a semicolon or a bar. Values in double quotes may contain the separator. A row that is too short for that column is compared as a whole line. For a grid view of the whole table, open the CSV editor.

Other tools for lists

• Put lines in orderc b a → a b cSort lines from A to Z, by number or at random.
• Clean up pasted texta   b → a bThe text cleaner removes extra spaces and hidden characters.
• Replace a word everywherecolour → colorFind and replace, with match case and regex patterns.
• Compare two textsthe old new catText compare marks what was added and what was removed.