Remove Duplicate Words (Keep First Order)
Split on spaces, drop every repeat of a word and keep only its first occurrence, so the output stays in the original reading order instead of being sorted. Comparison ignores case, and the result lists one word per line.
How to use
- Paste text with space-separated words; mixed scripts work, but Chinese is not split.
- The output is one word per line, keeping the spelling and position of each word's first appearance.
- Compare the output with the original to see which words you repeat most.
FAQ
How is this different from a plain dedupe?
It preserves first-appearance order: the output follows the order in which each word first occurs, with no sorting, so the original sequence stays visible.
Why is Chinese not split?
Splitting uses whitespace only and does not detect Chinese word boundaries, so a Chinese passage becomes a single token. English and other space-delimited text suit it best.
Are The and the the same word?
Yes. Comparison is case-insensitive and the first spelling is kept. Normalise case first if you need case-sensitive results.