Text file doctor: fix line endings, BOM and encoding

Check any text file or folder for encoding, BOM, line endings, trailing spaces and hidden characters, then fix them and download the files.

How it works

  1. Add text files, choose a folder, or paste text
  2. Read what is wrong with each file and pick what to fix
  3. Download the fixed files, or a zip that keeps the folder paths

What it checks in each file

To fix line endings, a byte order mark or the encoding of text files, add the files or a whole folder, read what the page finds in each one, and download the fixed copies one by one or as a zip that keeps the folder paths. For every file it reports the encoding, a BOM, the line endings with counts, whether the last line ends with a line break, lines with trailing spaces or tabs, tabs and spaces in indentation, invisible characters, and lines over 500 characters. Worked example: the Try sample files button adds five files. app.js has a BOM, CRLF endings, a tab and a space-indented line, trailing spaces, a zero width space and no final line break. notes.txt is UTF-16, legacy.csv is Windows-1252, README.md is clean except one stray space, and logo.bin is binary. With the default options 4 files are fixed and 1 is skipped, and app.js comes out as: Byte order mark removed, 5 line endings changed, line break added at the end, 1 line trimmed, 1 invisible character removed.

How the encoding is worked out

A file that starts with the bytes EF BB BF is UTF-8 with a BOM, and FF FE or FE FF marks UTF-16. Without a BOM, a file with a zero byte, a known binary signature (such as PNG or PDF) or many control bytes is treated as binary and skipped, which also covers UTF-16 saved without a BOM and UTF-32. Otherwise the bytes are checked as strict UTF-8, which rejects overlong forms, surrogates and values past U+10FFFF. If they are valid, the file is UTF-8 or plain ASCII. If not, and the file has no UTF-8 BOM, the page reads it as Windows-1252 (a file with a UTF-8 BOM but broken bytes after it, or with UTF-16 that has unpaired surrogates, is skipped as damaged), using the table in the WHATWG Encoding Standard, and says it is a guess, because Latin-1 and other legacy encodings use the same bytes for some letters. A decoded file that already contains U+FFFD, the replacement character, is flagged because some text was lost before it reached you.

What each fix does

Line endings can become LF or CRLF for the whole file, or stay as they are. The BOM option removes only a UTF-8 BOM. The final line break is added when the last line has none, and blank lines at the end are left alone. Trimming removes spaces and tabs at the end of every line, but in .md and .markdown files a line ending in two or more spaces is kept, because Markdown reads it as a line break. Indentation converts only the whitespace at the start of a line, measured in columns with the tab width you pick, so a tab after code is never touched; leave it on Keep for Makefiles. Invisible characters are removed with the same table as the Invisible Character Detector page: zero width characters, direction controls, soft hyphens and tag characters, while joiners inside emoji stay. UTF-16 and Windows-1252 files are converted to UTF-8, or fixed and saved in their own encoding if you turn that off.

Limits and what it does not do

It takes up to 50 files at a time and 5 MB for each file. Choose a folder to keep paths; files inside node_modules, .git, .hg and .svn are left out, and a folder dropped onto the page is not read, so use the button. Pasting text with Ctrl+V or Cmd+V adds it as a file. A file with nothing to change comes back exactly as it was when it is fixed together with other files; if every file is already clean, Fix stays off. The page does not rename files, change permissions, sort lines or format code, and it cannot tell whether tabs or spaces are right for a language. Python and YAML depend on indentation, so check those after converting.

Frequently asked questions

How do I convert CRLF to LF in many files at once?

Choose a folder or add the files, leave Line endings on LF and press Fix. Every CRLF and lone CR becomes LF. The result row says, for example, 5 line endings changed, and the zip keeps the folder paths.

How do I remove a BOM from a UTF-8 file?

Leave the BOM option on. A UTF-8 BOM is the three bytes EF BB BF at the start of a file. The page deletes them and keeps everything else, and the row reports byte order mark removed. A UTF-16 file loses its BOM when it is converted to UTF-8.

How can I tell whether a file is UTF-8 or Windows-1252?

Add it and read the first line under its name. If every byte sequence is valid UTF-8 it says UTF-8. If not, it says Windows-1252 or Latin-1 (a guess). A byte E9 alone is é in Windows-1252 and is not valid UTF-8, which is the usual sign.

Why was my file skipped as binary?

It has a zero byte and no UTF-16 BOM, starts with the signature of a known binary format (PNG, PDF, ZIP, GIF, JPEG and others), or is full of control bytes. Images, PDFs and archives nearly always match one of these. A UTF-16 text file saved without a BOM also looks like that, so open it in an editor and resave it with a BOM to include it.

Does it change my code?

Only whitespace at the end of lines or at the start of a line, line breaks, invisible characters and the file encoding. A tab inside a line stays. Check Python, YAML and Makefile files after converting indentation.

Are my files uploaded?

No. The files are processed in your browser and not uploaded. Only the fix options you choose are saved in this browser, never the files.