2 # Copyright and License
4 # Copyright (C) 2009-2014 Erwin Waterlander
7 # Redistribution and use in source and binary forms, with or without
8 # modification, are permitted provided that the following conditions
10 # 1. Redistributions of source code must retain the above copyright
11 # notice, this list of conditions and the following disclaimer.
12 # 2. Redistributions in binary form must reproduce the above copyright
13 # notice in the documentation and/or other materials provided with
16 # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY
17 # EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18 # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
19 # PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE
20 # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
21 # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT
22 # OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
23 # BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
24 # WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
25 # OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
26 # IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30 # To learn what TOP LEVEL section to use in manual pages,
31 # see POSIX/Susv standard and "Utility Description Defaults" at
32 # http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap01.html#tag_01_11
34 # This is manual page in Perl POD format. Read more at
35 # http://perldoc.perl.org/perlpod.html or run command:
37 # perldoc perlpod | less
39 # To check the syntax:
43 # Create manual page with command:
45 # pod2man PAGE.N.pod > PAGE.N
54 dos2unix - DOS/Mac to Unix and vice versa text file format converter
58 dos2unix [options] [FILE ...] [-n INFILE OUTFILE ...]
59 unix2dos [options] [FILE ...] [-n INFILE OUTFILE ...]
63 The Dos2unix package includes utilities C<dos2unix> and C<unix2dos> to convert
64 plain text files in DOS or Mac format to Unix format and vice versa.
66 In DOS/Windows text files a line break, also known as newline, is a combination
67 of two characters: a Carriage Return (CR) followed by a Line Feed (LF). In Unix
68 text files a line break is a single character: the Line Feed (LF). In Mac text
69 files, prior to Mac OS X, a line break was single Carriage Return (CR)
70 character. Nowadays Mac OS uses Unix style (LF) line breaks.
72 Besides line breaks Dos2unix can also convert the encoding of files. A few
73 DOS code pages can be converted to Unix Latin-1. And Windows Unicode (UTF-16)
74 files can be converted to Unix Unicode (UTF-8) files.
76 Binary files are automatically skipped, unless conversion is forced.
78 Non-regular files, such as directories and FIFOs, are automatically skipped.
80 Symbolic links and their targets are by default kept untouched. Symbolic links
81 can optionally be replaced, or the output can be written to the symbolic link
82 target. Writing to a symbolic link target is not supported on Windows.
84 Dos2unix was modelled after dos2unix under SunOS/Solaris. There is one
85 important difference with the original SunOS/Solaris version. This version does
86 by default in-place conversion (old file mode), while the original
87 SunOS/Solaris version only supports paired conversion (new file mode). See also
88 options C<-o> and C<-n>.
96 Treat all following options as file names. Use this option if you want to
97 convert files whose names start with a dash. For instance to convert
98 a file named "-foo", you can use this command:
104 dos2unix -n -- -foo out.txt
108 Convert only line breaks. This is the default conversion mode.
112 Conversion between DOS and ISO-8859-1 character set. See also section
117 Use Windows code page 1252 (Western European).
121 Use DOS code page 437 (US). This is the default code page used for ISO conversion.
125 Use DOS code page 850 (Western European).
129 Use DOS code page 860 (Portuguese).
133 Use DOS code page 863 (French Canadian).
137 Use DOS code page 865 (Nordic).
141 Convert 8 bit characters to 7 bit space.
143 =item B<-b, --keep-bom>
145 Keep Byte Order Mark (BOM). When the input file has a BOM, write a BOM in
146 the output file. This is the default behavior when converting to DOS line
147 breaks. See also option C<-r>.
149 =item B<-c, --convmode CONVMODE>
151 Set conversion mode. Where CONVMODE is one of:
152 I<ascii>, I<7bit>, I<iso>, I<mac>
153 with ascii being the default.
157 Force conversion of binary files.
161 Display help and exit.
163 =item B<-i[FLAGS], --info[=FLAGS] FILE ...>
165 Display file information. No conversion is done.
167 The following information is printed, in this order: number of DOS line breaks,
168 number of Unix line breaks, number of Mac line breaks, byte order mark, text
169 or binary, file name.
173 6 0 0 no_bom text dos.txt
174 0 6 0 no_bom text unix.txt
175 0 0 6 no_bom text mac.txt
176 6 6 6 no_bom text mixed.txt
177 50 0 0 UTF-16LE text utf16le.txt
178 0 50 0 no_bom text utf8unix.txt
179 50 0 0 UTF-8 text utf8dos.txt
180 2 418 219 no_bom binary dos2unix.exe
182 Optionally extra flags can be set to change the output. One or more flags
189 Print number of DOS line breaks.
193 Print number of Unix line breaks.
197 Print number of Mac line breaks.
201 Print the byte order mark.
205 Print if file is text or binary.
209 Print only the files that would be converted.
211 With the C<c> flag dos2unix will print only the files that contain DOS line breaks,
212 unix2dos will print only file names that have Unix line breaks.
218 Show information for all *.txt files:
222 Show only the number of DOS line breaks and Unix line breaks:
226 Show only the byte order mark:
228 dos2unix --info=b *.txt
230 List the files that have DOS line breaks.
234 List the files that have Unix line breaks.
238 =item B<-k, --keepdate>
240 Keep the date stamp of output file same as input file.
242 =item B<-L, --license>
244 Display program's license.
246 =item B<-l, --newline>
248 Add additional newline.
250 B<dos2unix>: Only DOS line breaks are changed to two Unix line breaks.
251 In Mac mode only Mac line breaks are changed to two Unix
254 B<unix2dos>: Only Unix line breaks are changed to two DOS line breaks.
255 In Mac mode Unix line breaks are changed to two Mac line breaks.
257 =item B<-m, --add-bom>
259 Write a Byte Order Mark (BOM) in the output file. By default an UTF-8 BOM
262 When the input file is UTF-16, and the option C<-u> is used, an UTF-16
265 Never use this option when the output encoding is other than UTF-8 or UTF-16.
266 See also section UNICODE.
269 =item B<-n, --newfile INFILE OUTFILE ...>
271 New file mode. Convert file INFILE and write output to file OUTFILE.
272 File names must be given in pairs and wildcard names should I<not> be
273 used or you I<will> lose your files.
275 The person who starts the conversion in new file (paired) mode will be the owner
276 of the converted file. The read/write permissions of the new file will be the
277 permissions of the original file minus the umask(1) of the person who runs the
280 =item B<-o, --oldfile FILE ...>
282 Old file mode. Convert file FILE and overwrite output to it. The program
283 defaults to run in this mode. Wildcard names may be used.
285 In old file (in-place) mode the converted file gets the same owner, group, and
286 read/write permissions as the original file. Also when the file is converted by
287 another user who has write permissions on the file (e.g. user root). The
288 conversion will be aborted when it is not possible to preserve the original
289 values. Change of owner could mean that the original owner is not able to read
290 the file any more. Change of group could be a security risk, the file could be
291 made readable for persons for whom it is not intended. Preservation of owner,
292 group, and read/write permissions is only supported on Unix.
296 Quiet mode. Suppress all warnings and messages. The return value is zero.
297 Except when wrong command-line options are used.
299 =item B<-r, --remove-bom>
301 Remove Byte Order Mark (BOM). Do not write a BOM in the output file.
302 This is the default behavior when converting to Unix line breaks.
303 See also option C<-b>.
307 Skip binary files (default).
309 =item B<-u, --keep-utf16>
311 Keep the original UTF-16 encoding of the input file. The output file will be
312 written in the same UTF-16 encoding, little or big endian, as the input file.
313 This prevents transformation to UTF-8. An UTF-16 BOM will be written
314 accordingly. This option can be disabled with the C<-ascii> option.
316 =item B<-ul, --assume-utf16le>
318 Assume that the input file format is UTF-16LE.
320 When there is a Byte Order Mark in the input file the BOM has priority over
323 When you made a wrong assumption (the input file was not in UTF-16LE format) and
324 the conversion succeeded, you will get an UTF-8 output file with wrong text.
325 You can undo the wrong conversion with iconv(1) by converting the UTF-8 output
326 file back to UTF-16LE. This will bring back the original file.
328 The assumption of UTF-16LE works as a I<conversion mode>. By switching to the default
329 I<ascii> mode the UTF-16LE assumption is turned off.
331 =item B<-ub, --assume-utf16be>
333 Assume that the input file format is UTF-16BE.
335 This option works the same as option C<-ul>.
337 =item B<-v, --verbose>
339 Display verbose messages. Extra information is displayed about Byte Order Marks
340 and the amount of converted line breaks.
342 =item B<-F, --follow-symlink>
344 Follow symbolic links and convert the targets.
346 =item B<-R, --replace-symlink>
348 Replace symbolic links with converted files
349 (original target files remain unchanged).
351 =item B<-S, --skip-symlink>
353 Keep symbolic links and targets unchanged (default).
355 =item B<-V, --version>
357 Display version information and exit.
363 In normal mode line breaks are converted from DOS to Unix and vice versa.
364 Mac line breaks are not converted.
366 In Mac mode line breaks are converted from Mac to Unix and vice versa. DOS
367 line breaks are not changed.
369 To run in Mac mode use the command-line option C<-c mac> or use the
370 commands C<mac2unix> or C<unix2mac>.
372 =head1 CONVERSION MODES
378 In mode C<ascii> only line breaks are converted. This is the default conversion
381 Although the name of this mode is ASCII, which is a 7 bit standard, the
382 actual mode is 8 bit. Use always this mode when converting Unicode UTF-8
387 In this mode all 8 bit non-ASCII characters (with values from 128 to 255)
388 are converted to a 7 bit space.
392 Characters are converted between a DOS character set (code page) and ISO
393 character set ISO-8859-1 (Latin-1) on Unix. DOS characters without ISO-8859-1
394 equivalent, for which conversion is not possible, are converted to a dot. The
395 same counts for ISO-8859-1 characters without DOS counterpart.
397 When only option C<-iso> is used dos2unix will try to determine the active code
398 page. When this is not possible dos2unix will use default code page CP437,
399 which is mainly used in the USA. To force a specific code page use options
400 C<-437> (US), C<-850> (Western European), C<-860> (Portuguese), C<-863> (French
401 Canadian), or C<-865> (Nordic). Windows code page CP1252 (Western European) is
402 also supported with option C<-1252>. For other code pages use dos2unix in
403 combination with iconv(1). Iconv can convert between a long list of character
406 Never use ISO conversion on Unicode text files. It will corrupt UTF-8 encoded files.
410 Convert from DOS default code page to Unix Latin-1
412 dos2unix -iso -n in.txt out.txt
414 Convert from DOS CP850 to Unix Latin-1
416 dos2unix -850 -n in.txt out.txt
418 Convert from Windows CP1252 to Unix Latin-1
420 dos2unix -1252 -n in.txt out.txt
422 Convert from Windows CP1252 to Unix UTF-8 (Unicode)
424 iconv -f CP1252 -t UTF-8 in.txt | dos2unix > out.txt
426 Convert from Unix Latin-1 to DOS default code page
428 unix2dos -iso -n in.txt out.txt
430 Convert from Unix Latin-1 to DOS CP850
432 unix2dos -850 -n in.txt out.txt
434 Convert from Unix Latin-1 to Windows CP1252
436 unix2dos -1252 -n in.txt out.txt
438 Convert from Unix UTF-8 (Unicode) to Windows CP1252
440 unix2dos < in.txt | iconv -f UTF-8 -t CP1252 > out.txt
442 See also L<http://czyborra.com/charsets/codepages.html>
443 and L<http://czyborra.com/charsets/iso8859.html>.
451 There exist different Unicode encodings. On Unix and Linux Unicode files are
452 typically encoded in UTF-8 encoding. On Windows Unicode text files can be
453 encoded in UTF-8, UTF-16, or UTF-16 big endian, but are mostly encoded in
458 Unicode text files can have DOS, Unix or Mac line breaks, like regular text
461 All versions of dos2unix and unix2dos can convert UTF-8 encoded files, because
462 UTF-8 was designed for backward compatibility with ASCII.
464 Dos2unix and unix2dos with Unicode UTF-16 support, can read little and big
465 endian UTF-16 encoded text files. To see if dos2unix was built with UTF-16
466 support type C<dos2unix -V>.
468 UTF-16 encoded files are by default converted to UTF-8. On Unix/Linux it is
469 required that the locale character encoding is set to UTF-8. Use the locale(1)
470 command to find out what the locale character encoding is. UTF-8 formatted
471 text files are well supported on both Windows and Unix/Linux.
473 UTF-16 and UTF-8 encoding are fully compatible, there will no text be lost in
474 the conversion. When an UTF-16 to UTF-8 conversion error occurs, for instance
475 when the UTF-16 input file contains an error, the file will be skipped.
477 When option C<-u> is used, the output file will be written in the same UTF-16
478 encoding as the input file. Option C<-u> prevents conversion to UTF-8.
480 Dos2unix and unix2dos have no option to convert UTF-8 files to UTF-16.
482 ISO and 7-bit mode conversion do not work on UTF-16 files.
484 =head2 Byte Order Mark
486 On Windows Unicode text files typically have a Byte Order Mark (BOM), because
487 many Windows programs (including Notepad) add BOMs by default. See also
488 L<http://en.wikipedia.org/wiki/Byte_order_mark>.
490 On Unix Unicode files typically don't have a BOM. It is assumed that text files
491 are encoded in the locale character encoding.
493 Dos2unix can only detect if a file is in UTF-16 format if the file has a BOM.
494 When an UTF-16 file doesn't have a BOM, dos2unix will see the file as a binary
497 Use option C<-ul> or C<-ub> to convert an UTF-16 file without BOM.
499 Dos2unix writes by default no BOM in the output file. With option C<-b>
500 Dos2unix writes a BOM when the input file has a BOM.
502 Unix2dos writes by default a BOM in the output file when the input file has a
503 BOM. Use option C<-r> to remove the BOM.
505 Dos2unix and unix2dos write always a BOM when option C<-m> is used.
507 =head2 Unicode examples
509 Convert from Windows UTF-16 (with BOM) to Unix UTF-8
511 dos2unix -n in.txt out.txt
513 Convert from Windows UTF-16LE (without BOM) to Unix UTF-8
515 dos2unix -ul -n in.txt out.txt
517 Convert from Unix UTF-8 to Windows UTF-8 with BOM
519 unix2dos -m -n in.txt out.txt
521 Convert from Unix UTF-8 to Windows UTF-16
523 unix2dos < in.txt | iconv -f UTF-8 -t UTF-16 > out.txt
527 Read input from 'stdin' and write output to 'stdout'.
532 Convert and replace a.txt. Convert and replace b.txt.
535 dos2unix -o a.txt b.txt
537 Convert and replace a.txt in ascii conversion mode.
541 Convert and replace a.txt in ascii conversion mode.
542 Convert and replace b.txt in 7bit conversion mode.
544 dos2unix a.txt -c 7bit b.txt
545 dos2unix -c ascii a.txt -c 7bit b.txt
546 dos2unix -ascii a.txt -7 b.txt
548 Convert a.txt from Mac to Unix format.
550 dos2unix -c mac a.txt
553 Convert a.txt from Unix to Mac format.
555 unix2dos -c mac a.txt
558 Convert and replace a.txt while keeping original date stamp.
563 Convert a.txt and write to e.txt.
565 dos2unix -n a.txt e.txt
567 Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt.
569 dos2unix -k -n a.txt e.txt
571 Convert and replace a.txt. Convert b.txt and write to e.txt.
573 dos2unix a.txt -n b.txt e.txt
574 dos2unix -o a.txt -n b.txt e.txt
576 Convert c.txt and write to e.txt. Convert and replace a.txt.
577 Convert and replace b.txt. Convert d.txt and write to f.txt.
579 dos2unix -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt
581 =head1 RECURSIVE CONVERSION
583 Use dos2unix in combination with the find(1) and xargs(1) commands to
584 recursively convert text files in a directory tree structure. For instance to
585 convert all .txt files in the directory tree under the current directory type:
587 find . -name *.txt |xargs dos2unix
595 The primary language is selected with the environment variable LANG. The LANG
596 variable consists out of several parts. The first part is in small letters the
597 language code. The second is optional and is the country code in capital
598 letters, preceded with an underscore. There is also an optional third part:
599 character encoding, preceded with a dot. A few examples for POSIX standard type
603 export LANG=nl_NL Dutch, The Netherlands
604 export LANG=nl_BE Dutch, Belgium
605 export LANG=es_ES Spanish, Spain
606 export LANG=es_MX Spanish, Mexico
607 export LANG=en_US.iso88591 English, USA, Latin-1 encoding
608 export LANG=en_GB.UTF-8 English, UK, UTF-8 encoding
610 For a complete list of language and country codes see the gettext manual:
611 L<http://www.gnu.org/software/gettext/manual/gettext.html#Language-Codes>
613 On Unix systems you can use to command locale(1) to get locale specific
618 With the LANGUAGE environment variable you can specify a priority list of
619 languages, separated by colons. Dos2unix gives preference to LANGUAGE over LANG.
620 For instance, first Dutch and then German: C<LANGUAGE=nl:de>. You have to first
621 enable localization, by setting LANG (or LC_ALL) to a value other than
622 "C", before you can use a language priority list through the LANGUAGE
623 variable. See also the gettext manual:
624 L<http://www.gnu.org/software/gettext/manual/gettext.html#The-LANGUAGE-variable>
626 If you select a language which is not available you will get the
627 standard English messages.
630 =item B<DOS2UNIX_LOCALEDIR>
632 With the environment variable DOS2UNIX_LOCALEDIR the LOCALEDIR set
633 during compilation can be overruled. LOCALEDIR is used to find the
634 language files. The GNU default value is C</usr/local/share/locale>.
635 Option B<--version> will display the LOCALEDIR that is used.
637 Example (POSIX shell):
639 export DOS2UNIX_LOCALEDIR=$HOME/share/locale
646 On success, zero is returned. When a system error occurs the last system error will be
647 returned. For other errors 1 is returned.
649 The return value is always zero in quiet mode, except when wrong command-line options
654 L<http://en.wikipedia.org/wiki/Text_file>
656 L<http://en.wikipedia.org/wiki/Carriage_return>
658 L<http://en.wikipedia.org/wiki/Newline>
660 L<http://en.wikipedia.org/wiki/Unicode>
664 Benjamin Lin - <blin@socs.uts.edu.au>
665 Bernd Johannes Wuebben (mac2unix mode) - <wuebben@kde.org>,
666 Christian Wurll (add extra newline) - <wurll@ira.uka.de>,
667 Erwin Waterlander - <waterlan@xs4all.nl> (Maintainer)
669 Project page: L<http://waterlan.home.xs4all.nl/dos2unix.html>
671 SourceForge page: L<http://sourceforge.net/projects/dos2unix/>