Correcting Line Termination Differences

[scotbuff]

The most common problem found with line termination is the appearance of ^M characters at the end of lines in text files that were built on Windows systems. You will find the ^M characters when they are transferred from one system to another using scp or ftp in binary mode instead of ASCII mode.  The easiest fix on AIX and Linux is to issue the tr command, which translates, squeezes and/or deletes characters from standard input, writing back to standard output.

The ^M is a carriage return and I generally handle these in one of two ways.  If the file has become one long line of carriage returns, I use the tr command to replace the carriage return with a linefeed with the following command.

cat web.php.test |tr '\015' '\012' >web.php.test

In some cases though, the linefeed is already there and you really do not want the file to have a space between each line, particularly for configuration files.  You can remove the carriage return and leave the spacing of the file alone with the following command.

cat web.php.test |tr -d '\015' >web.php.test

In my example I am redirecting the standard output directly into and over the existing file.  If you are working with sensitive data, you would probably want to redirect your output file to a different name than the original file.

[scotbuff]

If you ever try to edit a MS-DOS file, you'll notice that each line ends with a ^M character. This is caused by the funny way that MS-DOS treats the end-of-line. (For some background on this problem take a look at The EOL Story.

To remove the ^M characters from a MS-DOS file, enter the command:

   :1,$s/{Ctrl+V}{Ctrl+M}//{Enter}

This command starts with a colon (:) to tell Vim to enter ex mode. All ex start with a line number range, in this case its from the first line (1) to the last ($). The slash indicates the start of the "from text". The {Ctrl+V} tells Vim to treat the next character as a regular character even if it's a special one. The next character is {Ctrl+M}. (This would be treated as {Enter} without the {Ctrl+V}.) The next slash ends the "from text". What follows is the "to text" enclosed by slashes. In this case it's nothing (//).