I've been working with variations of Unix for a long time now and thought I'd jot down some of my favorite tips and tricks. They are mostly OS/distribution/shell/language independent (unless I indicate otherwise...)
- Get rid of blank lines in a file
grep . inputfile > outputfile
This matches (and thus prints) only lines that contain some text, not blank (empty) lines. - comm
Many people never cross paths with the comm command, but it is very useful. I works similarly to diff, but outputs the contents of two compared files into three columns. The first column is content only in the first file, second column is content only in the second file, and third column is content that is in both files (matches between the two files.) While this may not seem useful at first, you can select which columns to output, so if you only want to know what is in both file1 and file 2 (column 2) you'd suppress columns 1 and 2, by running:
comm -12 file1 file2
Don't forget that your input files must be sorted. - paste
Systems administrators frequently use thecut>command to parse files, but many people I run into have never used thepastecommand. Thepastecommand will concatenate two files line by line (as opposed to file by file, likecat.) - less instead of more
This is not available on all Unix-based OSes, but thelesscommand works very similar to more, but will let you move through a file forwards and backwards more easily. Want to jump to the end of the file, typeShift-GDepending on the version oflessyou are running, it will provide context highlighting when you search for a pattern. - Jump to vi from more
While paging through a file in more, press "v" to jump to editting the file in vi at the current position in the file. - Jump to a line number when editing a file with vi
vi +linenumber filename
will open up the file with the cursor automatically moved down to the specified line. This is useful when you get an error that indicates "syntax error on line 2047." You can jump straight to the problem without fumbling around. - Invisible characters become visible
Sometimes you'll end up with carriage returns on each line in a file originally created on a DOS/Windows system, or filenames with spaces, tab, or other control characters in them, but you can't see them typically.
Thecatcommand provides three useful options -v, -e, and -t that will let you understand these invisible characters
-v (displays non-printing characters)
-e (prints a "$" at the end of each line to indicate a NL character)
-t (prints "^I" for each Tab in the file)
cat -vet filename |more - Remove DOS ^M from ends of lines
The "^M" characters are visible when editing in vi but here are two approaches to remove the characters.
sed 's/<Ctrl-V><Ctrl-M>//g' -i filename
or in vi:<Esc>:%s/<Ctrl-V><Ctrl-M>//g
If you ever do end up with me interviewing you, I'll likely work one or two of these into the discussion to explore your level of knowledge about Unix. If you need more information, remember: man pages are your friend.
