Day 5 - Week 2 Comprehensive Review: Everything You Need to Survive on the Command Line

📌 Key Takeaways

• This week's tools are not isolated knowledge. find → grep → awk → tar → vi mesh together in one flow.
• Files: cp -r (required for copying directories), rmdir only for empty directories, and {} and \; in find -exec.
• Text: grep (line filter) · sed s///g (substitute) · awk $1 (field) · sort|uniq -c (frequency).
• vi: three modes, entering with i/a/o, dd·yy·p, :%s/old/new/g, the difference between :wq and :x.
• Compression: tar czvf (create) · xzvf (extract) · tzvf (view). For help: man · info · whatis · apropos.

Day 1 Recap: Files and Directories

The command-line mindset. The essence of the command line is "connect small tools with pipes." Instead of one giant program handling everything, you assemble complex jobs by chaining simple tools like find, grep, sort, and awk with |. The commands you learned this week are all parts for this assembly.

• Paths come in two kinds: an absolute path that starts from the root (/) and a relative path based on your current location.
• The key symbols for relative paths are . (current), .. (parent), and ~ (home).

⚠️ Pitfall roundup: Day 1 Copying a directory requires cp -r (error without it), rmdir deletes only empty directories, not gzip but rm has no recycle bin, and locate is DB-based so it may not find the newest files. In find's -exec, {} is the file-name substitution and \; is the end of the command.

Day 2 Recap: Text Processing

The starting point of text processing is the standard streams and the pipe (|). It connects the output of one command to the input of the next.

• Key regex metacharacters: . (any single character), * (zero or more of the preceding character), ^ (start of line), $ (end of line), [] (character set).
• In extended regular expressions (ERE, grep -E), +, ?, |, and () work without a backslash.

Commonly used pipeline patterns. "Count errors" is grep error log | wc -l, "unique IP frequency" is awk '{print $1}' log | sort | uniq -c | sort -nr, and "info for a specific user" is grep user /etc/passwd | cut -d: -f1,6. Once these combinations are in your fingers, log analysis becomes fun.

Day 3 Recap: vi/vim

The three modes of vi are as follows. Command mode (default at startup, movement/editing), insert mode (entered with i/a/o, text input), and last-line mode (entered with :, save/quit/substitute).

⚠️ Pitfall roundup: Day 3 a types after the cursor (i before it), o is the line below/O the line above, :wq always saves/:x and ZZ save only on change, and the % in :%s is the whole file. In nano, save is Ctrl+O and exit is Ctrl+X.

Day 4 Recap: Compression and Help

Archiving (bundling) and compression (shrinking size) are different jobs. tar bundles, and gzip/bzip2/xz compress.

• tar options: c (create), x (extract), t (list), v (verbose), f (file name), z (gzip), j (bzip2), J (xz).
• The compression ratio order is xz > bzip2 > gzip.
• For help, use man (section-based manuals), info (detailed documents), whatis (one-line summary, man -f), and apropos (keyword search, man -k).
• Manual section 1 is commands, 5 is file formats, and 8 is system administration.

The Unix philosophy running through the week. This week's tools all follow the Unix philosophy set down by Ken Thompson and Dennis Ritchie in the 1970s. "Do one thing well," "make programs work together," "handle text streams." grep, sort, and tar each do just one thing well and cooperate through pipes. This is why the design is still valid fifty years later.

Try It Yourself: Comprehensive Exercise

This is a comprehensive exercise that weaves everything you learned into one flow.

# 1. Create a workspace and log data
mkdir -p ~/review/logs
cd ~/review
cat > logs/access.log << 'EOF'
192.168.0.1 GET /index 200
192.168.0.2 POST /login 401
192.168.0.1 GET /about 200
192.168.0.3 GET /index 500
192.168.0.2 GET /index 200
EOF
 
# 2. Find the log with find
find . -name "*.log" -type f
 
# 3. Count errors (401, 500) with grep + wc
grep -E "401|500" logs/access.log | wc -l
 
# 4. Rank IP access frequency with awk + sort + uniq
awk '{print $1}' logs/access.log | sort | uniq -c | sort -nr
 
# 5. Mark status code 200 as OK with sed
sed 's/ 200$/ OK/' logs/access.log
 
# 6. Back up with tar and verify
tar czvf backup.tar.gz logs/
tar tzvf backup.tar.gz
 
# 7. Check with help
whatis awk
 
# 8. Clean up
cd ~
rm -rf ~/review

• If you can get through this exercise without getting stuck, you have more than achieved the goal of Week 2.
• Experimenting by changing the options at each step deepens your understanding.

📖 Glossary

• Absolute path / relative path : A complete address starting from /, and an address based on your current location.
• cp -r : The recursive option that is essential when copying a whole directory.
• find -exec : An option that runs a command for each found file. {} is replaced with the file name and it is terminated with \;.
• locate : A fast search that digs through a pre-built database. It may not find the newest files.
• Pipe (|) : A connector that links the output of the preceding command to the input of the following command. The core of the command-line mindset.
• sort | uniq -c : A combination that sorts and then counts duplicates. The fundamental of log frequency analysis.
• ERE (extended regular expressions) : The syntax used by grep -E. You can use + ? | ( ) without a backslash.
• Command mode / insert mode / last-line mode : vi's three states. They handle editing, typing, and save/quit respectively.
• :%s/old/new/g : The vi command that substitutes across the whole file. % means "all lines."
• Archive vs compression : Bundling many into one (tar) and reducing size (gzip, etc.) are different from each other.
• man sections : Manual classification numbers. 1 = commands, 5 = file formats, 8 = system administration commands.
• apropos : A tool that finds command names in reverse by a function keyword. Same as man -k.