Level 3 · 3A Systems and the terminal · Lesson 2 of 3
Searching files and reading logs
Servers write down almost everything that happens in files called logs: who logged in, who failed, which page was requested. They are the best clue to know whether someone is attacking. The problem is that they can have thousands of lines, and reading them by eye does not work.
grep: keep only what matters
grep word file shows only the lines that contain that word. With -c it counts how many there are, with -i it ignores upper and lower case, and with -v it shows the lines that do NOT contain it.
grep Failed /var/log/auth.log
grep -c Failed /var/log/auth.log- The first line shows every line of the log that contains Failed.
- The second one only counts how many there are.
Pipes: chaining commands
The | symbol (pipe) takes the output of one command and passes it to the next one. With cut you take out a column, sort puts lines in order, uniq -c counts repeated lines and sort -rn puts the biggest counts at the top. That way you build a complete analysis in a single line.
grep 401 acceso.log | cut -d" " -f1 | sort | uniq -c | sort -rn- grep 401 keeps the rejected requests.
- cut -d" " -f1 takes out the first column, which is the IP.
- sort puts them in order and uniq -c counts how many times each one appears.
- sort -rn puts the IP with the most attempts at the top.
Going deeper
Why order matters
uniq only groups repeated lines that are right next to each other, which is why you always write sort first. Then a second sort -rn orders by count, from highest to lowest. Each command does one single thing and the pipe connects them: that idea is the foundation of the Linux toolbox.
Review questions
What does grep -c Failed auth.log do?
Answer: Counts how many lines contain Failed. The -c option makes grep count the matching lines instead of showing them.
In a chain of commands, what does the | symbol do?
Answer: Passes the output of one command as the input of the next one. The pipe connects commands: what one produces, the next one receives.
An IP shows up 300 times with "wrong password" in five minutes. What is most likely going on?
Answer: A bot trying passwords (brute force). A person does not try 300 passwords in five minutes. It is an automated attack.