Authentication Log Analysis: Reconstructing Logins
- Locate auth events on any host — auth.log, secure, or the journal — and know which one your distro actually writes to
- Read sshd log lines fluently: tell a failed password, an invalid user, and a successful Accepted login apart at a glance
- Use last, lastb, and lastlog to read the binary login records (who got in, who failed, when each user last connected)
- Run the failures-per-source-IP workflow that exposes a brute force, then find the first Accepted from that IP — the moment of compromise
Why reconstruct logins at all
Frame auth-log analysis as answering who got in, when, and from where — on a host you are authorized to investigate.
You are handed a host and one question: did someone log in who shouldn't have? Before an attacker can run a process, open a port, or plant a backdoor, they almost always have to authenticate first — and authentication is one of the most reliably logged events on a Linux system. Every SSH attempt, every
sudo, every console login leaves a line behind.This lesson follows one incident from start to finish: an SSH brute force from
203.0.113.66that hammered the host with failed passwords and then, eventually, succeeded. Your job is to reconstruct that story purely from the auth records and pin down the exact moment of compromise.Think of it like a building's badge-entry log. A door reader records every swipe — accepted or denied. A legitimate employee shows up as a single clean accepted swipe each morning. But a thief trying stolen badges leaves a very different trace: a long run of denied swipes at one door, late at night, then suddenly one accepted swipe. That one accepted swipe — right after the wall of denials — is the break-in moment. Auth logs are that badge reader, and
203.0.113.66is the person at the door.Where auth events live
Find the auth log on any distro — auth.log, secure, or the journal — instead of guessing.
The single most common way an auth investigation stalls is looking in the wrong file, finding nothing, and wrongly concluding the box is clean. Where sshd,
sudo, and PAM write their authentication events depends on the distribution:- Debian / Ubuntu →
/var/log/auth.log - RHEL / CentOS / Fedora / Rocky →
/var/log/secure - systemd hosts → the same events also live in the journal, queryable by unit (
sshorsshd) or by command name (_COMM=sshd)
On most modern systems both exist: rsyslog writes the flat file and journald keeps a binary copy. Knowing all three means you are never stuck on an unfamiliar box.
ls -l /var/log/auth.log /var/log/secure 2>/dev/null-rw-r----- 1 root adm 184204 Jun 23 02:13 /var/log/auth.log
Only auth.logexists here, so this is a Debian/Ubuntu host — the auth events are in/var/log/auth.log. On a RHEL box you would see/var/log/secureinstead andauth.logwould be missing. Running this first, before anything else, tells you which file to grep. The2>/dev/nullhides the 'No such file' error for whichever one isn't present.sudo journalctl -u ssh -u sshd --no-pager | tail -n 4Jun 23 02:09:51 web01 sshd[4418]: Failed password for root from 203.0.113.66 port 51110 ssh2 Jun 23 02:11:03 web01 sshd[4502]: Accepted password for webadmin from 203.0.113.66 port 51990 ssh2 Jun 23 02:11:03 web01 sshd[4502]: pam_unix(sshd:session): session opened for user webadmin by (uid=0) Jun 23 02:14:22 web01 sudo[4530]: webadmin : TTY=pts/0 ; PWD=/home/webadmin ; USER=root ; COMMAND=/usr/bin/apt install curl
- -u ssh -u sshd
- Filter to the SSH service unit. Distros name it differently —
sshon Debian/Ubuntu,sshdon RHEL — so asking for both covers either without you having to know which. - --no-pager
- Dump straight to stdout so you can pipe to tail/grep instead of landing inside less.
The journal holds the same auth events as the flat file, structured and timestamped. Note you can query the same data by command name with journalctl _COMM=sshdwhen you don't know the unit name. This is your fallback when the flat file has been rotated away — or tampered with (next steps).Check yourself
You're triaging a RHEL host and
cat /var/log/auth.logsays 'No such file or directory'. Which file holds the sshd and sudo auth events on this distro?- Debian / Ubuntu →
Reading sshd log lines
Tell failed, invalid-user, and Accepted lines apart, and read the connection-closed and PAM session lines around them.
sshd narrates every connection in a small, predictable vocabulary. Learn the handful of phrases and you can read an attack like a sentence. Here are the lines that matter, from a single sequence in our incident:
sudo grep sshd /var/log/auth.log | grep '203.0.113.66' | headJun 23 02:08:31 web01 sshd[4390]: Failed password for invalid user admin from 203.0.113.66 port 50902 ssh2 Jun 23 02:08:31 web01 sshd[4390]: Connection closed by invalid user admin 203.0.113.66 port 50902 [preauth] Jun 23 02:08:55 web01 sshd[4402]: Invalid user oracle from 203.0.113.66 port 50988 Jun 23 02:09:51 web01 sshd[4418]: Failed password for root from 203.0.113.66 port 51110 ssh2 Jun 23 02:11:03 web01 sshd[4502]: Accepted password for webadmin from 203.0.113.66 port 51990 ssh2
Five distinct phrases, each meaning something specific: - `Failed password for invalid user admin` — wrong password AND adminis not even a real account on this host. Theinvalid usertag means the username doesn't exist in/etc/passwd— a dead giveaway of guessing. - `Connection closed ... [preauth]` — the client gave up (or was dropped) before authenticating.[preauth]means it happened before login completed; a flood of these is automated guessing, not a human. - `Invalid user oracle` — sshd announcing the target account doesn't exist, separate from the password check. - `Failed password for root` — wrong password for a real account (rootexists). Noinvalid usertag here, because root is real — but direct root SSH should usually be disabled anyway. - `Accepted password for webadmin` — the password worked. This is the one that matters.webadminis a real account and the credential succeeded. From203.0.113.66, right after the failures, at 02:11 — this is the compromise.Two more line types complete the picture — what happens after a successful login. PAM (the Pluggable Authentication Modules layer) logs the session lifecycle, and any privilege escalation shows up as
sudoorsulines:sudo grep -E 'session opened|session closed|sudo|su\[' /var/log/auth.log | headJun 23 02:11:03 web01 sshd[4502]: pam_unix(sshd:session): session opened for user webadmin by (uid=0) Jun 23 02:14:22 web01 sudo[4530]: webadmin : TTY=pts/0 ; PWD=/home/webadmin ; USER=root ; COMMAND=/usr/bin/apt install curl Jun 23 02:14:22 web01 sudo[4530]: pam_unix(sudo:session): session opened for user root by webadmin(uid=1000) Jun 23 02:18:40 web01 su[4602]: Successful su for root by webadmin Jun 23 02:19:55 web01 sshd[4502]: pam_unix(sshd:session): session closed for user webadmin
- -E
- Extended regex so the
|alternation works without backslashes — match any of the four session/escalation patterns in one pass.
This is what the attacker did once in: - `session opened for user webadmin by (uid=0)` — PAM confirms the interactive session started. The matching `session closed` at 02:19 brackets the session: webadminwas on the box from 02:11 to 02:19. - `sudo ... USER=root ; COMMAND=/usr/bin/apt install curl` —webadminescalated to root and ran a command.sudologs the exact command, so you can see they pulled incurl(often a precursor to downloading a second-stage payload). - `Successful su for root by webadmin` — a second escalation, this time viasu. Bothsudoandsulines are where a foothold turns into root. The session-open/close pair plus these escalations are the bridge from who logged in to what they did — the pivot you'll follow next.Check yourself
Which sshd phrase marks the actual successful login — the moment a credential worked — as opposed to a failed or rejected attempt?
The binary login records: last, lastb, lastlog
Read the wtmp/btmp/lastlog binary databases to see successful logins, failed logins, and per-user last login.
Alongside the text logs, the kernel and login programs keep three binary databases you read with dedicated tools — not with
catorgrep, because they're not text files. Each answers a different question:- `last` reads `/var/log/wtmp` → a history of successful logins, logouts, and reboots
- `lastb` reads `/var/log/btmp` → a history of failed login attempts (root-only)
- `lastlog` reads `/var/log/lastlog` → the single most recent login time for every account
The trap to internalize now:
lastandlastbread different files. Look in the wrong one and you'll see the wrong outcome.last -a | headwebadmin pts/0 Tue Jun 23 02:11 still logged in 203.0.113.66 webadmin pts/0 Mon Jun 22 09:02 - 17:40 (08:38) 198.51.100.42 reboot system boot Mon Jun 22 08:55 still running 6.1.0-21-amd64 webadmin pts/0 Sun Jun 21 09:14 - 18:01 (08:47) 198.51.100.42
- -a
- Put the remote hostname/IP in the LAST column so long addresses aren't truncated — critical when the source IP is the thing you care about.
lastreadswtmp, the successful-login record. Read it for what breaks the pattern.webadminnormally logs in around 09:00 from the internal198.51.100.42. The top line is a login at 02:11 from the external203.0.113.66— odd hour, unfamiliar source, still connected. That external login at 02:11 is exactly theAccepted passwordline from the previous step. Therebootline shows the box last booted Monday 08:55.sudo lastb 2>/dev/null | head || echo "no btmp"oracle ssh:notty 203.0.113.66 Tue Jun 23 02:09 - 02:09 (00:00) root ssh:notty 203.0.113.66 Tue Jun 23 02:09 - 02:09 (00:00) admin ssh:notty 203.0.113.66 Tue Jun 23 02:08 - 02:08 (00:00) admin ssh:notty 203.0.113.66 Tue Jun 23 02:08 - 02:08 (00:00)
- 2>/dev/null
- btmp may not exist on a freshly built host; suppress the error so the
|| echofallback can report that cleanly instead of a confusing stderr message.
lastbreadsbtmp, the failed-login record — the mirror image oflast, and root-only. A burst of failures across many usernames (oracle,root,admin) from the same203.0.113.66in the same minute is the unmistakable shape of a brute force or password spray. It's the same IP that succeeded aswebadminat 02:11 — so the binary records alone already tell you: spray at 02:08–02:09, success at 02:11. An emptylastbon a clean box is itself a useful (negative) result.lastlog | grep -vi 'never logged in'Username Port From Latest root pts/0 198.51.100.42 Mon Jun 22 21:30:14 +0000 2026 webadmin pts/0 203.0.113.66 Tue Jun 23 02:11:02 +0000 2026
lastlogreads/var/log/lastlogand prints the most recent login for every account in one table. Filtering out the 'Never logged in' system accounts leaves the accounts that actually log in.webadmin's latest login is from203.0.113.66at 02:11 — confirming the compromised account at a glance, and flagging that its newest login came from the attacker IP rather than the usual internal address.Check yourself
Which command shows you FAILED login attempts — the signature of a brute force — by reading /var/log/btmp?
The workflow: count failures, find the breach
Aggregate failures by source IP to expose the brute force, then find the first Accepted from that IP — the compromise moment.
Now you put it together into the workflow that turns thousands of log lines into one conclusion. A real attack buries its one successful login under a mountain of failures — you can't read them line by line. So you aggregate: count failed passwords per source IP. The IP with hundreds of failures is the brute force; everything else is noise.
This is the badge-log method from step 1, made concrete: tally the denied swipes by who was swiping, find the one address responsible for the wall of denials, then look for that same address's first accepted swipe.
sudo grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head412 203.0.113.66 6 198.51.100.42 2 198.51.100.51- grep "Failed password"
- Keep only the failed-authentication lines — the brute-force fuel.
- awk '{print $(NF-3)}'
- Print the 4th-from-last field. In '...from <IP> port <N> ssh2' the source IP sits 3 fields before the end, so NF-3 extracts the IP regardless of how long the username is.
- sort | uniq -c
- uniq -c needs its input sorted; together they collapse identical IPs into one line prefixed with a count.
- sort -rn
- Sort that count numerically (-n), highest first (-r), so the worst offender floats to the top.
The verdict in three lines. 203.0.113.66is responsible for 412 failed passwords — orders of magnitude more than the handful of fat-fingered failures from the legitimate internal hosts (198.51.100.x, 2–6 each). That lopsided count is the brute force. The two internal IPs with a few failures each are normal human typos, not an attack. You've found the attacker's address without reading a single line individually.You have the attacker IP. Now answer the question that actually matters: did they ever get in, and when? Search that exact IP for the first
Acceptedline. A brute force with zeroAcceptedlines failed; one with anAcceptedline succeeded, and that line's timestamp is the moment of compromise.sudo grep "Accepted" /var/log/auth.log | grep "203.0.113.66" | head -n 1Jun 23 02:11:03 web01 sshd[4502]: Accepted password for webadmin from 203.0.113.66 port 51990 ssh2
This single line is the breach. The brute-force IP 203.0.113.66finally guessedwebadmin's password at 02:11:03. Everything matters here: it'spasswordauth (not a key, so a guessable credential), the account is the realwebadmin, the IP is the one with 412 failures, and the time sits immediately after the 02:08–02:09 spray.head -n 1gives you the first success — the entry point. From here you pivot: grep the samesshd[4502]PID and thewebadminsession to follow exactly what they did after getting in (thesudo/sulines from step 3).Check yourself
After your aggregation pipeline names 203.0.113.66 as the brute-force source, what single phrase do you grep for (filtered to that IP) to find out whether — and exactly when — they got in?
Try it in your live terminal
Run the auth-log reconstruction yourself on the real sandbox to your right.
You have a real root shell. Run these read-only reads in order — the same sequence you'd use on a 'did someone log in?' call. There may be nothing malicious on this box; the point is to make the moves reflexive. Each command is safe and read-only: you are reading existing records, not changing anything. Note anything that breaks the pattern — odd hours, external IPs, a lopsided failure count — as you go.
last -a | headSuccessful logins from wtmp, source IP in the last column. Look for unfamiliar source addresses or logins at odd hours. This is your who-got-in baseline. sudo lastb 2>/dev/null | head || echo "no btmp"Failed logins from btmp. A burst from one IP across many usernames is a brute force. An empty result (or 'no btmp') is a clean, reassuring sign — not an error. sudo journalctl -q -u ssh -u sshd --no-pager 2>/dev/null | tail | grep . || sudo grep -h sshd /var/log/auth.log /var/log/secure 2>/dev/null | tailThe detailed sshd lines, journal-first with a flat-file fallback. Grepping both auth.log and secure means it works on Debian and RHEL alike. Read the tail for recent Accepted/Failed/Invalid user lines. Two details make the fallback actually fire on a host with no journal: -qsuppresses journalctl's-- No entries --banner, and the trailing| grep .turns 'produced no output' into a non-zero exit the||can see. Without them the pipeline ends intail, which succeeds on empty input, so||would never run and you'd stare at a blank result believing the box was clean.sudo grep -h "Failed password" /var/log/auth.log /var/log/secure 2>/dev/null | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | headThe failures-per-IP aggregation against whichever auth file exists. The top line by count is your brute-force candidate. Then grep Acceptedfor that IP to find the first successful login — the compromise moment.
- Auth events live in
/var/log/auth.log(Debian) or/var/log/secure(RHEL), and in the journal (journalctl -u ssh -u sshd/_COMM=sshd) — confirm which sink the host uses before trusting an empty grep. - sshd speaks a small vocabulary:
Failed password,Invalid user,Connection closed [preauth]are misses;Accepted password/publickeyis the hit. The presence of 'invalid user' tells you whether the target account even exists. last(wtmp) shows successful logins,lastb(btmp) shows failed ones,lastlogshows each account's most-recent login — read the right database for the outcome you want, and cross-check the journal against tampered flat files.- The workflow names the attack and times the breach: count
Failed passwordby source IP to expose the brute force, then find the firstAcceptedfrom that IP — that timestamp is the moment of compromise and the anchor for everything after. - Judge the pattern, not the line: hundreds of failures across many usernames from one IP is an attack; a single failed password is a typo.
| Artifact / command | Source file | What it proves |
|---|---|---|
grep sshd auth.log / journalctl _COMM=sshd | /var/log/auth.log, /var/log/secure, journal | Per-attempt sshd narrative: failed, invalid-user, and Accepted logins |
last -a | /var/log/wtmp | Successful logins, logouts, and reboots — who got in, from where, when |
lastb | /var/log/btmp | Failed login attempts — the brute-force / password-spray fingerprint (root-only) |
lastlog | /var/log/lastlog | The single most-recent login per account — spot a compromised account fast |
sudo / su lines | /var/log/auth.log / secure | Privilege escalation after login — the exact command run as root |
Failures-by-IP pipeline → first Accepted | auth.log / secure / journal | Names the brute-force source IP and timestamps the moment of compromise |
