Operating Running Systems: Boot Targets, Processes, Logs, Tuning

Old SysV init had runlevels — a single number (0-6) that described the machine's mode: 3 was multi-user text, 5 was multi-user with a graphical login, 0 was halt, 6 was reboot. systemd dropped the magic numbers in favour of named targets.

A target is just a special unit (name.target) that groups other units. Think of a target as a stage manager's cue sheet: calling a cue doesn't itself do the work — it tells everyone in that scene to take their places. graphical.target cues up everything in multi-user.target plus the display manager.

The two you must know cold:

• `multi-user.target` — full text/networked system, no GUI. The old runlevel 3.
• `graphical.target` — everything in multi-user plus a graphical login. The old runlevel 5.

There are two different questions, exactly like the active/enabled split from the systemctl lesson:

• What target should the machine boot into next time? → set-default
• What target should the machine switch to right now, without rebooting? → isolate

set-default rewrites the /etc/systemd/system/default.target symlink. isolate stops everything not part of the named target and starts everything that is — switching the running system live.

No root password, no record of it, and you're locked out. Because you have physical (or console) access to the boot loader, you can interrupt boot and get a root shell before the system finishes coming up. The trick is rd.break, which drops you to a shell in the initramfs — the tiny early-boot environment — while the real root filesystem is still mounted read-only under /sysroot.

The procedure has a sting in the tail. On a SELinux-enforcing system (the RHCSA default), passwd writes to /etc/shadow, and a file written from the initramfs gets the wrong SELinux label. If you skip the relabel, the system boots but sshd/login can't read shadow correctly and you're locked out again. The fix is one touch — and it's the single most-forgotten step on the exam.

Step 1 — interrupt the boot loader. At the GRUB menu, press e to edit the default entry, find the line that starts with linux (the kernel line), and append rd.break to its end:

Step 2 — remount the real root read-write. You land at a switch_root:/# prompt. The real filesystem is mounted read-only at /sysroot, so you must remount it writable before you can change anything:

Step 3 — chroot into the real system, then set the password. chroot /sysroot makes /sysroot become / for the shell, so passwd edits the real /etc/shadow:

Step 4 — the step everyone forgets: schedule an SELinux relabel. Then exit out and let the boot continue:

systemd collects logs centrally in the journal, and you read it with journalctl. Instead of grepping a dozen text files under /var/log, you ask the journal structured questions: which boot? how severe? which service? since when? Think of it as a flight recorder with a query language — every event is timestamped and tagged, so you filter instead of scroll.

The four filters that cover almost every troubleshooting task:

• -b → just the current boot (or a past boot)
• -p → by priority (severity)
• -u → by unit (one service)
• --since / --until → by time window

By default the journal lives in /run/log/journal, which is memory-backed and wiped on reboot — so journalctl -b -1 (last boot) shows nothing. To keep logs across reboots, make the journal persistent by creating its on-disk directory:

"The box is slow" is a question about which process. ps aux prints every process with the columns that answer it: `%CPU` (share of a CPU it has used), `%MEM` and `RSS` (resident memory, in KiB, actually held in RAM), and the `PID` you will act on. Sort by the column you care about and the culprit floats to the top. Here is a snapshot from a loaded database server:

top is the live version: it refreshes every few seconds, and inside it `P` sorts by CPU and `M` by memory (q quits). top -bn1 prints one batch snapshot you can pipe or save.

Once you have the PID, the action is a signal, and choosing the signal is the skill. kill -l lists them all, and man 7 signal explains what each one means and whether a process can catch it. Three matter daily:

• `SIGTERM` (15), the default for kill PID: please exit. The process can clean up first.
• `SIGKILL` (9): the kernel removes the process at once. It can't be caught or cleaned up after, so it's the last resort for a process that ignored TERM.
• `SIGHUP` (1): for a daemon, the convention is re-read your configuration. The process keeps running with the new settings.

pgrep -af NAME finds PIDs by name, and pkill NAME signals by name.

Two levers control performance. One is per-process: the nice value, a politeness score from -20 (greedy, highest priority) to +19 (very polite, lowest). Higher niceness = nicer to others = less CPU for itself. Think of it as boarding-group numbers: a low number boards first and grabs the overhead bin; a high number waits.

The other is system-wide: tuned ships ready-made profiles (throughput-performance, powersave, virtual-guest, balanced...) that twist dozens of kernel and hardware knobs at once for a given workload — so you pick a profile instead of hand-tuning sysctls.

You have a real root shell. These are read-only — they inspect state without changing the boot target or resetting any password, so they're safe to run as-is. (The password-reset procedure deliberately breaks a box if done wrong, so practise that one in a throwaway VM, not here.) Run them in order and read what each tells you.

A note on journalctl here: this lab host doesn't run systemd's journald as PID 1, so journalctl may print nothing (and --list-boots shows a single volatile entry). Run it to learn its shape — an empty journal means this host logs to flat files under `/var/log`, which is where you'd actually read on a box like this. The systemctl reads work as shown.