LPIC-2 Capacity Planning: Measure, Troubleshoot, and Predict Resource Usage

Advanced30 min7 steps

What you'll learn

Read the live performance toolkit — vmstat, iostat, mpstat, free, top — and interpret the fields that actually matter
Tell a CPU-bound box from a disk-bound or memory-starved one by reading wa, si/so, and the run queue
Avoid the classic interval-sampling trap that makes the first sample lie to you
Attribute a resource spike to a single process with iotop and pidstat
Turn on sysstat so sar archives history, then project a growth trend to plan capacity before you run out

Why capacity planning is a discipline, not a guess
Frame the four jobs — measure now, find the bottleneck, blame the process, predict the future.
When a server gets slow, the panicked instinct is to throw hardware at it. Capacity planning is the opposite of panic: you measure what the machine is actually doing, identify which of its four resources — CPU, memory, disk I/O, network — is the real bottleneck, attribute the load to a specific process, and then project the trend so you add capacity before the wall, not after the outage.
Think of it like a doctor reading vitals instead of guessing at symptoms. A patient who is short of breath might have a lung problem or a heart problem — the treatment is completely different, and you only know which by reading the right gauge. A Linux box that "feels slow" might be CPU-saturated, swapping itself to death, or stuck waiting on a slow disk. Each looks like "slow" from the outside and each has a different fix.
This lesson gives you the gauges (vmstat, iostat, mpstat, free, top), teaches you which numbers to trust, and shows you the one sampling trap that fools almost everyone the first time.
Why LPIC-2 weights this so heavily
Objective 200 ("Capacity Planning") is split into measuring resource usage and predicting future needs. The exam isn't testing whether you can recite tool names — it's testing whether you can look at a vmstat line and say "that's a disk bottleneck, not a CPU one." Misreading the fields is the failure mode this lesson is built to prevent.
The live dashboard: vmstat
Read vmstat's run queue, swap, and wait columns as a single system-wide vital-signs panel.
vmstat (virtual-memory statistics) is the closest thing Linux has to a single vital-signs monitor: in one line it reports CPU, memory, swap, and I/O. You read it left to right like a heart monitor — most columns are noise until one of three spikes.
The trick is knowing which three columns separate the diagnoses:
- `r` (under procs) — processes runnable, i.e. wanting CPU right now. Compare this to your core count.
- `si`/`so` (under swap) — kilobytes swapped in/out per second. Anything sustained here means you're out of RAM.
- `wa` (under cpu) — percent of CPU time idle while waiting on I/O. High wa means the disk, not the processor, is the holdup.
```
$ vmstat 1 3
```
```
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 5  0  20480  98244  41200 612880    0    0    12    30  220  410 12  4 82  2  0
 6  1  20480  96120  41200 613004  256  512   180  4096  990 1820 18  9  8 65  0
 7  2  20480  95880  41200 613112  300  640   210  4200 1010 1900 20 10  6 64  0
```
1
Interval: refresh every 1 second.
3
Count: take 3 samples total, then exit (instead of running forever).
On a 4-core box, `r` climbing to 5, 6, 7 means more processes want CPU than there are cores — the run queue is backing up (CPU saturation). At the same time `wa` is 64–65%: two-thirds of CPU time is spent idle, waiting on the disk, and `so` (swap-out) is 512–640 KB/s: the box is also shoving memory to swap. This single screen is showing CPU pressure, a disk bottleneck, AND memory exhaustion at once — a textbook "everything is on fire" line.
Run queue vs. core count is the rule
A run-queue value r that is consistently above your number of CPU cores means tasks are queuing for a processor that's already full — that's CPU saturation. r of 8 on a 16-core box is healthy; r of 8 on a 4-core box is a problem. Always read r relative to nproc, never as an absolute number.
Check yourself
On a 4-core server, vmstat shows r steady at 9 and wa near 0. Is the bottleneck CPU or disk?
The classic gotcha: the first sample lies
Internalize that sample #1 from iostat/sar/mpstat is a since-boot average, and learn to discard it.
This is the single most common mistake on the whole topic, and it is worth a step of its own.
Tools like iostat, mpstat, and sar work by reading the kernel's cumulative counters — totals that have been adding up since the machine booted. To turn cumulative totals into a rate (per-second activity), the tool subtracts two readings taken a moment apart.
But the very first sample has nothing earlier to subtract from, so it falls back to dividing the total-since-boot by the seconds-since-boot. The first sample is therefore the average since boot — not what's happening right now. On a box that's been up for 40 days and idle all weekend, sample #1 will look calm even during a live emergency.
Trusting the first iostat / sar / mpstat sample
If you run iostat -x with no interval, you get one line — and that line is the lifetime average, which is almost useless for diagnosing a current spike. The fix is simple: always pass an interval and a count, and ignore the first sample. iostat -x 1 4 gives four samples; you read samples 2, 3, and 4. The first is throwaway.
Example — not runnable on your lab box
```
$ iostat -x 2 3
```
```
Linux 5.15.0 (db01)   06/23/2026   _x86_64_   (4 CPU)

Device  r/s   w/s   rkB/s    wkB/s  await  aqu-sz  %util   <-- SAMPLE 1: since-boot avg, IGNORE
sda     2.10  8.40   84.0    420.0   1.20    0.02   3.10

Device  r/s   w/s   rkB/s    wkB/s  await  aqu-sz  %util   <-- SAMPLE 2: the real "now"
sda     6.00 210.00  240.0  61440.0  48.50    9.80  99.20

Device  r/s   w/s   rkB/s    wkB/s  await  aqu-sz  %util
sda    5.50 205.00  220.0  60100.0  47.10    9.40  98.70
```
-x
Extended statistics — adds await (latency) and %util (how busy the device is), the columns that actually diagnose I/O.
2
Interval: produce a fresh sample every 2 seconds.
3
Count: stop after 3 samples.
Watch the deception. Sample 1 shows %util 3.1% and await 1.2ms — the disk looks nearly idle, because that's the lifetime average. Samples 2 and 3 are the truth: %util is pinned at ~99% (the disk is saturated, no spare capacity) and await (average request latency) has jumped to ~48ms. If you had read only sample 1 you'd have declared the disk healthy during an active I/O meltdown.
Check yourself
Why must you discard the first sample from iostat, sar, and mpstat when interval-sampling?
Is it really CPU? mpstat, free, and top
Separate a true CPU bottleneck from disk-wait and from memory pressure using per-core and memory views.
vmstat gives the whole-system summary; mpstat, free, and top let you confirm which resource is actually the limit before you spend money on the wrong one.
mpstat -P ALL breaks CPU usage out per core, which matters because a single-threaded process can peg one core to 100% while the system-wide average looks like a comfortable 25% on a 4-core box. The average lies; the per-core view tells the truth.
Example — not runnable on your lab box
```
$ mpstat -P ALL 1 2
```
```
Linux 5.15.0 (app02)   06/23/2026   _x86_64_   (4 CPU)

(first sample omitted — since-boot average)

08:14:22  CPU   %usr  %sys  %iowait  %idle
08:14:23  all   24.0   3.0     38.0   35.0
08:14:23    0   96.0   3.0      1.0    0.0
08:14:23    1    0.0   1.0     78.0   21.0
08:14:23    2    0.0   1.0     76.0   23.0
08:14:23    3    0.0   1.0      0.0   99.0
```
-P ALL
Report every CPU core separately, plus an 'all' aggregate line.
1
Interval: one sample per second.
2
Count: two samples (the first is the throwaway since-boot average).
The all line averages to a sleepy-looking 24% usr / 35% idle, but that average is hiding the story. CPU 0 is at 96% `%usr` — one core pinned by a single-threaded job. Meanwhile CPUs 1 and 2 show ~77% `%iowait` — they're not busy, they're blocked on the disk. The aggregate would have led you to add cores; the per-core view says you have one CPU-bound task and a disk bottleneck instead.
For memory, free -h answers the question everyone gets wrong: *how much RAM is really free?* The available column — not free — is the number that matters, because Linux deliberately uses spare RAM as disk cache and will hand it back the instant a program needs it.
```
$ free -h
```
```
               total        used        free      shared  buff/cache   available
Mem:            7.7Gi       5.9Gi       210Mi        64Mi       1.6Gi       1.4Gi
Swap:           2.0Gi       1.1Gi       0.9Gi
```
-h
Human-readable units (Gi/Mi) instead of raw kibibytes.
The scary-looking `free` of 210Mi is a red herring — that's RAM not yet touched. The real headroom is `available` 1.4Gi, which includes the 1.6Gi of reclaimable buff/cache. The genuine warning sign here is the Swap line: 1.1Gi used — the box has already pushed over a gig of memory to disk, which lines up with the si/so activity vmstat flagged. Memory is tight.
Reading high `wa` (or %iowait) as a CPU problem
High wa/%iowait does not mean the CPU is overworked — it means the CPU is idle, twiddling its thumbs waiting for the disk to return data. Adding CPUs to a high-wa box buys you nothing; the cores are already mostly idle. The fix lives on the storage side: faster disks, more IOPS, less I/O. Confusing I/O-wait for CPU load is the costliest misdiagnosis in capacity planning.
```
$ top -bn1 | head -12
```
```
top - 08:15:02 up 41 days,  3:22,  2 users,  load average: 7.81, 6.44, 5.90
Tasks: 212 total,   3 running, 209 sleeping
%Cpu(s): 24.0 us,  3.0 sy,  0.0 ni, 35.0 id, 38.0 wa,  0.0 hi,  0.0 si
MiB Mem :   7860.0 total,    210.0 free,   6040.0 used,   1610.0 buff/cache
MiB Swap:   2048.0 total,    920.0 free,   1128.0 used.   1432.0 avail Mem

    PID USER      PR  NI    VIRT    RES   %CPU  %MEM     TIME+ COMMAND
   3120 postgres  20   0  890.4m 512.0m   98.0   6.5   42:11.7 postgres
   3470 app       20   0  1.2g   240.0m    4.0   3.0   12:40.2 ruby
    980 root      20   0   58.0m  17.2m    0.0   0.2    0:05.1 nginx
```
-b
Batch mode: print plain text and exit instead of running an interactive screen — ideal for piping/logging.
-n1
Take exactly one sample, then quit.
The header confirms everything in one place: load average 7.81 (high for 4 cores), `38.0 wa` (heavy I/O wait), and Swap 1128 used (memory pressure). The process table then names a suspect: PID 3120 `postgres` at 98% CPU. top is the bridge from "the system is stressed" to "this process is doing it" — which is exactly where the next step picks up.
Check yourself
mpstat -P ALL shows the all line at 30% idle, but CPU 0 sits at 99% %usr. What kind of workload does that pattern indicate?
Blame the process: iotop and pidstat
Attribute a measured spike to one PID — especially for disk I/O, which top hides.
Knowing the disk is saturated is only half the answer; you need to know which process is hammering it. top shows CPU and memory per process, but it does not show per-process disk I/O. Two tools fill that gap.
iotop is top for disk — a live, sorted view of which processes are reading and writing right now. pidstat is the scriptable, non-interactive counterpart: it samples per-process resource use (CPU, memory, and I/O with -d) on an interval, which makes it perfect for logging or for grabbing one clean reading.
Example — not runnable on your lab box
```
$ sudo iotop -bo -n2
```
```
Total DISK READ:   1.20 M/s | Total DISK WRITE:  58.40 M/s
Current DISK READ: 1.18 M/s | Current DISK WRITE: 57.90 M/s
  TID  PRIO  USER     DISK READ  DISK WRITE   IO>    COMMAND
 3120  be/4  postgres   0.90 M/s   56.20 M/s  97.8 % postgres: autovacuum
 3470  be/4  app        0.20 M/s    1.10 M/s   3.1 % ruby app/worker.rb
  980  be/4  root       0.00 B/s    0.00 B/s   0.0 % nginx: worker
```
-b
Batch mode — print lines instead of a full-screen UI, so it can be logged or piped.
-o
Only show processes actually doing I/O, hiding the idle majority.
-n2
Take 2 samples then exit (the first establishes a baseline, like the other tools).
The IO> column is the percent of time each process spent blocked waiting on I/O, and PID 3120 (postgres autovacuum) is at 97.8%, writing 56 MB/s. That single process is the source of the disk saturation iostat measured. You've now connected the symptom (%util 99%) to a cause (autovacuum) you can actually act on — reschedule it, tune it, or move it to off-peak hours.
Example — not runnable on your lab box
```
$ pidstat -d 1 3
```
```
Linux 5.15.0 (db01)   06/23/2026   _x86_64_   (4 CPU)

(first interval omitted)

08:16:40   UID    PID    kB_rd/s    kB_wr/s  iodelay  Command
08:16:41   111   3120     920.00   57548.00      812  postgres
08:16:41  1000   3470     205.00    1124.00       14  ruby
08:16:42   111   3120     880.00   56100.00      790  postgres
```
-d
Report per-process DISK I/O (read/write KB/s and iodelay) — the data top can't show.
1
Interval: sample every second.
3
Count: three samples then exit.
pidstat -d confirms iotop in numbers you can log: PID 3120 is writing ~57 MB/s (kB_wr/s 57548) with a high `iodelay` (clock ticks the process was blocked on I/O). Because pidstat prints plain timestamped lines, you can pipe it to a file and build a per-process history — the bridge from 'which process now' to 'is this getting worse over time', which the final step tackles.
Reach for pidstat when iotop isn't installed
iotop needs root and isn't on every minimal box; pidstat ships with the sysstat package you'll install in the next step anyway. Knowing both means you're never stuck: iotop for an eyeball at the console, pidstat -d for anything you need to script, log, or capture as evidence.
Check yourself
Which tool gives you per-process disk I/O that top cannot show?
Predict the future: sysstat, sar, and trending
Enable historical collection and project a growth rate to a threshold so you add capacity in advance.
Everything so far measures the present. The 'planning' half of capacity planning is about the future: a disk filling at 2% a week, RAM use creeping up after each deploy, a load average trending upward month over month. You can't see a trend in a single snapshot — you need history.
The sysstat package provides that history. Once enabled, a cron job runs sar (System Activity Reporter) on a schedule and archives binary snapshots to `/var/log/sa/`. Days or weeks later you replay them to see the curve. Think of it as a flight recorder that's always running: when something breaks at 3 a.m., you don't need to have been watching — the data is already on disk.
```
$ sudo systemctl enable --now sysstat
```
```
Created symlink /etc/systemd/system/multi-user.target.wants/sysstat.service → /usr/lib/systemd/system/sysstat.service.
```
enable
Start the periodic collector at every boot (creates the symlink shown).
--now
Also start collection immediately, without waiting for a reboot.
This turns on the scheduled collector (often sa1, driven by cron or a systemd timer) that writes a fresh data file to /var/log/sa/saDD — where DD is the day of the month — every few minutes. From this point on you're accumulating the history you'll trend against. On some distros you must also set ENABLED="true" in /etc/default/sysstat first.
Example — not runnable on your lab box
```
$ sar -r -f /var/log/sa/sa15 | head
```
```
Linux 5.15.0 (db01)   06/15/2026   _x86_64_   (4 CPU)

00:00:01  kbmemfree  kbavail  %memused  kbcached  kbcommit  %commit
00:10:01    2104320  2240100     59.20    1601200   4980200    61.30
02:00:01    1810400  1944200     63.10    1598000   5310400    65.40
08:00:01    1402100  1530880     71.60    1592400   5990100    73.80
14:00:01     996300  1120440     78.90    1588900   6620300    81.50
20:00:01     612800   720110     85.40    1581200   7240800    89.10
```
-r
Report memory utilization statistics (%memused, committed memory).
-f /var/log/sa/sa15
Read from an archived file (the 15th of the month) instead of live data — this is how you look at the past.
Reading the archive back, `%memused` climbs 59% → 85% across a single day, and it does not drop overnight — the curve only goes up. That's a memory leak or steady growth, not a daily peak that recovers. A single live snapshot at noon (71%) would have looked fine; only the trend across the archived day reveals the box is on track to exhaust memory. sar -b does the same for I/O, sar -u for CPU.
Trending turns a metric into a deadline
If %memused rises a steady ~3.3 points per hour and your action threshold is 95%, you don't just know the box is busy — you can compute roughly when it hits the wall and schedule the RAM upgrade or the leak fix before the outage. That projection — 'at this growth rate we cross the threshold in N days' — is the entire deliverable of capacity planning, as opposed to capacity monitoring.
Check yourself
Which directory does sysstat archive its sar snapshots to for historical trending?
Try it in your live terminal
Run the read-only measurement sweep yourself on the real sandbox to your right.
You have a real root shell. These commands are all read-only — they observe, they change nothing. Run them in order and read the fields the way this lesson taught: run queue vs. cores, wa for disk, swap for memory, and remember to ignore the first interval sample where it applies.
```
$ vmstat 1 3
```
Your live vital-signs panel. Watch r against your core count (nproc), and glance at si/so and wa. Three samples, one second apart.
```
$ free -h
```
Read the available column, not free, for real headroom — and check whether Swap used is climbing.
```
$ iostat -x 1 2 2>/dev/null || true
```
Per-disk extended stats. Two samples — ignore the first (since-boot average) and read %util and await on the second. The || true keeps the lesson moving if sysstat isn't installed here.
```
$ top -bn1 | head
```
One-shot snapshot: read the load average and wa in the header, then scan the process table for the top CPU and memory consumers.

Recap

Diagnose before you spend: r above core count = CPU saturation, si/so = swapping (out of RAM), high wa = disk-bound. Each 'slow' has a different fix.
The first sample from iostat/sar/mpstat is the since-boot average — always use an interval+count and throw sample 1 away.
High wa/%iowait is a disk problem, not a CPU one; the cores are idle waiting on storage. Adding CPUs won't help.
iotop and pidstat -d attribute disk load to a specific PID — the per-process I/O that top can't show.
Enable sysstat so sar archives to /var/log/sa/, then project the growth rate to your threshold to add capacity before the wall, not after the outage.

Symptom → tool → reading
Symptom / question	Tool	Field to read	Watch for
Is the CPU saturated?	`vmstat 1 3`	`r` (run queue)	`r` sustained above core count
Is the box out of RAM?	`vmstat` / `free -h`	`si`/`so`, Swap `used`, `available`	Sustained swap-out; low `available`
Is the disk the bottleneck?	`vmstat` / `iostat -x`	`wa`, `%util`, `await`	High `wa`; `%util` near 100%, rising `await`
Which core is pegged?	`mpstat -P ALL 1 2`	per-core `%usr` / `%iowait`	One core at ~100% (single-threaded)
Which process is eating disk?	`iotop -bo` / `pidstat -d`	`IO>` / `kB_wr/s`	One PID dominating writes
Is usage trending up?	`sar -f /var/log/sa/saDD`	`%memused` / `%util` over time	A curve that climbs and never recovers
(any of the above)	iostat/sar/mpstat	the FIRST sample	It's a since-boot average — discard it

LPIC-2 Capacity Planning: Measure, Troubleshoot, and Predict Resource Usage

Why capacity planning is a discipline, not a guess

The live dashboard: vmstat

The classic gotcha: the first sample lies

Is it really CPU? mpstat, free, and top

Blame the process: iotop and pidstat

Predict the future: sysstat, sar, and trending

Try it in your live terminal