Advanced Bash: Arrays, Expansion & Substitution

Advanced35 min6 steps

What you'll learn

Hold a list in an indexed array — build it, count it, index the ends, and iterate it safely
Reshape strings with parameter expansion instead of forking cut/sed/basename
Strip prefixes/suffixes, substitute, default, and change case with ${...} operators
Capture a command's output with command substitution, and feed output as a file with process substitution
Recognise the quoting rule that keeps arrays and expansions from silently word-splitting

Beyond one string: arrays hold a list
See why a real list beats re-splitting a string every time
A plain variable holds one value. The moment you need several — a set of hostnames, the files to back up, a queue of ports — people reach for a space-separated string and re-split it on every use. That works until a value contains a space, and then it shatters silently.
An indexed array is Bash's real list type: an ordered, numbered collection you build once and then count, index, slice, and iterate — without ever re-parsing. It's the difference between keeping a shopping list on a card versus re-reading a run-on sentence and guessing where each item ends.
Why this matters on real boxes
Every non-trivial script gathers a set of things — matched files, target hosts, failed units — and then does something to each. Arrays are how you carry that set around correctly. The alternative (space-joined strings) is the single most common source of "it worked until a filename had a space in it" bugs.
Check yourself
What breaks a space-separated string used as a pretend list?
Indexed arrays: build, count, index, iterate
Create an array and pull the pieces out of it
Declare an array with parentheses; elements are separated by whitespace. Then four operators do almost everything: ${arr[i]} reads one element, ${#arr[@]} counts them, ${arr[@]} expands to all of them, and a negative index counts from the end. Always iterate `"${arr[@]}"` in double quotes — that expands to one word per element, spaces and all.
```
$ hosts=(web1 db1 "cache node" gw)
echo "count: ${#hosts[@]}"
echo "first: ${hosts[0]}  last: ${hosts[-1]}"
for h in "${hosts[@]}"; do echo "- $h"; done
```
```
count: 4
first: web1  last: gw
- web1
- db1
- cache node
- gw
```
${#arr[@]}
number of elements in the array
${arr[-1]}
last element (negative indices count from the end)
"${arr[@]}"
all elements, each as its own word — the quotes are mandatory
Four elements — note 'cache node' is ONE element because it was quoted at build time, and it stays one line in the loop because we iterated "${hosts[@]}" quoted. ${#hosts[@]} is the length; hosts[-1] is the last element without knowing the count.
```
$ mapfile -t lines < /etc/hostname
echo "read ${#lines[@]} line(s); first is '${lines[0]}'"
```
```
read 1 line(s); first is 'web01'
```
mapfile -t
read stdin/a file into an array, one line per element, no trailing newlines
mapfile -t (a.k.a. readarray) loads a file into an array, one line per element, stripping the trailing newline (-t). This is the correct way to read a file into a list — never 'for line in $(cat file)', which word-splits.
The unquoted-array trap
for h in ${hosts[@]} (no quotes) re-splits every element on whitespace — cache node becomes two iterations. And ${hosts} with no index is not the whole array; it's just element 0. Two habits kill both bugs: index explicitly, and always quote "${hosts[@]}".
Check yourself
How do you get the number of elements in an array named files?
Associative arrays: key → value maps
Use declare -A for lookups, counters, and sets keyed by strings, not numbers
An indexed array is numbered 0, 1, 2… An associative array is keyed by arbitrary strings — a map / dictionary, and the tool you reach for the moment you think "I need to look something up by name." Three classic uses:
- Lookup tables — config[timeout]=30, then read ${config[timeout]}.
- Counters — tally occurrences: count[$word]=$(( ${count[$word]:-0} + 1 )).
- Sets / membership — record what you've seen and test it later.
The operators mirror indexed arrays with one crucial addition: `${!map[@]}` gives the keys (note the !), ${map[@]} the values, and ${#map[@]} the count. The catch that bites everyone: you must `declare -A map` first. Without it, Bash treats the subscript as arithmetic and quietly collapses everything into index 0 — a map that appears to hold only its last value.
```
$ declare -A seen
for w in apple pear apple fig apple pear; do
  seen[$w]=$(( ${seen[$w]:-0} + 1 ))
done
echo "apple: ${seen[apple]}   pear: ${seen[pear]}   fig: ${seen[fig]}"
echo "distinct words: ${#seen[@]}"
```
```
apple: 3   pear: 2   fig: 1
distinct words: 3
```
declare -A
declare an associative (string-keyed) array — required before first use
${!map[@]}
the KEYS of the map (with !); ${map[@]} gives the values
${seen[$w]:-0}
value for key $w, or 0 if the key isn't set yet
declare -A makes seen a real map; each key counts its word (${seen[$w]:-0} defaults a missing key to 0 before incrementing). Reading by key gives the tally, and ${#seen[@]} is the number of distinct keys — a word-frequency counter in five lines, no sort|uniq -c pipeline.
Forgetting declare -A (and that order is unspecified)
Assign to a map key without declare -A first and Bash evaluates the key as an arithmetic expression — non-numeric keys become 0, so every entry overwrites the same slot. Always declare the map first. Second gotcha: associative arrays have no guaranteed iteration order, so if you need keys in order, iterate ${!map[@]} through sort rather than assuming insertion order.
Check yourself
For an associative array named m, what expands to its list of KEYS (not values)?
Parameter expansion: string surgery without subprocesses
Reshape a value using ${...} instead of forking cut/sed/basename
Every basename, dirname, cut, or sed in a loop forks a process — thousands of times over a big list, that's real wall-clock. Bash can slice strings itself with parameter expansion, no subprocess at all. The operators read oddly at first, but there are only a handful and they pattern-match:
- ${v#pat} / ${v##pat} — strip a prefix (shortest / longest match)
- ${v%pat} / ${v%%pat} — strip a suffix (shortest / longest)
- ${v/find/repl} / ${v//find/repl} — replace first / all
- ${v:-default} — use default if v is unset or empty
- ${#v} — length; ${v^^} / ${v,,} — upper / lower case
```
$ p=/var/log/nginx/access.log
echo "dir:  ${p%/*}"
base=${p##*/}
echo "base: $base"
echo "stem: ${base%.*}   ext: ${base##*.}"
echo "upper: ${base^^}"
```
```
dir:  /var/log/nginx
base: access.log
stem: access   ext: log
upper: ACCESS.LOG
```
${p%/*}
strip the shortest /... from the end → the directory
${p##*/}
strip the longest */ from the front → the filename
${base%.*}
strip the extension → the stem
${base##*.}
keep only the extension
A full path split into directory, filename, stem, and extension — plus an uppercase — using nothing but ${...}. %/ removes the shortest /... suffix (the directory); ##/ removes the longest */ prefix (the filename). No basename, no dirname, no fork.
```
$ name=${1:-anonymous}
echo "hello, ${name}"
csv="a,b,,c"
echo "cleaned: ${csv//,,/,}"
```
```
hello, anonymous
cleaned: a,b,c
```
${1:-anonymous}
value of $1, or 'anonymous' if unset/empty
${v//find/repl}
replace ALL occurrences (single slash = first only)
${1:-anonymous} supplies a default when the first argument is missing — the one-line way to make a script forgiving. ${csv//,,/,} replaces every empty field (double comma) globally. Two doubled operators, two different jobs.
:- is one of a small default family, and each member does a distinct job:
- `${v:-default}` — use default if v is unset/empty (v is unchanged).
- `${v:=default}` — use and assign default into v if it was unset/empty.
- `${v:+alt}` — use alt only if v is set (the inverse — handy for "add a flag only when the var exists").
- `${v:?message}` — error out with message if v is unset/empty. One line of required-argument validation at the top of a script: : "${API_KEY:?must be set}".
And for a substring, ${v:offset:length} slices by position: ${v:0:3} is the first three characters. The same :offset:length slices an array — "${arr[@]:1:2}" is two elements starting at index 1.
# is the front, % is the back
The memory hook that makes these stick: on a US keyboard # sits to the left of %, and in expansion # strips from the left (prefix), % strips from the right (suffix). Double the symbol (##, %%) to switch from the shortest match to the greedy longest one.
Check yourself
Given f=report.tar.gz, what does ${f%%.} produce, and why does it differ from ${f%.}?
Command & process substitution
Turn a command's output into a value, or into a file
Command substitution $(cmd) runs a command and drops its stdout in place — the workhorse for capturing a value into a variable. Its lesser-known sibling, process substitution <(cmd), is the trick that unlocks tools which insist on a filename: it hands the command's output to that tool as if it were a file, no temp file to create or clean up.
Example — not runnable on your lab box
```
$ now=$(date +%F)
echo "stamp: $now"
diff <(sort a.txt) <(sort b.txt)
```
```
stamp: 2026-07-16
3c3
< bravo
---
> delta
```
$(cmd)
run cmd, substitute its stdout as text
<(cmd)
run cmd, present its stdout as a readable file path
$(date +%F) captures today's date into a variable. Then the real magic: diff needs two files, but we compare the SORTED versions of two files without writing any temp files — <(sort a.txt) presents sort's output to diff as a filename. That /dev/fd path is created and removed for you.
Don't cross substitution with a captured newline
x=$(cat file) collapses trailing newlines and, unquoted, will word-split when reused. Capture into an array with mapfile -t x < file when you need the lines as a list, and always quote "$x" when you reuse a scalar capture. The bug shows up as a loop that runs the wrong number of times.
Check yourself
A tool takes only a filename argument, but you want to feed it the output of grep ERROR log. What syntax bridges the gap?
Now do it — in your live terminal
Build an array, slice its elements with expansion, and capture output
Muscle memory beats memorisation. Build a small array of paths, then reshape each element with parameter expansion and capture a summary — the exact moves the arrays and parameter-expansion labs will grade.
```
$ paths=(/etc/hosts /var/log/syslog /srv/app/config.yaml)
for p in "${paths[@]}"; do
  echo "${p##*/}  (in ${p%/*})"
done
```
"${paths[@]}"
each element as one word — the quotes prevent splitting
Iterate the quoted array and, for each path, print the filename (##/) and its directory (%/). Run it, then add a fourth path WITH a space in it and confirm the quoted loop keeps it intact.
```
$ mapfile -t users < <(cut -d: -f1 /etc/passwd)
echo "system has ${#users[@]} accounts; first: ${users[0]}"
```
< <(cut ...)
process substitution as mapfile's input — no temp file
Combine all three ideas: process substitution feeds cut's output into mapfile, which builds an array of usernames; ${#users[@]} counts them. Now try ${users[-1]} for the last account. This one line replaces a fragile 'for u in $(...)' loop.

Recap

Indexed arrays are Bash's list type: arr=(a b c); ${#arr[@]} counts, ${arr[0]}/${arr[-1]} index the ends, and you must iterate "${arr[@]}" quoted so spaces don't split elements.
mapfile -t arr < file (or < <(cmd)) is the correct way to read lines into a list — never 'for x in $(cat file)'.
Associative arrays (declare -A) are string-keyed maps for lookups, counters, and sets: set map[key]=val, read ${map[key]}, get keys with ${!map[@]}, count with ${#map[@]} — and declare -A FIRST or keys collapse to index 0.
Parameter expansion reshapes strings with no subprocess: # strips prefixes, % strips suffixes (double for greedy), / substitutes, and the default family :- (use) / := (use+assign) / :+ (alt if set) / :? (error), plus ${v:off:len} slices; ^^ / ,, change case.
$(cmd) captures output as text; <(cmd) presents output as a file so filename-only tools (diff, comm) can consume a stream.
The recurring bug across all of these is word-splitting — quote your expansions and prefer arrays over space-joined strings.

Concept → syntax
Concept	Syntax
Build / index / length	`arr=(a b c)` · `${arr[i]}` · `${#arr[@]}`
Iterate safely	`for x in "${arr[@]}"; do …`
Read file into array	`mapfile -t arr < file`
Strip prefix / suffix	`${v#pat}` / `${v%pat}` (double = greedy)
Substitute / default / case	`${v//a/b}` · `${v:-def}` · `${v^^}`
Capture output / as a file	`$(cmd)` · `<(cmd)`

Advanced Bash: Arrays, Expansion & Substitution

Beyond one string: arrays hold a list

Indexed arrays: build, count, index, iterate

Associative arrays: key → value maps

Parameter expansion: string surgery without subprocesses

Command & process substitution

Now do it — in your live terminal