Signal9Signal9 — home
online
all lessons

Shell Productivity: History, Job Control & Power Tricks

Intermediate45 min10 steps
What you'll learn
  • Search and reuse command history with Ctrl+r, !!, !$ and Alt+.
  • Tune history (HISTSIZE, HISTCONTROL, shopt -s histappend, HISTTIMEFORMAT) so it is long, deduplicated and never clobbered
  • Move between working directories with cd -, CDPATH and the pushd/popd/dirs stack
  • Generate filenames and sequences with brace expansion — and know how it differs from globbing
  • Feed multi-line input to a command inline with here documents (<<EOF vs <<'EOF')
  • Drive background work with &, Ctrl+z, jobs, fg and bg, capture a job's PID with $!, and keep it alive past logout
  • Split a pipeline to screen AND file with tee; re-run a command on a cadence with watch
  • Keep long-running work alive across SSH drops with a tmux or screen session
  1. Stop retyping: your history is a database

    Find any command you've ever run with Ctrl+r, and see recent history at will.

    There's a visible difference between someone who knows Linux and someone who's fast at it, and it isn't knowledge of more commands — it's that the fast one almost never types a full command twice, never loses work to a dropped SSH session, and never sits idle watching a slow job. This lesson is that layer: not new capabilities, but the control habits that multiply everything you already know. Each section is one habit; each habit pays daily for the rest of your career.

    Watch someone fast in a shell and you'll notice they barely type full commands — they retrieve them. The shell records everything you run (the history builtin shows it, ~/.bash_history stores it), and the retrieval tool is reverse incremental search:

    Press Ctrl+r and start typing any fragment of the command you're hunting — the newest match appears as you type. Press Ctrl+r again to step to older matches. Enter runs the found command, → / Esc puts it on the line to edit first, Ctrl+g bails out.

    It's called reverse (newest→oldest) incremental (matches update per keystroke) search — you'll see (reverse-i-search) as the prompt.

    (reverse-i-search)`conf': grep -rl 'debug = true' /etc/app --include='*.conf'
    What Ctrl+r looks like mid-search: typing just 'conf' resurfaced a whole grep you ran yesterday. Fragment in, command out — that's the habit.
    history | tail -5
      501  cd /var/log
      502  grep -c ERROR app.log
      503  tail -f app.log
      504  cd -
      505  history | tail -5
    The numbered record Ctrl+r searches. !503 re-runs entry 503 by number — occasionally handy, but Ctrl+r is almost always faster than reading numbers.
  2. Make history big, deduped and durable

    Tune the history settings so Ctrl+r has a long, clean, timestamped record to search.

    Ctrl+r is only as good as the history behind it, and bash's defaults work against you. The list is short (often 500 or 1000 commands), it records every duplicate ls you type, it stamps nothing with a time, and, worst of all, each shell overwrites ~/.bash_history when it exits: with two terminals open, whichever closes last wipes out the other's commands.

    Five lines in ~/.bashrc (or a drop-in it sources) fix all of it:

    `

    HISTSIZE=10000 # commands kept in memory for this shell

    HISTFILESIZE=20000 # lines kept in ~/.bash_history on disk

    HISTCONTROL=ignoreboth:erasedups # skip duplicates and space-prefixed commands

    shopt -s histappend # append to the file on exit, never overwrite it

    HISTTIMEFORMAT='%F %T ' # history shows when each command ran

    `

    ignoreboth is shorthand for ignorespace (a command typed with a leading space is never saved, handy for one that carries a secret) plus ignoredups (don't save a repeat of the previous line); erasedups also removes older copies. `shopt` is bash's switchboard for its on/off options: shopt -s NAME sets one, shopt -u NAME unsets it, and a bare shopt NAME reports its state.

    shopt -s histappend && shopt histappend
    histappend     	on
    -s
    set (enable) the named shell option
    shopt NAME
    with no -s/-u, print whether the option is on or off
    Turn histappend on, then ask shopt to report it: 'on'. The variables above are plain shell assignments, but histappend is a shell option, which is why it needs shopt rather than NAME=value. Like every setting here it lasts only for this shell until it lives in a startup file.
  3. Repeat & reuse: !!, !$ and Alt+.

    Reuse the last command and the last argument without retyping either.

    Three tiny idioms cover most repetition:

    • !! — the entire previous command. Classic: you forgot privileges, so sudo !!.
    • !$ — the last argument of the previous command. mkdir -p /srv/app/releases/v2 then cd !$.
    • Alt+. — inserts the previous command's last argument onto your line, visibly, before you run anything; press it again to walk back through older commands' last arguments.

    Example — not runnable on your lab box

    ls -l /etc/nginx/sites-available/default
    vim !$
    vim /etc/nginx/sites-available/default
    The shell echoes the expanded command before running it — inspect that echo. !$ saved retyping the whole path; the pattern 'inspect it, then act on it' is where !$ shines.
  4. Navigate faster: cd -, CDPATH and the directory stack

    Hop between the few directories you work in without retyping their full paths.

    A lot of typing goes into cd with long paths. Three tools cut it down:

    • `cd -` jumps back to the previous directory (and prints it). Run it again to toggle between two places.
    • `CDPATH` is a search path for cd, the way PATH is for commands. With CDPATH=.:/srv/shop, typing cd api from anywhere lands in /srv/shop/api if there is no ./api here. Keep . first so a local directory still wins, set it in ~/.bashrc, and don't export it: scripts that cd would start resolving names through it too.
    • The directory stack is a working set of directories. `pushd DIR` changes to DIR and pushes it onto the stack; `dirs -v` lists the stack, numbered, with entry 0 being where you are now; `popd` removes the top entry and drops you back into the one beneath it. Think of it as cd with a memory you can walk back through.
    mkdir -p /tmp/proj/api /tmp/proj/web /tmp/proj/db && cd /tmp
    pushd /tmp/proj/api && pushd /tmp/proj/web && pushd /tmp/proj/db
    dirs -v
    popd && pwd
    /tmp/proj/api /tmp
    /tmp/proj/web /tmp/proj/api /tmp
    /tmp/proj/db /tmp/proj/web /tmp/proj/api /tmp
     0  /tmp/proj/db
     1  /tmp/proj/web
     2  /tmp/proj/api
     3  /tmp
    /tmp/proj/web /tmp/proj/api /tmp
    /tmp/proj/web
    pushd DIR
    cd to DIR and push it onto the directory stack
    dirs -v
    list the stack one entry per line, numbered from 0 (the current directory)
    popd
    remove the top entry and cd to the one beneath it
    Each pushd changes directory and prints the whole stack, newest on the left, so the stack grows from the starting point /tmp to four entries. dirs -v prints the same stack one per line with index numbers: 0 is the current directory. popd pops /tmp/proj/db off the top, prints what is left, and puts you in the new top, /tmp/proj/web, which pwd confirms.
    mkdir -p /tmp/proj/web && cd / && CDPATH=.:/tmp/proj && cd web && pwd
    /tmp/proj/web
    /tmp/proj/web
    CDPATH=.:/tmp/proj
    cd looks in the current directory first, then in /tmp/proj
    From / there is no ./web, so cd searched CDPATH and found /tmp/proj/web. The first line is cd itself: when it finds the directory through CDPATH it prints where it went, so you are never surprised. The second line is pwd agreeing.

    Check yourself

    You pushd into /srv/a, then /srv/b, then /srv/c. Which directory are you in after one popd?

  5. Brace expansion: generate many from one

    Create sets and sequences of names without typing each one.

    Brace expansion manufactures strings before the command runs:

    `

    mkdir -p app/{src,tests,docs} # three directories, one command

    cp nginx.conf{,.bak} # expands to: cp nginx.conf nginx.conf.bak

    touch log-{01..05}.txt # sequences: log-01.txt … log-05.txt

    echo {a..e}{1..3} # cross-product: a1 a2 a3 b1 …

    `

    The {,.bak} backup idiom is the one you'll use weekly: an empty first alternative plus a suffix means 'the file, and the file with .bak stapled on'.

  6. Here documents: multi-line input, inline

    Feed a command several lines of stdin without a separate file.

    Sometimes a command wants a small file's worth of input and creating one is overkill. A here document supplies multi-line stdin inline, between <<MARKER and a line containing only MARKER (conventionally EOF):

    `

    cat > /etc/motd <<EOF

    Welcome to the build box.

    Deploys are announced in #ops.

    EOF

    `

    Quoting the marker changes everything: <<EOF lets the shell expand $variables inside the block; <<'EOF' (quoted) passes the text verbatim — essential when the content itself contains $ you want kept, like a script you're writing to disk. A <<-EOF variant additionally strips leading tabs so the block can be indented inside scripts.

    cat <<EOF
    user: $USER
    home: $HOME
    EOF
    user: root
    home: /root
    Unquoted marker → variables expanded. Re-run it as <<'EOF' and you'd get the literal strings $USER and $HOME — the quoting decision is per-heredoc, made at the marker.
  7. Job control: run it in the back, pull it to the front

    Move work between foreground and background deliberately.

    Your one terminal can run many things:

    • command & — start in the background; you get the prompt back immediately.
    • Ctrl+z — suspend (pause) the foreground job and return to the prompt.
    • jobs — list this shell's jobs with their numbers.
    • bg %1 — resume job 1 in the background (Ctrl+z then bg = 'keep going, but give me my prompt').
    • fg %1 — bring it back to the foreground (to watch it, or Ctrl+c it).

    The rhythm to internalize: started something long in the foreground by mistake? Ctrl+z, then bg. Same effect as if you'd remembered the &.

    sleep 300 &
    jobs
    [1] 4127
    [1]+  Running                 sleep 300 &
    & backgrounds it at launch; jobs shows [1] with its PID. fg %1 would reattach it to your keyboard.

    A job number like %1 only means something inside the shell that started the job. To manage it from a script or another terminal you need its PID, and the shell hands it to you: right after command &, the special variable `$!` holds the PID of the job you just backgrounded. Save it (echo $! > app.pid) and anything can later check it with kill -0 "$(cat app.pid)" or stop it with kill. If you didn't save it, `pgrep -f PATTERN` finds it again by matching the full command line, which is what you want for scripts, whose process name may just be the interpreter.

    sleep 300 &
    echo "started PID $!"
    jobs -l
    kill $!
    started PID 4127
    [1]+  4127 Running                 sleep 300 &
    $!
    PID of the most recently backgrounded job
    jobs -l
    list jobs with their PIDs
    $! was set the instant sleep went into the background, and it matches the PID jobs -l shows. kill $! then stops exactly that process, with no ps or pgrep lookup and no risk of hitting a different sleep. Your PID will differ: it is whatever the kernel assigned.
  8. tee and watch: see it AND keep it, run it again and again

    Capture a pipeline without losing the live view, and monitor a command on a cadence.

    tee duplicates a stream: whatever flows in goes to the screen and into a file — named after the T-shaped pipe fitting.

    `

    ./deploy.sh 2>&1 | tee deploy.log # watch it live, keep the transcript

    grep ERROR app.log | tee errors.txt | wc -l # save the middle of a pipeline

    echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf # the sudo-write idiom

    `

    That last one matters: sudo echo x >> file does NOT work (the shell doing the >> isn't root — sudo only elevated echo). Piping into sudo tee -a puts the elevated process on the writing end. -a appends instead of overwriting.

    watch re-runs a command every 2 seconds (-n to change) and repaints the screen — a live dashboard from any one-liner: watch -n1 'ss -tlnp', watch df -h. -d highlights what changed between runs.

    Check yourself

    A colleague runs sudo echo '127.0.0.1 build01' >> /etc/hosts and gets 'Permission denied'. Why — and what's the fix?

  9. tmux & screen: sessions that survive the disconnect

    Keep long-running work alive across SSH drops and pick it up from anywhere.

    SSH from a laptop, start a 3-hour migration, close the lid — dead session, dead migration. A terminal multiplexer breaks that link: the session runs on the server, and your terminal is just a window into it.

    tmux (the modern default):

    `

    tmux new -s migrate # named session

    # … start the long job …

    # Ctrl+b d # detach — job keeps running server-side

    tmux ls # later, even from a different machine:

    tmux attach -t migrate # right where you left it

    `

    screen (predecessor, still on older boxes): screen -S name, detach with Ctrl+a d, resume with screen -r name. Same idea, Ctrl+a instead of Ctrl+b.

    Multiplexers also split panes and juggle windows, but the career-saving feature is exactly this: detach, disconnect, reattach. Any long-running anything on a remote box belongs inside one.

    Check yourself

    You're 2 hours into a 3-hour database migration over SSH when your train enters a tunnel. Under what condition does the migration survive — and what do you type when you're back online?

  10. Now do it — wire the habits into your fingers

    Run each productivity move once, deliberately, in your live terminal

    Productivity habits are pure muscle memory — reading about Ctrl+r does nothing; pressing it thirty times does everything. Run this circuit once now, then notice for the rest of the week every time you retype something the circuit would have retrieved.

    ls -l /etc/hostname
    cd /tmp && echo 'try: vim !$ — then Alt+. to see the visible version'
    Set up a last-argument, then reuse it both ways: !$ (blind expansion — fine for inspection commands) and Alt+. (the argument appears ON your line before you commit — the safe default). Feel the difference; near an rm, only one of these is acceptable.
    mkdir -p /tmp/prod-lab/app/{src,tests,docs} && ls /tmp/prod-lab/app
    touch /tmp/prod-lab/app/config.yml && cp /tmp/prod-lab/app/config.yml{,.bak} && ls /tmp/prod-lab/app
    Brace expansion twice: a three-directory skeleton in one command, then the {,.bak} backup idiom — the one you'll use before every risky config edit for the rest of your career. Verify with the ls: config.yml and config.yml.bak, no retyping.
    sleep 300
    # Ctrl+z … then:  bg  … then:  jobs  … then:  fg %1  … then Ctrl+c
    The job-control rescue rhythm, end to end: start something long in the foreground 'by mistake', Ctrl+z to suspend, bg to let it keep running while you get your prompt back, jobs to see it, fg to reclaim it, Ctrl+c to kill it. Once through this loop deliberately and you'll never be trapped behind a slow foreground command again.
    df -h | tee /tmp/prod-lab/disk.txt | tail -2 && cat /tmp/prod-lab/disk.txt | head -2
    tee proving both halves of its promise: the pipeline still flowed (tail printed), AND the transcript exists (cat shows it). Wrap up by trying watch -n1 date for five seconds — Ctrl+c to exit — and you've touched every habit in the lesson.
Recap
  • Ctrl+r reverse-searches history incrementally — fragment in, full command out; press again for older matches
  • Make history worth searching: HISTSIZE/HISTFILESIZE for length, HISTCONTROL=ignoreboth:erasedups against duplicates, shopt -s histappend so shells append instead of overwriting, HISTTIMEFORMAT for timestamps
  • !! repeats the last command (sudo !!), !$ reuses its last argument — but Alt+. shows the argument before you commit, which is the safe default near destructive commands
  • cd - toggles back, CDPATH lets cd find directories from anywhere, and pushd/popd/dirs -v keep a stack of directories to hop between
  • Brace expansion generates strings ({1..5}, file{,.bak}); globs only match what already exists
  • Here documents feed multi-line stdin inline: <<EOF expands variables, <<'EOF' is verbatim
  • & backgrounds at launch; Ctrl+z then bg backgrounds after the fact; jobs/fg/%N manage them; $! is the last background PID (pgrep -f finds it later); nohup or a multiplexer for anything that must outlive the session
  • tee splits a stream to screen and file (and | sudo tee -a is how you append to root-owned files); watch re-runs any command on a cadence
  • tmux/screen sessions live on the server: detach with Ctrl+b d, reattach with tmux attach — long-running remote work always goes inside one