Signal9Signal9 — home
online
all lessons

vim Power Tools: Marks, Filters, Config, Windows and Many Files

Intermediate40 min8 steps
What you'll learn
  • Set marks, jump between them, and use marks as the ends of a range
  • Use the yank, numbered and black-hole registers, and copy or move lines with :t and :m
  • Filter lines through external commands (:%!cmd, :'a,'b!cmd, !!cmd) and read command output into a file (:r !cmd)
  • Transform text in place with :sort, :g/:v and case operators (gU, gu, ~, \U, \L)
  • Configure vim with :set and ~/.vimrc, query options, and write mappings that work
  • Tell buffers, windows and tabs apart, and navigate each
  • Apply one edit across many files with :args, :argdo and :bufdo, saving only what changed
  1. Marks: bookmarks inside a file

    Set marks, jump back to them, and use two marks as the ends of a range.

    A mark is a bookmark on a position. Press m and a letter: ma marks the cursor position as a. Later, 'a (apostrophe) jumps to the line of mark a, and ` a ` (backtick) jumps to the exact *line and column*. Lowercase marks a–z belong to one file. Uppercase marks A–Z are **file marks**: 'C` takes you back to that position even from another file, which makes them handy for "the config I keep returning to".

    Vim also sets marks for you. '' jumps back to where you were before your last jump (a search, a G, a :42), and pressing it again returns, so two presses toggle between two places. ` . ` goes to the position of your last change. :marks` lists everything that's set.

    The real power is that a mark is a line address, so it can end a range. Mark the first line of a block with ma, move to its last line and mark it with mb, and now :'a,'bd deletes the block, :'a,'bs/old/new/g replaces only inside it, and :'a,'b!sort filters just those lines (step 3). Visual mode does the same thing behind the scenes: after a selection, : pre-fills :'<,'>, which are the marks for the start and end of the selection.

    printf 'line1\nline2\nline3\nline4\nline5\nline6\n' > /tmp/vp-marks.txt && vim -es -c '2mark a' -c '4mark b' -c "'a,'bd" -c 'wq' /tmp/vp-marks.txt && cat /tmp/vp-marks.txt
    line1
    line5
    line6
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    'a,'bd
    delete (d) the range from mark a to mark b
    :2mark a is the Ex form of pressing ma on line 2, and :4mark b of pressing mb on line 4. Then :'a,'bd deletes everything from mark a to mark b inclusive: lines 2, 3 and 4. Interactively it's ma, move, mb, then :'a,'bd.

    Check yourself

    You marked a function's first line with ms and its last with me. What Ex command replaces foo with bar only inside the function?

  2. Cut, copy and paste, the parts the basics skip

    Use the yank, delete and black-hole registers deliberately, and copy or move lines with :t and :m.

    You already know yy, dd and p. The surprise comes next: you yank a line, delete some junk, press p, and get the junk instead of what you yanked. Every delete also goes into the unnamed register "", which is what p pastes. Vim keeps the pieces apart for you:

    • "0 always holds your last yank, untouched by deletes, so "0p pastes what you copied even after deleting in between.
    • "1 to "9 hold your last nine line deletes, newest first.
    • "_ is the black hole: "_dd deletes without touching any register.
    • "a to "z are yours: "ayy yanks into a, and "Ayy (capital) appends to it.
    • "+ is the system clipboard, on builds that have it (vim --version | grep clipboard). Over SSH it usually isn't there.

    :reg shows them all.

    For whole lines, Ex has two commands that skip the registers entirely. :t (copy, short for transfer) and :m (move) take a destination line: :5t. copies line 5 below the cursor, :1t$ copies line 1 to the end of the file, and :m0 moves the current line to the top. They take ranges too: :'a,'bt$ duplicates a marked block at the end. When you know the line numbers, this is faster than jumping, yanking, jumping back and pasting.

    printf 'alpha\nbravo\ncharlie\n' > /tmp/vp-lines.txt && vim -es -c '1t$' -c '2m0' -c 'wq' /tmp/vp-lines.txt && cat /tmp/vp-lines.txt
    bravo
    alpha
    charlie
    alpha
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    1t$
    copy line 1 to after the last line ($)
    2m0
    move line 2 to after line 0, i.e. the top
    :1t$ copies line 1 (alpha) to the end. Then :2m0 moves line 2 (bravo) to the top. Notice the second command sees the file as the first left it, so line numbers shift as you go.

    Check yourself

    You yanked a line, then ran dd twice to delete two other lines. Which register still holds the yanked line?

  3. Filter through the shell: vim plus every Unix tool

    Pipe lines through an external command, and read a command's output into the file.

    Vim doesn't need its own sort, uniq, column or jq, because it can filter text through any of them. A filter sends lines to a command's standard input and replaces them with its standard output:

    • :%!sort -u filters the whole file (%) through sort -u.
    • :'a,'b!column -t filters a marked range; :'<,'>!… does the same for a visual selection.
    • !!cmd in Normal mode filters the current line, and !}cmd filters to the end of the paragraph. ! followed by any motion works.

    The mirror image is reading a command's output into the file without replacing anything: :r !cmd inserts its output below the cursor, and :3r !cmd below line 3. :r file does the same with a file.

    printf 'web02\ndb01\nweb01\nweb02\ncache01\n' > /tmp/vp-hosts.txt && vim -es -c '%!sort -u' -c 'wq' /tmp/vp-hosts.txt && cat /tmp/vp-hosts.txt
    cache01
    db01
    web01
    web02
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    %!sort -u
    filter every line through sort -u
    The whole buffer went through sort -u and came back sorted with the duplicate web02 gone. The file is only written by wq, so until then u would undo the filter like any other edit.
    printf '# keep this header\nzeta\nalpha\nmike\n# keep this footer\n' > /tmp/vp-block.txt && vim -es -c '2,4!sort' -c 'wq' /tmp/vp-block.txt && cat /tmp/vp-block.txt
    # keep this header
    alpha
    mike
    zeta
    # keep this footer
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    2,4!sort
    filter lines 2 to 4 only
    Only lines 2 to 4 went through sort, and the header and footer stayed put. Filtering a range instead of the whole file is what makes this safe on real configs.
    printf 'kernel:\n' > /tmp/vp-report.txt && vim -es -c '1r !uname -s' -c 'wq' /tmp/vp-report.txt && cat /tmp/vp-report.txt
    kernel:
    Linux
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    1r !uname -s
    read (r) the command's output in below line 1
    :1r !uname -s ran the command and inserted its output below line 1. The same trick drops a checksum, a date or a hostname into a file exactly where you want it.
    printf 'release v1.4\n' > /tmp/vp-release.txt && printf '# release digest:\n# end\n' > /tmp/vp-digest.txt && vim -es -c '1r !sha256sum /tmp/vp-release.txt' -c 'wq' /tmp/vp-digest.txt && cat /tmp/vp-digest.txt
    # release digest:
    834a929794b4fd3a5de04213cac18152a20526fc02ed6cc7967e9c839df2c53c  /tmp/vp-release.txt
    # end
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    1r !sha256sum FILE
    insert the digest line that sha256sum prints below line 1
    The checksum case: sha256sum prints a file's SHA-256 digest followed by its name, and :1r ! dropped that line directly under the # release digest: label, between the two comments, with nothing retyped by hand. Interactively you would put the cursor on the label line and type :r !sha256sum release.tar: with no line number, :r inserts below the cursor.

    Check yourself

    What Ex command replaces the whole buffer with the output of sort -n on it?

  4. Transform without leaving vim: :sort, :g and case

    Sort, delete by pattern, and change case with vim's own commands.

    Several transforms don't need the shell at all, which matters on a stripped-down box.

    • :sort sorts by byte value, so capitals come first. :sort u also drops duplicates, :sort n sorts numerically (so 9 comes before 10), :sort i ignores case and :sort! reverses. It takes a range like any Ex command: :'a,'bsort.
    • :g/pattern/cmd runs an Ex command on every line that matches: :g/^#/d deletes all comment lines, and :g/TODO/t$ copies every TODO line to the end. :v/pattern/cmd (or :g!) does the same for lines that don't match.
    • Case: in Normal mode ~ flips one character, and gU{motion} / gu{motion} upper- or lowercase a motion, so gUiw capitalises the word under the cursor and guu lowercases the line. In a substitution, \U and \L switch the replacement's case: :%s/.*/\L&/ lowercases every line.
    printf 'b\nA\na\nc\nb\n' > /tmp/vp-s.txt && vim -es -c 'sort u' -c 'wq' /tmp/vp-s.txt && cat /tmp/vp-s.txt
    A
    a
    b
    c
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    sort u
    sort the buffer and keep only unique lines
    :sort u sorted by byte value (capital A before lowercase) and removed the duplicate b. Add i (:sort ui) and A and a would count as the same line.
    printf '10\n9\n100\n' > /tmp/vp-n.txt && vim -es -c 'sort n' -c 'wq' /tmp/vp-n.txt && cat /tmp/vp-n.txt
    9
    10
    100
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    sort n
    sort by the first number on each line
    Numeric sort: 9, 10, 100. A plain :sort would give 10, 100, 9, because it compares characters and 1 sorts before 9.
    printf '# comment\nport=80\n\n# another\nhost=web01\n' > /tmp/vp-g.txt && vim -es -c 'g/^#/d' -c 'g/^$/d' -c 'wq' /tmp/vp-g.txt && cat /tmp/vp-g.txt
    port=80
    host=web01
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    g/^#/d
    on every line matching ^#, run d (delete)
    Two global commands: delete every line starting with #, then every empty line. What's left is the live configuration, the same view grep -v '^#' | grep -v '^$' would give you, but edited in place.
    printf 'hostname=web01\nrole=frontend\n' > /tmp/vp-case.txt && vim -es -c '%s/^[a-z]\+/\U&/' -c 'wq' /tmp/vp-case.txt && cat /tmp/vp-case.txt
    HOSTNAME=web01
    ROLE=frontend
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    \U&
    the matched text, uppercased
    & is the whole match, and \U uppercases what follows it in the replacement. So only the key before each = changed, and the values kept their case.

    Check yourself

    Which command deletes every line that does NOT contain the word ERROR?

  5. Configure vim: :set now, ~/.vimrc forever

    Change, query and persist options, and write a mapping that works.

    Every option can be changed live with :set, and it lasts until you quit. Put the same lines in `~/.vimrc` (or ~/.vim/vimrc) and they apply every time vim starts.

    • :set number turns a boolean option on, :set nonumber turns it off, and :set number! or :set invnumber toggles it.
    • :set tabstop=4 sets a value. :set tabstop? asks for the current value, and :set all lists every option.
    • Options worth knowing: tabstop/shiftwidth (how wide a tab and an indent are), expandtab (insert spaces instead of tabs), number/relativenumber, ignorecase with smartcase (case-insensitive search unless you type a capital), hlsearch/incsearch, and list (show tabs and line ends).

    Mappings bind a key to keystrokes. Two rules keep them working:

    1. Use the `noremap` forms: nnoremap for Normal mode, inoremap for Insert mode. Plain nmap/map re-expand other mappings inside the right-hand side, which leads to surprises and loops.

    2. A mapping replays the keys literally. nnoremap <F5> :set invlist types the command but never presses Enter, so nothing happens. It needs <CR> at the end: nnoremap <F5> :set invlist<CR>.

    After editing ~/.vimrc, :source ~/.vimrc reloads it without restarting. vim -u NONE starts with no configuration at all, which is the quickest way to find out whether your vimrc is causing a problem.

    printf 'set tabstop=4 shiftwidth=4 expandtab\nset ignorecase smartcase\nnnoremap <F5> :set invlist<CR>\n' > /tmp/vp-demo.vimrc && vim -es -u /tmp/vp-demo.vimrc -c 'set ts? sw? et? ic? scs?' -c 'q' </dev/null 2>&1
      tabstop=4  shiftwidth=4  expandtab  ignorecase  smartcase
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    -u /tmp/vp-demo.vimrc
    use this file as the vimrc
    set ts? sw?
    print the current values (short option names work too)
    -u FILE starts vim with that file as its vimrc, and :set opt? prints each option's value. This is how you check a vimrc does what you meant without opening an editor: the values printed are the ones vim is actually using.
    vim -es -u /tmp/vp-demo.vimrc -c 'verbose nmap <F5>' -c 'q' </dev/null 2>&1
    n  <F5>        * :set invlist<CR>
    	Last set from /tmp/vp-demo.vimrc line 3
    -es
    silent Ex mode: run the -c commands with no screen, then exit
    verbose nmap <F5>
    list the Normal-mode mapping for F5 and where it was set
    :nmap <F5> shows what F5 is bound to in Normal mode (:verbose in front also says which file and line set it, which is how you track down a mapping you didn't expect). The * means non-recursive, made with nnoremap, and the <CR> at the end is what actually runs the command.

    Check yourself

    You want line numbers every time vim starts. Where does set number go, and how do you apply it to the vim you have open right now?

  6. Buffers, windows and tabs

    Tell the three apart, and open, move between and close each.

    Three words that trip everyone up, from the inside out:

    • A buffer is a file loaded into memory. :e file opens one, :ls lists them with numbers, :b 3 or :b nam<Tab> switches, :bn/:bp step next and previous, and :bd unloads one. A buffer can stay open without being visible.
    • A window is a viewport onto a buffer. :sp file splits horizontally, :vs file splits vertically, and two windows can show the same buffer. Ctrl-w is the window prefix: Ctrl-w w cycles, Ctrl-w h/j/k/l moves in a direction, Ctrl-w q closes this window, Ctrl-w o closes all the others, and Ctrl-w = equalises sizes.
    • A tab page is a whole layout of windows. :tabnew file opens one, gt/gT move to the next or previous tab, and :tabclose closes it.

    From the shell: vim -o a b opens files in horizontal splits, vim -O a b in vertical splits, and vim -p a b in tabs.

    Example — not runnable on your lab box

    vim -O /etc/hostname /etc/hosts
    (two vertical windows: /etc/hostname on the left, /etc/hosts on the right)
    -O
    open each file in its own vertical split
    Two files side by side, one per window. Ctrl-w l moves right, Ctrl-w h moves back, and :qa quits everything at once. This needs a real screen, so run it in your lab terminal, not in a script.

    Check yourself

    You split the screen and both halves show app.conf. Is that one buffer or two?

  7. Many files, one command: :args, :argdo, :bufdo

    Run the same edit across a set of files and save only what changed.

    :wn walks files one at a time. When the same change applies to twenty files, let vim loop instead. The argument list is the set of files vim was started with, or that you give it later: :args /etc/app/*.conf replaces the list, and :args **/*.conf recurses. Then:

    • :argdo CMD runs an Ex command in every file in the list.
    • :bufdo CMD runs it in every open buffer, and :windo CMD in every visible window.

    The standard pattern is :argdo %s/^old=/new=/e | update. The e flag stops a file that has no match from aborting the loop with an error. update writes the file only if it changed, where w would rewrite every file. After a big argdo, :args shows the list and :ls shows which buffers were modified.

    mkdir -p /tmp/vp-fleet && for h in web01 web02 db01; do printf 'timeout=30\nconnect_timeout=5\n' > /tmp/vp-fleet/$h.conf; done && vim -es -c 'args /tmp/vp-fleet/*.conf' -c 'argdo %s/^timeout=/deadline=/e | update' -c 'qa' && head -n 2 /tmp/vp-fleet/*.conf
    ==> /tmp/vp-fleet/db01.conf <==
    deadline=30
    connect_timeout=5
    
    ==> /tmp/vp-fleet/web01.conf <==
    deadline=30
    connect_timeout=5
    
    ==> /tmp/vp-fleet/web02.conf <==
    deadline=30
    connect_timeout=5
    -es
    silent Ex mode: run the -c commands with no screen, then exit. Every keystroke command here has an Ex-command twin, which is what makes these demos runnable
    args …
    set the argument list
    argdo … | update
    run the substitution in each file, write only changed ones
    Three files, one command. The ^ anchor matters: connect_timeout=5 also contains timeout=, and without the anchor it would have become connect_deadline=5. A pattern that's right on one file can be wrong on twenty, so anchor it.

    Check yourself

    Why update rather than w at the end of :argdo %s/x/y/e | update?

  8. Try it in your live terminal

    Practise each technique on a scratch file, then take on the three labs.

    Each of these runs on any lab box. They create their own scratch files under /tmp, so try them, then open the same files interactively with vim and do the same edits with the keystroke forms: ma/mb, !}sort, gUiw, :sp, Ctrl-w w. The three vim labs grade the same skills on a real file: Filter Pipeline (ranges, filters and :r !cmd), Make vim Yours (a vimrc that vim itself verifies), and Fleet Refactor (:argdo across a directory without collateral damage).

    printf '3\n1\n2\n1\n' > /tmp/vp-try.txt && vim -es -c '%!sort -un' -c 'wq' /tmp/vp-try.txt && cat /tmp/vp-try.txt
    A whole-file filter through sort -un (numeric, unique).
    vim -es -c '$r !date +%Y' -c 'wq' /tmp/vp-try.txt && tail -n 1 /tmp/vp-try.txt
    Read a command's output below the last line ($).
    vim -es -u NONE -c 'set ts=2 et' -c 'set ts? et?' -c 'q' </dev/null 2>&1
    Set two options, then query them. Nothing is saved: without a vimrc, :set lasts for this session only.
Recap
  • A mark is a line address: ma sets it, 'a jumps to it, and :'a,'b… uses two marks as a range. '' jumps back to where you were.
  • "0 always holds your last yank, and "_ deletes without touching the registers. :t and :m copy and move lines by number.
  • :%!cmd filters the file through any Unix tool, :'a,'b!cmd filters a range, and :r !cmd reads a command's output in.
  • :sort (with u, n, i), :g/pat/cmd and :v/pat/cmd, and the case operators transform text without leaving vim.
  • :set changes an option now and ~/.vimrc makes it permanent. :set opt? asks for the value. Mappings need nnoremap and a closing <CR>.
  • A buffer is a file in memory, a window is a view of a buffer, and a tab page is a layout of windows.
  • :argdo %s/^old=/new=/e | update edits a whole file set, and writes only the files that changed. Anchor the pattern.