Streams & Redirection: stdin, stdout, stderr, and the Pipe

Beginner35 min5 steps

What you'll learn

Name the three standard streams (stdin 0, stdout 1, stderr 2) and explain why they're kept separate
Redirect output with >, >>, 2>, and &>, and read input from a file with <
Explain what a pipe actually connects (stdout → stdin) and why stderr isn't carried
Merge streams with 2>&1, and turn a list of items into commands with xargs / find -exec

Every command has three wires
Picture stdin, stdout, and stderr as three separate channels — the idea the whole lesson rests on.
Here's the one mental model that makes redirection and pipes suddenly obvious — get this and the rest is syntax.
Every program, the instant it starts, has three wires already plugged in. Think of a command as a little machine with an in socket and two out sockets:
- stdin (0) — the in wire, where it reads input from. By default that's your keyboard.
- stdout (1) — the out wire for normal results. By default, your screen.
- stderr (2) — a second out wire, just for errors and diagnostics. Also your screen, by default.
Those numbers (0, 1, 2) are the streams' file descriptors — the shell refers to each wire by its number, which is exactly how you'll aim them in a moment.
The part that trips people up — and the part that matters most: stdout and stderr are two separate wires that merely happen to both point at your screen. They look like one stream because both land in the same place, but they're independent. And because they're independent, you can send the real results one direction and the error noise another: keep the answer, bin the complaints. That single fact is the reason > and 2> exist and why they're so useful.
Why split results from errors at all?
Run find / -name secret.txt as a normal user and two very different things come back: the paths you want (on stdout) and a flood of `Permission denied` messages for directories you can't read (on stderr). Because they travel on different wires, find / -name secret.txt 2>/dev/null throws the errors away and leaves you a clean list. If everything shared one stream, separating signal from noise like that would be impossible.
Redirection: aim the wires at files
Re-point each stream from the screen to a file (or from a file into a command).
Redirection is unplugging a wire from the screen and plugging it into a file instead. The shell does this around the command — the program has no idea it happened, which is why redirection works with every command ever written:
- `> file` — send stdout to file, overwriting whatever was there.
- `>> file` — send stdout to file, appending to the end.
- `2> file` — send stderr to file (that 2 is stderr's descriptor from the last step).
- `&> file` (or the older > file 2>&1) — send both stdout and stderr to file.
- `< file` — feed stdin from file instead of the keyboard.
Hold onto one warning we'll come back to: > wipes the file's old contents before the command even runs. When you mean "add to," reach for >>.
```
$ grep root /etc/passwd > users.txt 2> errors.txt; cat users.txt
```
```
root:x:0:0:root:/root:/bin/bash
```
> users.txt
stdout → users.txt (overwrite)
2> errors.txt
stderr → errors.txt
Two redirections on one line: the matching lines (stdout) land in users.txt, any error (stderr) lands in errors.txt, and nothing prints to the screen — until we cat the file back to see it worked. Notice grep itself is completely unchanged; it doesn't know or care where its wires point. Aiming them is the shell's job, not the command's.
```
$ echo 'first line' > note.txt && echo 'second line' >> note.txt && cat note.txt
```
```
first line
second line
```
>
overwrite the file with this output
>>
append this output to the end of the file
The overwrite-vs-append distinction, shown: > creates note.txt with the first line, then >> adds the second instead of replacing it, so both survive. Swap that second >> for a > and you'd be left with only second line — the whole point of the next callout.
> clobbers, and 'sort file > file' eats your data
echo a > log; echo b > log leaves only b — the second > truncated the file before echo ran. When building a file up across commands, use >> after the first write. And never sort file > file: the shell empties `file` to prepare the redirect before sort gets to read it, so you lose the data entirely. Use a temp file (sort file > file.tmp && mv file.tmp file) or sort -o file file. Interactive shells also have a safety catch for the first trap: `set -o noclobber` makes > refuse to overwrite a file that already exists (bash: notes.txt: cannot overwrite existing file), while >> still appends. When you really do mean to replace the file, >| overrides it for that one command, and set +o noclobber turns the catch off again.
> file 2>&1, not 2>&1 > file
To send both streams to one file, the order of the two redirections matters, because the shell applies them left to right and 2>&1 means "point stderr wherever stdout points right now". cmd > all.txt 2>&1 first aims stdout at the file, then copies that aim to stderr: both land in all.txt. cmd 2>&1 > all.txt copies stdout's aim (still the screen) to stderr first and only then moves stdout to the file, so the errors still hit your screen. &> all.txt is bash's shorthand for the correct order.
What a pipe actually connects
See that | wires one command's stdout into the next's stdin — and that stderr is left behind.
A pipe is the same idea as redirection, but instead of pointing a wire at a file, it points it straight at another command. A | B plugs A's stdout directly into B's stdin — no temporary file, the data just streams through memory from one to the next.
This is the secret behind every pipeline you've admired: each small tool reads stdin, does one job, writes stdout, and the pipe snaps them together like plumbing. grep … | cut … | sort … | uniq is four machines wired out-to-in, out-to-in.
And here's the consequence almost everyone misses at first: a pipe carries stdout only. A's errors still spill onto your screen — they do not flow into B. So if B needs to see A's error lines too, you have to merge them onto the stdout wire first, with 2>&1.
```
$ ls /etc /nope 2>&1 | grep -c cannot
```
```
1
```
2>&1
redirect stderr (2) into wherever stdout (1) currently points
grep -c
count matching lines instead of printing them
2>&1 merges stderr into stdout before the pipe, so grep receives the ls: cannot access '/nope' error line and counts it — giving 1. Drop the 2>&1 and that error would skip the pipe entirely, print to your screen, and grep would count 0. Position matters: 2>&1 sits right after the command and before the |, because it means "send wire 2 wherever wire 1 is pointing right now."
Check yourself
A pipe (|) feeds the LEFT command's stdout into the right command's what?
Turning a list into commands: xargs and find -exec
Feed a stream of items as ARGUMENTS to another command.
Pipes feed data as stdin, but a lot of commands — rm, cp, wc, chmod — don't read stdin; they want their targets as arguments on the command line. So find . -name '*.tmp' | rm does nothing useful, because rm ignores stdin. Two tools bridge that gap between "a list of items" and "arguments to a command":
- `… | xargs CMD` — gather the incoming lines and hand them to CMD as arguments. find . -name '*.tmp' | xargs rm actually deletes them.
- `find … -exec CMD {} +` — find runs CMD itself, substituting each match for {}. No pipe at all.
When filenames might contain spaces, prefer the space-safe forms find … -print0 | xargs -0 … or -exec … {} +. For a quick one-off list, plain xargs reads the clearest.
```
$ find /var/log -name '*.log' | xargs wc -l | tail -1
```
```
  1428 total
```
xargs
build an argument list from stdin, then run the command with it
-exec … {} +
find runs the command itself, {} standing in for each match
Read it left to right: find emits matching paths on stdout → xargs turns that list into arguments for wc -l (which counts lines in each file and prints a total) → tail -1 keeps just the total line. The pipe carried the file list; xargs was the adapter that converted list-into-arguments. find /var/log -name '*.log' -exec wc -l {} + does the identical job without a pipe.
A taste of process substitution
Occasionally a command wants a file, not a pipe — diff needs two of them, for instance. diff <(sort a.txt) <(sort b.txt) uses process substitution: <(…) runs the command and hands its output to diff as if it were a temporary file. It's how you 'pipe' into a tool that only accepts filenames — and, fun fact, it's exactly the trick the graders in these labs use under the hood.
Now do it — plumb some pipes
Read it, then run it. Aim a few wires and build a pipeline in your live terminal.
You have a real, resettable shell to the right. Run these and watch the wires move — where output lands, what the pipe carries, and what redirection captures. This plumbing is the backbone of every script and one-liner you'll ever write, so a couple of minutes here pays off constantly.
```
$ echo hello > /tmp/s.txt && echo world >> /tmp/s.txt && cat /tmp/s.txt
```
Create a file with > , add to it with >>, then read it back. You should see both lines — proof that >> appended instead of clobbering. Try it again with two > and watch only the last line survive.
Example — not runnable on your lab box
```
$ ls /etc /does-not-exist > /tmp/out.txt 2> /tmp/err.txt; echo '--- out ---'; cat /tmp/out.txt | head -3; echo '--- err ---'; cat /tmp/err.txt
```
Split the two out-wires into two files: the real listing goes to out.txt (stdout), the 'cannot access' complaint goes to err.txt (stderr). Reading them back separately makes the two-wire model concrete — the results and the error genuinely travelled different paths.
```
$ cat /etc/passwd | grep -c bash
```
A minimal pipe: cat's stdout becomes grep's stdin, and grep -c counts how many accounts use bash. (Purists would write grep -c bash /etc/passwd and skip the cat — but seeing the pipe wire two commands together is the point here.)
Tip
Debugging a pipeline that gives the wrong answer? Build it up one stage at a time — run find … alone, eyeball its output, then add | xargs …, then the next stage. Watching what each wire carries at each step is the fastest way to find where a pipeline goes wrong.

Recap

Every command has three wires: stdin (0) in, stdout (1) normal output, stderr (2) errors. stdout and stderr are SEPARATE wires — that's what lets you keep results and discard noise.
Redirect with > (overwrite), >> (append), 2> (stderr), &> or >file 2>&1 (both), < (stdin from a file). > clobbers before the command runs — use >> to build a file up, and never sort file > file. set -o noclobber makes > refuse to overwrite (>| forces it). For both streams in one file the order is > f 2>&1, never 2>&1 > f.
A pipe wires LEFT stdout → RIGHT stdin and carries stdout ONLY; merge errors onto it with 2>&1 (placed before the |) if the next command needs them.
Pipes feed stdin, but rm/cp/wc want arguments: xargs turns a stream of items into arguments, and find -exec {} + does the same without a pipe. Use -print0 | xargs -0 (or -exec) when names may contain spaces.

Redirection & piping cheat-sheet
Syntax	Meaning
cmd > f	stdout → f (overwrite)
cmd >> f	stdout → f (append)
cmd 2> f	stderr → f
cmd &> f	stdout AND stderr → f
cmd < f	read stdin from f
A \| B	A's stdout → B's stdin
A 2>&1 \| B	A's stdout AND stderr → B's stdin
… \| xargs C	incoming lines become arguments to C
diff <(a) <(b)	process substitution: command output as a file

Streams & Redirection: stdin, stdout, stderr, and the Pipe

Every command has three wires

Redirection: aim the wires at files

What a pipe actually connects

Turning a list into commands: xargs and find -exec

Now do it — plumb some pipes