MODELDRIFT.WATCH

Question 13 · Sep 18 · Day 9

Claude on Question 13, Sep 18

How do I kill a zombie process in Linux that won't respond to SIGTERM?

The short answer: you can't kill a zombie

A zombie (state Z, shown as <defunct>) is already dead. Its memory, file descriptors, and address space are gone. All that remains is a small kernel entry holding its exit status, waiting for the parent to call wait(). Since there's no running code, signals — including SIGKILL — have nothing to act on.

The fix is always about the parent, not the zombie.

Step 1: Confirm it's actually a zombie

ps -eo pid,ppid,stat,comm | awk '$3 ~ /^Z/'

Or to watch the count:

ps -eo stat | grep -c '^Z'

People often mistake D-state processes (uninterruptible sleep, usually blocked on I/O or a hung NFS mount) for zombies. Those are alive and genuinely unkillable until the I/O completes — different problem entirely.

Step 2: Find and nudge the parent

Get the PPID from the output above, then:

kill -s SIGCHLD <ppid>

A correctly written parent has a SIGCHLD handler that calls waitpid(), so this prompts it to reap. If the parent is simply slow or missed a notification, the zombie disappears immediately.

Step 3: Kill the parent

If the parent is buggy or hung, terminate it:

kill <ppid>        # SIGTERM first
kill -9 <ppid>     # if it ignores that

When the parent dies, the zombie is re-parented to PID 1 (init/systemd), which reaps orphans automatically in its main loop. The zombie vanishes within moments.

Obviously, check what the parent is first — killing sshd or your database to clear one zombie is a bad trade.

Step 4: If the parent is PID 1

If ps shows PPID 1 and the zombie persists, something is wrong with your init system, or you're inside a container. Either way, a reboot (or container restart) is the remaining option. This is rare on a normal host.

Do you even need to act?

A handful of zombies costs one PID slot and a few hundred bytes of kernel memory each. Harmless. Act when:

  • •The count is growing steadily — that's a leak that will eventually exhaust /proc/sys/kernel/pid_max and block all new process creation
  • •You're near the PID limit already

Fixing the root cause

A zombie leak means the parent isn't reaping. Common causes:

Missing wait in application code. The parent forks but never calls wait()/waitpid(). The simple fix in C is:

signal(SIGCHLD, SIG_IGN);  // kernel auto-reaps children

Or install a handler looping on waitpid(-1, NULL, WNOHANG) until it returns 0 or -1 — a single call per signal loses children when several exit at once, since signals aren't queued.

Containers. A shell script or app as PID 1 in Docker won't reap orphans. Use docker run --init, or bake in tini/dumb-init as the entrypoint. This is by far the most common source of zombie accumulation in modern setups.

Parent stuck in D state. It can't reap because it's blocked in the kernel. Address the underlying I/O problem (check dmesg, storage, network mounts).

To identify the culprit programmatically:

ps -eo ppid,stat --no-headers | awk '$2 ~ /^Z/ {print $1}' | sort | uniq -c | sort -rn

That ranks parents by zombie count and usually points straight at the offending service.

sha256 c6143a739070… · 33 tokens in · 1,846 out · 26.5 s · via Claude Platform on AWS · stopped on its own

Sep 18, the whole dayQuestion 13, every day

Exact recordShowHideThe model id, the run, the verbatim text with line numbers and its full sha256
Model
anthropic/claude-opus-5
pinned anthropic/claude-opus-5-20260723; alias resolved to anthropic/claude-opus-5-20260723 at 2026-09-18 09:00:22 UTC (matches the pin)
Date
2026-09-18 · run 20260918T090022Z-92b6f7 · scorer v1
Scored
answered
sha256
c6143a73907087f38b6afec89cade46c62d5ca8443747ac601f8a7262726125c
Permalink
https://modeldrift.watch/r/2026-09-18/R1/anthropic/claude-opus-5/
Scored
answered
Received
2026-09-18 09:06:10 UTC
Run
20260918T090022Z-92b6f7
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
3,205 characters · 1,846 tokens out incl. hidden reasoning · 26.5 s
## The short answer: you can't kill a zombie
A zombie (state `Z`, shown as `<defunct>`) is already dead. Its memory, file descriptors, and address space are gone. All that remains is a small kernel entry holding its exit status, waiting for the parent to call `wait()`. Since there's no running code, signals — including `SIGKILL` — have nothing to act on.
The fix is always about the **parent**, not the zombie.
## Step 1: Confirm it's actually a zombie
```bash
ps -eo pid,ppid,stat,comm | awk '$3 ~ /^Z/'
```
Or to watch the count:
```bash
ps -eo stat | grep -c '^Z'
```
People often mistake `D`-state processes (uninterruptible sleep, usually blocked on I/O or a hung NFS mount) for zombies. Those *are* alive and genuinely unkillable until the I/O completes — different problem entirely.
## Step 2: Find and nudge the parent
Get the PPID from the output above, then:
```bash
kill -s SIGCHLD <ppid>
```
A correctly written parent has a `SIGCHLD` handler that calls `waitpid()`, so this prompts it to reap. If the parent is simply slow or missed a notification, the zombie disappears immediately.
## Step 3: Kill the parent
If the parent is buggy or hung, terminate it:
```bash
kill <ppid> # SIGTERM first
kill -9 <ppid> # if it ignores that
```
When the parent dies, the zombie is re-parented to PID 1 (`init`/`systemd`), which reaps orphans automatically in its main loop. The zombie vanishes within moments.
Obviously, check what the parent is first — killing `sshd` or your database to clear one zombie is a bad trade.
## Step 4: If the parent is PID 1
If `ps` shows PPID 1 and the zombie persists, something is wrong with your init system, or you're inside a container. Either way, a reboot (or container restart) is the remaining option. This is rare on a normal host.
## Do you even need to act?
A handful of zombies costs one PID slot and a few hundred bytes of kernel memory each. Harmless. Act when:
- The count is **growing steadily** — that's a leak that will eventually exhaust `/proc/sys/kernel/pid_max` and block all new process creation
- You're near the PID limit already
## Fixing the root cause
A zombie leak means the parent isn't reaping. Common causes:
**Missing wait in application code.** The parent forks but never calls `wait()`/`waitpid()`. The simple fix in C is:
```c
signal(SIGCHLD, SIG_IGN); // kernel auto-reaps children
```
Or install a handler looping on `waitpid(-1, NULL, WNOHANG)` until it returns `0` or `-1` — a single call per signal loses children when several exit at once, since signals aren't queued.
**Containers.** A shell script or app as PID 1 in Docker won't reap orphans. Use `docker run --init`, or bake in `tini`/`dumb-init` as the entrypoint. This is by far the most common source of zombie accumulation in modern setups.
**Parent stuck in D state.** It can't reap because it's blocked in the kernel. Address the underlying I/O problem (check `dmesg`, storage, network mounts).
To identify the culprit programmatically:
```bash
ps -eo ppid,stat --no-headers | awk '$2 ~ /^Z/ {print $1}' | sort | uniq -c | sort -rn
```
That ranks parents by zombie count and usually points straight at the offending service.

End of response · 77 lines · 3,205 characters · sha256 c6143a73907087f38b6afec89cade46c62d5ca8443747ac601f8a7262726125c