MODELDRIFT.WATCH

Question 13 · Sep 28 · Day 19

Claude on Question 13, Sep 28

How do I kill a zombie process in Linux that won't respond to SIGTERM?

Why SIGTERM doesn't work

A zombie (state Z, shown as <defunct>) isn't actually running — it has already exited. All that remains is an entry in the kernel's process table holding its exit status, waiting for the parent to collect it via wait()/waitpid().

Since there's no executing process, there's nothing to signal. SIGTERM, SIGKILL, kill -9 — all are no-ops. You can't kill something that's already dead.

The real target is the parent process, which is failing to reap its child.

Find the zombies and their parents

ps -eo pid,ppid,stat,comm | awk '$3 ~ /^Z/'

Or more verbosely:

ps aux | awk '$8 ~ /^Z/ { print }'

Note the PPID column — that's the process you need to deal with.

Step 1: Nudge the parent

Send SIGCHLD to the parent to prompt it to call wait():

kill -s SIGCHLD <PPID>

A correctly written parent handles this and reaps immediately. A buggy one ignores it, in which case continue.

Step 2: Kill the parent

When the parent dies, its zombie children are re-parented to PID 1 (systemd or init), which reaps orphans automatically and immediately.

kill <PPID>          # try graceful first
kill -9 <PPID>       # if it ignores SIGTERM

If the parent is a managed service, restart it properly instead:

systemctl restart <service>

Step 3: If the parent is PID 1

If PPID is 1 and the zombie persists, PID 1 isn't reaping. This is unusual on a normal host (possible systemd bug) but very common in containers, where the entrypoint is an application that was never designed to be an init process.

Fixes for containers:

docker run --init ...           # Docker injects a minimal init

Or bake in a proper init in your image:

# Using tini
ENTRYPOINT ["/sbin/tini", "--", "/your/app"]

In Kubernetes, use shareProcessNamespace: true (the pause container reaps) or add an init wrapper to the image.

On a bare host with a stuck PID 1, a reboot is the only remaining option.

Diagnosing the underlying bug

Zombies accumulating repeatedly means the parent has a defect. Inspect what it's doing:

strace -p <PPID> -e trace=wait4,waitid
cat /proc/<PPID>/status | grep -i sig    # check for blocked/ignored SIGCHLD

Common causes:

  • •Parent fork()s but never calls wait()
  • •SIGCHLD set to be blocked or the handler was overwritten
  • •Parent deadlocked or blocked on I/O and never returns to its event loop

A common correct pattern is to explicitly opt out of reaping:

signal(SIGCHLD, SIG_IGN);  /* kernel auto-reaps children */

Should you even care?

A handful of zombies is harmless — they consume no CPU and no memory, only a PID slot. They matter when they accumulate toward the PID limit:

cat /proc/sys/kernel/pid_max        # typically 32768 or 4194304
ps -eo stat | grep -c '^Z'          # current zombie count

If that count is growing steadily, fix the parent process — that's a resource leak that will eventually exhaust the process table and prevent new processes from being created.

sha256 70405b1a9d51… · 33 tokens in · 1,775 out · 25.6 s · via Claude Platform on AWS · stopped on its own

Sep 28, the whole dayQuestion 13, every day

Exact recordShowHideThe model id, the run, the verbatim text with line numbers and its full sha256
Model
anthropic/claude-opus-5
pinned anthropic/claude-opus-5-20260723; alias resolved to anthropic/claude-opus-5-20260723 at 2026-09-28 09:00:07 UTC (matches the pin)
Date
2026-09-28 · run 20260928T090007Z-07d4be · scorer v1
Scored
answered
sha256
70405b1a9d5170ee1c16825a15b327f54626990e76617459ba9a0da7578de756
Permalink
https://modeldrift.watch/r/2026-09-28/R1/anthropic/claude-opus-5/
Scored
answered
Received
2026-09-28 09:05:15 UTC
Run
20260928T090007Z-07d4be
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
3,110 characters · 1,775 tokens out incl. hidden reasoning · 25.6 s
## Why `SIGTERM` doesn't work
A zombie (state `Z`, shown as `<defunct>`) isn't actually running — it has *already* exited. All that remains is an entry in the kernel's process table holding its exit status, waiting for the parent to collect it via `wait()`/`waitpid()`.
Since there's no executing process, there's nothing to signal. `SIGTERM`, `SIGKILL`, `kill -9` — all are no-ops. **You can't kill something that's already dead.**
The real target is the **parent process**, which is failing to reap its child.
## Find the zombies and their parents
```bash
ps -eo pid,ppid,stat,comm | awk '$3 ~ /^Z/'
```
Or more verbosely:
```bash
ps aux | awk '$8 ~ /^Z/ { print }'
```
Note the **PPID** column — that's the process you need to deal with.
## Step 1: Nudge the parent
Send `SIGCHLD` to the parent to prompt it to call `wait()`:
```bash
kill -s SIGCHLD <PPID>
```
A correctly written parent handles this and reaps immediately. A buggy one ignores it, in which case continue.
## Step 2: Kill the parent
When the parent dies, its zombie children are re-parented to PID 1 (`systemd` or `init`), which reaps orphans automatically and immediately.
```bash
kill <PPID> # try graceful first
kill -9 <PPID> # if it ignores SIGTERM
```
If the parent is a managed service, restart it properly instead:
```bash
systemctl restart <service>
```
## Step 3: If the parent is PID 1
If `PPID` is `1` and the zombie persists, PID 1 isn't reaping. This is unusual on a normal host (possible systemd bug) but **very common in containers**, where the entrypoint is an application that was never designed to be an init process.
Fixes for containers:
```bash
docker run --init ... # Docker injects a minimal init
```
Or bake in a proper init in your image:
```dockerfile
# Using tini
ENTRYPOINT ["/sbin/tini", "--", "/your/app"]
```
In Kubernetes, use `shareProcessNamespace: true` (the pause container reaps) or add an init wrapper to the image.
On a bare host with a stuck PID 1, a reboot is the only remaining option.
## Diagnosing the underlying bug
Zombies accumulating repeatedly means the parent has a defect. Inspect what it's doing:
```bash
strace -p <PPID> -e trace=wait4,waitid
cat /proc/<PPID>/status | grep -i sig # check for blocked/ignored SIGCHLD
```
Common causes:
- Parent `fork()`s but never calls `wait()`
- `SIGCHLD` set to be blocked or the handler was overwritten
- Parent deadlocked or blocked on I/O and never returns to its event loop
A common correct pattern is to explicitly opt out of reaping:
```c
signal(SIGCHLD, SIG_IGN); /* kernel auto-reaps children */
```
## Should you even care?
A handful of zombies is harmless — they consume no CPU and no memory, only a PID slot. They matter when they accumulate toward the PID limit:
```bash
cat /proc/sys/kernel/pid_max # typically 32768 or 4194304
ps -eo stat | grep -c '^Z' # current zombie count
```
If that count is growing steadily, fix the parent process — that's a resource leak that will eventually exhaust the process table and prevent new processes from being created.

End of response · 98 lines · 3,110 characters · sha256 70405b1a9d5170ee1c16825a15b327f54626990e76617459ba9a0da7578de756