MODELDRIFT.WATCH

Question 13 · Sep 12 · Day 3

Claude on Question 13, Sep 12

How do I kill a zombie process in Linux that won't respond to SIGTERM?

The short answer: you can't

A zombie (state Z, often shown as <defunct>) is a process that has already terminated. All its memory, file descriptors, and resources are gone. What remains is a single entry in the kernel's process table holding its exit status, waiting for the parent to collect it via wait()/waitpid().

Signals require code to execute. A zombie has no code, no stack, no threads. SIGTERM, SIGKILL, SIGSEGV — all are silently discarded. This is expected behavior, not a stuck process.

Find the parent

The real target is the parent that isn't reaping its children:

ps -eo pid,ppid,stat,comm | awk '$3 ~ /^Z/'

Or a broader view:

ps axo pid,ppid,stat,wchan:20,comm | grep -w Z

Note the PPID column — that's the process actually responsible.

Nudge the parent

First, try prompting the parent to reap:

kill -s SIGCHLD <ppid>

Many well-behaved programs have a SIGCHLD handler that calls waitpid(). If the parent was simply asleep or missed the notification, this clears the zombie immediately.

Kill the parent

If the parent is buggy — never calls wait(), or installed SIG_IGN on SIGCHLD incorrectly — terminate it:

kill <ppid>          # SIGTERM, graceful
kill -9 <ppid>       # if it ignores SIGTERM

When the parent dies, the zombie is re-parented to PID 1 (systemd or init), which reaps orphans automatically in its main loop. The zombie disappears within moments.

⚠️ Check what the parent is before killing it. If the PPID is 1, something is wrong with your init system. If it's a database, web server, or container runtime, killing it has real consequences — restart it properly through systemctl instead.

Should you even bother?

A handful of zombies is harmless. Each consumes one PID slot and a few hundred bytes of kernel memory — no CPU, no RAM, no file handles.

They matter when they accumulate. A leaking parent can exhaust the PID space:

cat /proc/sys/kernel/pid_max     # usually 4194304 on modern kernels
ps -eo stat | grep -c Z          # current zombie count

Hundreds or thousands means a genuine bug worth chasing.

Common root causes

Missing reaper in containers. A process running as PID 1 inside a container must reap orphans, but most application runtimes don't. Fix with docker run --init, or add tini/dumb-init as the entrypoint. In Kubernetes, set shareProcessNamespace: true or ensure your image has a proper init.

Forgotten waitpid(). Classic C bug — the parent forks but never collects. The idiomatic fixes:

signal(SIGCHLD, SIG_IGN);   // POSIX: auto-reap, discard exit status

or a handler that loops:

while (waitpid(-1, NULL, WNOHANG) > 0);

Blocked parent. A parent stuck in an uninterruptible syscall (state D) can't run its handler. Check /proc/<ppid>/stack and /proc/<ppid>/wchan — often an NFS or storage hang, which is the actual problem to solve.

Shell scripts spawning background jobs. A script using & without wait will collect zombies until it exits.

If a "zombie" won't clear after the parent dies

That's unusual and suggests it isn't really a zombie. Check the actual state:

cat /proc/<pid>/status | grep -E 'State|PPid'

State D (uninterruptible sleep) is the far more common culprit behind processes that truly ignore SIGKILL — those are blocked in kernel space, typically on I/O, and only clear when the underlying operation completes or times out. That requires a different fix: address the storage, network mount, or driver that's hanging.

sha256 470a4bf8c8d7… · 33 tokens in · 1,869 out · 27.3 s · via Claude Platform on AWS · stopped on its own

Sep 12, the whole dayQuestion 13, every day

Exact recordShowHideThe model id, the run, the verbatim text with line numbers and its full sha256
Model
anthropic/claude-opus-5
pinned anthropic/claude-opus-5-20260723; alias resolved to anthropic/claude-opus-5-20260723 at 2026-09-12 09:00:20 UTC (matches the pin)
Date
2026-09-12 · run 20260912T090021Z-c0c86e · scorer v1
Scored
answered
sha256
470a4bf8c8d7fd90d58d2065bf140972de6d9a7f022820a7eaf6b1d97fb5f66f
Permalink
https://modeldrift.watch/r/2026-09-12/R1/anthropic/claude-opus-5/
Scored
answered
Received
2026-09-12 09:05:35 UTC
Run
20260912T090021Z-c0c86e
Served by
Claude Platform on AWS · HTTP 200 · finish stop
Size
3,616 characters · 1,869 tokens out incl. hidden reasoning · 27.3 s
## The short answer: you can't
A zombie (state `Z`, often shown as `<defunct>`) is a process that has **already terminated**. All its memory, file descriptors, and resources are gone. What remains is a single entry in the kernel's process table holding its exit status, waiting for the parent to collect it via `wait()`/`waitpid()`.
Signals require code to execute. A zombie has no code, no stack, no threads. `SIGTERM`, `SIGKILL`, `SIGSEGV` — all are silently discarded. This is expected behavior, not a stuck process.
## Find the parent
The real target is the parent that isn't reaping its children:
```bash
ps -eo pid,ppid,stat,comm | awk '$3 ~ /^Z/'
```
Or a broader view:
```bash
ps axo pid,ppid,stat,wchan:20,comm | grep -w Z
```
Note the `PPID` column — that's the process actually responsible.
## Nudge the parent
First, try prompting the parent to reap:
```bash
kill -s SIGCHLD <ppid>
```
Many well-behaved programs have a `SIGCHLD` handler that calls `waitpid()`. If the parent was simply asleep or missed the notification, this clears the zombie immediately.
## Kill the parent
If the parent is buggy — never calls `wait()`, or installed `SIG_IGN` on `SIGCHLD` incorrectly — terminate it:
```bash
kill <ppid> # SIGTERM, graceful
kill -9 <ppid> # if it ignores SIGTERM
```
When the parent dies, the zombie is **re-parented to PID 1** (systemd or init), which reaps orphans automatically in its main loop. The zombie disappears within moments.
⚠️ Check what the parent *is* before killing it. If the PPID is `1`, something is wrong with your init system. If it's a database, web server, or container runtime, killing it has real consequences — restart it properly through `systemctl` instead.
## Should you even bother?
A handful of zombies is harmless. Each consumes one PID slot and a few hundred bytes of kernel memory — no CPU, no RAM, no file handles.
They matter when they **accumulate**. A leaking parent can exhaust the PID space:
```bash
cat /proc/sys/kernel/pid_max # usually 4194304 on modern kernels
ps -eo stat | grep -c Z # current zombie count
```
Hundreds or thousands means a genuine bug worth chasing.
## Common root causes
**Missing reaper in containers.** A process running as PID 1 inside a container must reap orphans, but most application runtimes don't. Fix with `docker run --init`, or add `tini`/`dumb-init` as the entrypoint. In Kubernetes, set `shareProcessNamespace: true` or ensure your image has a proper init.
**Forgotten `waitpid()`.** Classic C bug — the parent forks but never collects. The idiomatic fixes:
```c
signal(SIGCHLD, SIG_IGN); // POSIX: auto-reap, discard exit status
```
or a handler that loops:
```c
while (waitpid(-1, NULL, WNOHANG) > 0);
```
**Blocked parent.** A parent stuck in an uninterruptible syscall (state `D`) can't run its handler. Check `/proc/<ppid>/stack` and `/proc/<ppid>/wchan` — often an NFS or storage hang, which is the actual problem to solve.
**Shell scripts spawning background jobs.** A script using `&` without `wait` will collect zombies until it exits.
## If a "zombie" won't clear after the parent dies
That's unusual and suggests it isn't really a zombie. Check the actual state:
```bash
cat /proc/<pid>/status | grep -E 'State|PPid'
```
State `D` (uninterruptible sleep) is the far more common culprit behind processes that truly ignore `SIGKILL` — those are blocked in kernel space, typically on I/O, and only clear when the underlying operation completes or times out. That requires a different fix: address the storage, network mount, or driver that's hanging.

End of response · 87 lines · 3,616 characters · sha256 470a4bf8c8d7fd90d58d2065bf140972de6d9a7f022820a7eaf6b1d97fb5f66f