Bali, Indonesia
Insights

How to Check If a Stuck Mail Queue Ran Your Server Out of Inodes

October 23, 2026
0:00
Mail stops moving, the MTA log fills with 'No space left on device', and the first command anyone runs is df -h. It shows plenty of free space, so the error looks wrong. Usually it is not. A Linux filesystem can run out of inodes long before it runs out of bytes, and a mail queue packed with small stuck messages is one of the most common ways to get there. This piece walks through that failure the way a post-mortem would: what breaks first, how the investigation usually goes, where the inodes turn out to be, how to get mail flowing again without throwing away real messages, and what to change so it does not come back.
An inode is the record a filesystem keeps for each file or directory: owner, permissions, timestamps, and where the data lives. On ext4 the total number of inodes is fixed when the filesystem is created. Every file uses one, no matter how small. A 500-byte file and a 5 GB file cost the same single inode. When the last inode is gone, the kernel returns the same error it uses for a full disk, ENOSPC, which programs print as 'No space left on device'. That shared error message is why the problem gets misread so often. For a mail server the impact shows up in a few places at once. Postfix or Exim cannot create new queue files, so incoming SMTP connections typically get a temporary 4xx reply instead of being accepted. Sending servers treat that as 'try again later', so legitimate inbound mail is usually delayed rather than lost, at least for a while. Local scripts that hand mail to sendmail fail outright. And because the filesystem is shared, other things break too: PHP cannot write session files, cron jobs fail, log rotation stops. The first ticket that comes in is often 'I can't log in to my site', not 'email is down', which sends the investigation in the wrong direction.
The turning point in this kind of incident is running df -i next to df -h. The first shows bytes, the second shows inodes. If IUse% sits at or near 100% while the byte usage looks normal, the cause is confirmed and the question changes from 'what is filling the disk' to 'what created all these files'. These commands are read-only and safe to run on a live server, even one that is already out of inodes:
BASH
# Bytes vs inodes on the filesystem that holds the mail spool
df -h /var/spool
df -i /var/spool

# Which directories hold the most files (GNU du; -x stays on one filesystem)
du --inodes -x -d 2 /var/spool 2>/dev/null | sort -n | tail -n 10

# Queue size
postqueue -p | tail -n 1    # Postfix: last line shows the request count
exim -bpc                   # Exim: number of messages in the queue

# Postfix: which destination domains dominate the deferred queue
qshape deferred | head -n 15
Illustrative output format for an exhausted filesystem, with round numbers:
TEXT
Filesystem      Inodes   IUsed IFree IUse% Mounted on
/dev/vda1      1000000 1000000     0  100% /
If /var/spool is not on its own partition, the inode count you see is for the root filesystem, and the queue is competing with everything else on the server. In that case run the du --inodes line against / as well, keeping -x so it does not wander into other mounts. On a large filesystem that scan can take a while.
The reason a stuck queue eats inodes so efficiently is that each message is more than one file. Postfix keeps its queue under /var/spool/postfix, split into directories such as incoming, active, deferred and hold. A deferred message has its queue file in deferred/ and a delivery status record in defer/, so it typically costs at least two inodes. Exim stores each message as a header file ending in -H and a data file ending in -D under input/, plus a per-message log in msglog/. That is roughly three inodes per message. The spool lives at /var/spool/exim on cPanel and RHEL-family systems and /var/spool/exim4 on Debian and Ubuntu. So when du --inodes points at deferred and defer for Postfix, or input and msglog for Exim, and the queue count from postqueue -p or exim -bpc is far larger than normal for that server, the link is made. The next question is who put the mail there. qshape deferred groups the Postfix queue by destination domain and age, which shows whether one destination is refusing everything or the whole queue is stuck. For Exim, reading a sample of the queue with exim -bp and looking at senders and recipients does the same job. The patterns that tend to cause this are fairly consistent:
  • A compromised website script or mailbox account sending spam, most of which is deferred or bounced.
  • A cron job whose output is mailed to an address that cannot be delivered, generating a message on every run.
  • Bounces generated for spam with forged sender addresses, which then cannot be delivered either. In Exim these often end up frozen.
Before blaming the queue, confirm it really is the largest consumer. Sometimes du points somewhere else entirely, like a PHP session directory or an application cache, and the mail errors are just a side effect.
The order matters here. Stop the source first, then clean up. If a compromised script is still running, any inodes you free will be used again within minutes. Disable the script or account, fix the cron MAILTO, or block the sending user before touching the queue. Avoid two tempting shortcuts. Deleting everything with postsuper -d ALL or wiping the Exim spool by hand removes legitimate mail along with the junk, and there is no undo. Flushing the queue makes things worse when the backlog is made of rate-limit deferrals, a trade-off covered in Should You Run postqueue -f or Let Postfix Retry Deferred Mail on Its Own?. Instead, remove only the messages you have identified as junk. Always run the listing half of each pipeline on its own first and read what it matches. The deletions below are permanent and do not send a bounce to the sender.
BASH
# Postfix: hold a message for review (reversible with postsuper -H)
postsuper -h QUEUE_ID

# Postfix: delete queued mail from one sender ($7 is the sender field in postqueue -p)
postqueue -p | tail -n +2 | awk 'BEGIN { RS = "" } 7 == "[email protected]" { print1 }' | tr -d '*!' | postsuper -d -

# Exim: list message IDs from one sender, review, then remove
exiqgrep -i -f '[email protected]'
exiqgrep -i -f '[email protected]' | xargs exim -Mrm

# Exim: remove frozen messages, which are often undeliverable bounces
exiqgrep -z -i | xargs exim -Mrm
If the filesystem is at exactly zero free inodes, even some admin tools may fail because they need temporary files. Clearing the most obvious junk first, such as frozen bounces, usually frees enough room to work normally. Re-run df -i as you go. Once inodes are free, the MTA starts accepting mail again, and senders that got a temporary error earlier will retry on their own schedule.
The long-term fix is less about the queue and more about visibility and limits. Monitor inodes, not just bytes. Most disk alerts only watch df -h style percentages, which is exactly why this failure surprises people. Adding an alert on IUse% turns a full outage into an early warning. Give stuck mail an expiry date. Postfix uses maximal_queue_lifetime and bounce_queue_lifetime to decide how long deferred mail and bounces stay before being returned or dropped. Exim has timeout_frozen_after and ignore_bounce_errors_after for frozen messages and failed bounces. Check what your config actually sets with postconf or by reading the Exim config, because panels and distros often change the defaults. Let Exim refuse mail before the disk is truly full. The check_spool_space and check_spool_inodes options make Exim reject incoming messages with a temporary error when free space or free inodes drop below a threshold, which leaves headroom for the rest of the system. Postfix has a byte-based queue_minfree setting but no inode equivalent, so on Postfix systems monitoring carries more of the load. Limit outbound volume per account so one compromised mailbox or script cannot fill the queue on its own, and make sure system cron output goes to a mailbox someone reads. Finally, consider the filesystem. On ext4 the inode count is set at creation time, so a mail-heavy server built with defaults meant for large files can hit the limit early. XFS allocates inodes dynamically, which makes this specific failure less likely. Putting /var/spool on its own partition keeps a runaway queue from taking down PHP sessions and logs, though it also gives the spool its own, smaller inode budget to watch. The practical rule: whenever you see 'No space left on device' and df -h looks fine, run df -i before anything else. If the inodes are gone and the spool is the reason, stop whatever is generating the mail, delete only what you have actually reviewed, and then put an inode alert in place so the next backlog shows up as a warning instead of an outage.