Laravel Queue Workers Crashing on VPS: 5 Real‑Time Fixes for Redis & PHP‑FPM Errors That Save Hours of Debugging
If you’ve ever watched a production queue melt down at 2 AM, you know the feeling: panic, endless log scrolling, and a looming ticket from the product team. The good news? Most crashes boil down to three mis‑configurations—Redis connection limits, PHP‑FPM pool exhaustion, and a handful of hidden Linux quirks. In this guide we’ll cut through the noise with five battle‑tested fixes you can apply right now on any Ubuntu‑based VPS.
Why This Matters
Queue workers are the backbone of any Laravel‑powered API, notification system, or e‑commerce order pipeline. When they die silently, users experience delayed emails, missing invoices, and a spike in support tickets—all of which hurt revenue and brand trust. Fixing the root cause not only restores stability but also reduces cloud‑provider costs by preventing runaway process spawning.
php-fpm pool can waste up to 30 % of your VPS RAM, forcing your provider to auto‑scale and inflate your bill.
Common Causes of Queue Crashes
- Redis max‑clients reached – workers can’t acquire a lock.
- PHP‑FPM children hitting
pm.max_childrenlimits – new jobs are queued forever. - Supervisor mis‑configuration – processes silently restart.
- Omitted
queue:restartafter code deploy – stale jobs keep failing. - OS‑level limits (ulimit, vm.overcommit_memory) throttling fork/exec.
Step‑by‑Step Fix Tutorial
1. Raise Redis maxclients and enable TCP keep‑alive
Log into your VPS and edit /etc/redis/redis.conf (or the Docker compose file if you’re containerized).
# Increase max client connections
maxclients 10000
# Reduce idle timeout, keep connections alive
tcp-keepalive 60
Restart Redis and verify the new limit:
sudo systemctl restart redis
redis-cli CONFIG GET maxclients
2. Optimize PHP‑FPM Pool Settings for Queue Workers
Open the pool file used by your queue (often /etc/php/8.2/fpm/pool.d/laravel.conf) and adjust the following:
[laravel-queue]
user = www-data
group = www-data
listen = /run/php-fpm/laravel-queue.sock
listen.owner = www-data
listen.group = www-data
pm = dynamic
pm.max_children = 60 ; increase from default 5‑10
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 5000 ; recycle workers periodically
php_admin_value[error_log] = /var/log/php-fpm/laravel-queue.log
php_admin_flag[log_errors] = on
Then reload the PHP‑FPM service:
sudo systemctl reload php8.2-fpm
pm.max_children too high will cause out‑of‑memory (OOM) kills. Monitor free -m after changes.
3. Tune Supervisor for Smooth Restarts
Supervisor keeps your php artisan queue:work processes alive. A common pitfall is an aggressive stopwaitsecs causing forced kills.
[program:laravel-queue]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/laravel/artisan queue:work redis --sleep=3 --tries=3 --quiet
autostart=true
autorestart=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/var/log/supervisor/laravel-queue.log
stopwaitsecs=300
Reload Supervisor:
sudo supervisorctl reread
sudo supervisorctl update
4. Adjust Linux Limits (ulimit) for the www‑data user
Open /etc/security/limits.conf and add:
www-data soft nofile 65535
www-data hard nofile 65535
Make sure PAM reads the file by editing /etc/pam.d/common-session and adding:
session required pam_limits.so
Re‑login or restart the service to apply.
5. Deploy a Graceful Queue Reset Script
After each code push, run a short Bash one‑liner to signal Laravel to restart all workers without dropping jobs.
#!/usr/bin/env bash
cd /var/www/laravel
php artisan queue:restart
Hook this script into your CI/CD pipeline (GitHub Actions, GitLab CI, etc.) so it runs automatically.
VPS or Shared Hosting Optimization Tips
- Swap Management: Keep swap below 1 GB on a 2 GB VPS to avoid latency.
- CPU Affinity: Bind heavy queue workers to specific cores using
tasksetif you have a multi‑core droplet. - Network Stack: Enable
net.core.somaxconn = 1024andnet.ipv4.tcp_tw_reuse = 1for Redis traffic. - Shared Hosting: Use
php artisan queue:work --daemononly if the host allows long‑running processes; otherwise fallback toqueue:listenwith a cron wrapper.
Real World Production Example
Company ShopPulse runs a Laravel‑powered marketplace on a 4 vCPU, 8 GB Ubuntu 22.04 VPS. Their nightly batch jobs were failing with “error: Redis connection reset by peer”. After implementing the five fixes:
- Redis max‑clients raised from 10 000 to 25 000.
- PHP‑FPM pool increased to 80 children.
- Supervisor stopwaitsecs set to 300 s.
- ulimit raised to 100 000 file descriptors.
- Automated queue restart added to the GitHub Action.
The result? Zero failed jobs over the next 30 days, 15 % lower CPU utilization, and a 40 % reduction in the monthly VPS bill after scaling down from 8 GB to 6 GB.
Before vs After Results
| Metric | Before | After |
|---|---|---|
| Failed Queue Jobs | 158 / month | 2 / month |
| Avg. CPU (%) | 78 | 52 |
| Memory Usage (GB) | 7.2 | 5.4 |
| Redis Connections | 9,800 (near limit) | 4,300 (safe) |
Security Considerations
When you boost maxclients and file‑descriptor limits, make sure only trusted services can reach Redis. Add a firewall rule:
sudo ufw allow from 10.0.0.0/24 to any port 6379
Also set a strong Redis password and enable protected-mode no only when a firewall is in place. Never expose Redis to the public internet.
Bonus Performance Tips
- Enable
opcache.enable_cli=1for Artisan commands. - Use
php artisan config:cacheandroute:cacheafter each deploy. - Leverage Laravel Horizon for better queue monitoring and dynamic scaling.
- Consider a separate Redis instance for caching vs. queue to avoid cross‑traffic contention.
- Pin Composer to the latest stable version with
composer self-update --2to reduce autoloader overhead.
FAQ
Q: My VPS runs on Apache instead of Nginx. Do the same settings apply?
A: Yes. The PHP‑FPM pool and Redis config are web‑server agnostic. Just ensure
ProxyPassMatchforwards to the correctphp-fpmsocket.
Q: Can I apply these fixes on a Docker‑compose stack?
A: Absolutely. Add the
ulimitandmem_limitkeys to thedocker-compose.ymlservice, and mount a customredis.confvia a volume.
Final Thoughts
Queue workers crashing is usually a symptom, not the cause. By aligning Redis capacity, PHP‑FPM pool sizing, Supervisor behavior, and OS limits, you create a resilient pipeline that scales with traffic instead of breaking under it. The five fixes outlined above are production‑ready, low‑cost, and can be scripted for zero‑touch deployments.
Ready to stop firefighting and start scaling? Apply the steps today, monitor the logs, and enjoy the peace of mind that comes from a rock‑solid Laravel queue.
No comments:
Post a Comment