Laravel Queue Workers Crashing on Production: How a Mis‑configured MySQL Pool Caused 5‑Minute Downtime and How I Fixed It in Minutes
If you’ve ever stared at a blank monitoring screen while your Laravel queue workers silently die, you know the gut‑punch of a production‑level outage. No alerts, no obvious errors—just a sudden drop in API speed and an angry support ticket. In my case, the culprit was a tiny MySQL connection pool setting that blew up the entire VPS for five minutes. The good news? The fix took less than ten minutes and saved us thousands in lost revenue.
Why This Matters
Queue workers are the heart of any Laravel‑powered SaaS, handling emails, notifications, and background data crunching. When they crash, you lose:
- Customer trust
- API latency guarantees
- Revenue from paid webhook calls
- Team sanity
Understanding the low‑level connection between PHP‑FPM, MySQL, and your VPS is the difference between a one‑hour outage and a smooth, autoscaled deployment.
Common Causes of Queue Crashes
- Exhausted MySQL connections
- Supervisor process limits
- PHP‑FPM pool misconfiguration
- Redis connection timeouts
- Insufficient RAM on low‑tier VPS
- Incompatible Composer autoload files after deployment
Step‑By‑Step Fix Tutorial
1. Verify the MySQL Connection Pool
Open your MySQL admin console and run:
SHOW VARIABLES LIKE 'max_connections';
If the result is 151 (the default) and you have 8 queue workers each spawning 5 processes, you’ll easily exceed the limit.
2. Adjust Laravel's DB Configuration
Edit config/database.php and add a reasonable options array:
'mysql' => [
'driver' => 'mysql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '3306'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'prefix' => '',
'strict' => true,
'engine' => null,
'options' => [
PDO::ATTR_PERSISTENT => true,
PDO::MYSQL_ATTR_MAX_BUFFER_SIZE => 1024 * 1024,
],
],
max_connections limit.
3. Raise MySQL Connection Limits
On the VPS, edit /etc/mysql/mysql.conf.d/mysqld.cnf (or /etc/my.cnf on CentOS) and add:
[mysqld]
max_connections = 500
wait_timeout = 180
interactive_timeout = 180
Then restart MySQL:
sudo systemctl restart mysql
4. Tune Supervisor (Queue) Settings
Open your Supervisor config, usually at /etc/supervisor/conf.d/laravel-queue.conf:
[program:laravel-queue]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work redis --sleep=3 --tries=3
autostart=true
autorestart=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/var/log/laravel/queue.log
stopwaitsecs=3600
Set numprocs to match the total connections you expect (workers + web processes). After editing, reload Supervisor:
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl restart laravel-queue:*
5. Verify PHP‑FPM Pool Size
In /etc/php/8.2/fpm/pool.d/www.conf adjust:
pm = dynamic
pm.max_children = 70
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
Restart PHP‑FPM:
sudo systemctl restart php8.2-fpm
VPS or Shared Hosting Optimization Tips
- Always enable swap on low‑memory VPS (e.g.,
sudo fallocate -l 2G /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile) - Prefer Ubuntu 22.04 LTS for its newer kernel and MySQL 8.0 support
- If on shared hosting, limit workers to 2‑3 processes and use
queue:listeninstead ofqueue:work - Use Cloudflare’s Rate Limiting to protect API endpoints while workers restart
- Monitor
htopandmysqladmin processlistduring spikes
Real World Production Example
Our SaaS runs on a 2 vCPU, 4 GB RAM DigitalOcean droplet. Prior to the fix:
- 8 queue workers, each spawning 5 child processes
- MySQL max_connections = 151
- PHP‑FPM max_children = 30
The moment a batch of 500 emails hit the queue, MySQL rejected new connections, causing Laravel workers to abort with SQLSTATE[HY000] [2002] Connection timed out. The result: a 5‑minute API outage, 1,200 failed webhook calls, and a $4,500 SLA penalty.
Before vs After Results
| Metric | Before Fix | After Fix |
|---|---|---|
| Max DB Connections Used | 170 (overflow) | 420 (within 500 limit) |
| Queue Crash Rate | 3‑4 per hour | 0 |
| Average Job Latency | 12 s | 3 s |
Security Considerations
Increasing max_connections can expose MySQL to brute‑force attacks. Apply these mitigations:
- Enable
mysql_native_passwordwith strong passwords - Configure
ufw allow from 10.0.0.0/8 to any port 3306if only internal services need DB access - Set
wait_timeoutlow (180) to close idle connections quickly - Use Cloudflare WAF to block SQL injection attempts before they hit your server
Bonus Performance Tips
- Cache frequently used lookup tables in Redis with
Cache::remember() - Run
composer install --optimize-autoloader --no-devduring deployment - Enable
opcache.enable_cli=1for Artisan commands - Use NGINX with
fastcgi_buffers 16 16k;andkeepalive_timeout 65; - Set
process_control_timeout = 3600in Supervisor for long‑running jobs
FAQ
Q: I can’t edit MySQL config on shared hosting. What now?
A: Reducenumprocsin Supervisor to match the host’s max connections, and switch toqueue:listenwhich reuses a single connection per worker.
Q: Does enabling persistent PDO connections increase memory usage?
A: Slightly, but the trade‑off is worth it for high‑throughput queues. Monitorps aux | grep php-fpmafter the change.
Final Thoughts
Queue crashes are rarely a code problem; they’re a resource‑allocation problem. By aligning MySQL limits, PHP‑FPM pool sizes, and Supervisor process counts, you turn a chaotic 5‑minute outage into a stable, scalable system.
Take the time to profile your production environment, document every limit, and automate the configuration with Ansible or Laravel Envoyer. The payoff is fewer alerts, happier customers, and a healthier bottom line.
Looking for cheap, secure VPS hosting that’s ready for Laravel, WordPress, and Redis? Check out Hostinger’s plans now – performance‑focused, 24/7 support, and a 30‑day money‑back guarantee.
No comments:
Post a Comment