Laravel Queue Workers Failing on VPS: 7 Fatal MySQL Timeout Errors That Fell Silent Until Nightly Sync Crash
If you’ve ever watched a production queue die at 2 am while the nightly sync runs, you know the gut‑punch of a silent MySQL timeout. One missed job can cascade into a full‑blown cascade of failed Laravel workers, missed emails, and angry users. The good news? This article shows exactly why those seven timeout errors go unnoticed and, most importantly, how to squash them forever on a VPS.
Why This Matters
In a SaaS‑oriented Laravel + WordPress stack, queue workers are the heart of asynchronous processing—emails, PDF generation, webhook dispatches, and API throttling all rely on them. When MySQL silently drops connections after wait_timeout expires, the php artisan queue:work process hangs, the supervisor restarts it, and you end up with a “failed job” record you never saw in the logs. These hidden failures waste CPU, inflate your bill on a VPS, and can break compliance‑critical notifications.
wait_timeout mis‑config on a 4‑core Ubuntu 22.04 VPS can add up to 30 % extra CPU usage during peak sync windows.
Common Causes of Silent MySQL Timeouts
- Default MySQL
wait_timeout(8 hours) clashes withidle_timeoutof PHP‑FPM or Supervisor. - Connection pooling with Laravel’s
database.php“persistent” flag left disabled. - Improper
max_execution_timeinphp.inicausing workers to be killed before MySQL can respond. - Low
innodb_buffer_pool_sizeon a low‑tier VPS causing lock‑waits that look like timeouts. - Network latency between the VPS and a managed MySQL instance (e.g., DigitalOcean Managed MySQL).
- Supervisor’s
stopwaitsecstoo low for long‑running jobs. - Redis queue driver falling back to database when Redis memory is exhausted.
Step‑by‑Step Fix Tutorial
1. Tune MySQL Server Settings
# /etc/mysql/mysql.conf.d/mysqld.cnf
[mysqld]
wait_timeout = 28800 # 8 hours (increase if you have very long jobs)
interactive_timeout = 28800
max_allowed_packet = 64M
innodb_buffer_pool_size = 2G # adjust to 70% of VPS RAM
After editing, restart MySQL:
sudo systemctl restart mysql
2. Enable Persistent PDO Connections in Laravel
// config/database.php
'connections' => [
'mysql' => [
'driver' => 'mysql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '3306'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'unix_socket' => env('DB_SOCKET', ''),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'prefix' => '',
'strict' => true,
'engine' => null,
'options' => extension_loaded('pdo_mysql') ? [
PDO::ATTR_PERSISTENT => true,
] : [],
],
],
3. Adjust PHP‑FPM Pools
# /etc/php/8.2/fpm/pool.d/www.conf
request_terminate_timeout = 300 ; 5 minutes max per request
php_admin_value[error_log] = /var/log/php-fpm/www-error.log
pm.max_children = 25
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 10
Restart PHP‑FPM:
sudo systemctl restart php8.2-fpm
4. Refine Supervisor Configuration
# /etc/supervisor/conf.d/laravel-queue.conf
[program:laravel-queue]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work redis --sleep=3 --tries=3 --timeout=300
autostart=true
autorestart=true
user=www-data
numprocs=4
redirect_stderr=true
stdout_logfile=/var/log/laravel/queue.log
stopwaitsecs=360 ; allow 6 minutes for graceful shutdown
Apply the new config:
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl restart laravel-queue:*
5. Deploy Redis as Primary Queue Driver
# /etc/redis/redis.conf
maxmemory 256mb
maxmemory-policy allkeys-lru
appendonly yes
Restart Redis and set the driver:
sudo systemctl restart redis
// .env
QUEUE_CONNECTION=redis
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379
QUEUE_RETRY_AFTER=300 in .env to match your --timeout value.
6. Harden Nginx Front‑End
# /etc/nginx/sites-available/laravel.conf
server {
listen 80;
server_name example.com www.example.com;
root /var/www/html/public;
add_header X-Frame-Options "SAMEORIGIN";
add_header X-Content-Type-Options "nosniff";
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location ~ \.php$ {
include fastcgi_params;
fastcgi_pass unix:/run/php/php8.2-fpm.sock;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_read_timeout 300;
}
location ~* \.(js|css|png|jpg|jpeg|svg|gif)$ {
expires 30d;
add_header Cache-Control "public, immutable";
}
}
Reload Nginx:
sudo nginx -t && sudo systemctl reload nginx
7. Composer Autoloader Optimization
composer install --optimize-autoloader --no-dev
php artisan config:cache
php artisan route:cache
php artisan view:cache
php artisan config:cache after changing .env variables will require a cache clear. Use php artisan config:clear first.
VPS or Shared Hosting Optimization Tips
- Allocate at least 2 GB RAM for MySQL on a 4 GB VPS; shared hosting often caps memory at 512 MB, causing frequent lock‑waits.
- Use
UFWto limit MySQL port exposure:sudo ufw allow from 127.0.0.1 to any port 3306. - On shared hosts, switch to the
databasequeue driver and enablequeue:listeninstead ofqueue:workto respect execution limits. - Enable Cloudflare “Auto Minify” for JS/CSS to reduce bandwidth for queue payloads that contain HTML snippets.
- Set
opcache.memory_consumption=128andopcache.max_accelerated_files=10000inphp.inifor faster job bootstrapping.
Real World Production Example
Acme SaaS runs 12 Laravel micro‑services behind a single Nginx reverse proxy on a 8‑core, 16 GB Ubuntu VPS. After implementing the steps above, the nightly data sync dropped from 45 minutes to 12 minutes, and queue failure rate fell from 4.3 % to <0.1 %.
Before vs After Metrics
| Metric | Before | After |
|---|---|---|
| Avg. Queue Runtime | 120 s | 45 s |
| MySQL Timeout Errors | 7 per night | 0 |
| CPU Utilization (peak) | 85 % | 42 % |
| Monthly VPS Bill | $32.00 | $28.00 |
Security Considerations
- Never expose MySQL to the public internet; use
bind-address = 127.0.0.1or a private VPC subnet. - Enable
ssl-mode=REQUIREDinconfig/database.phpwhen connecting to a managed DB. - Store queue secrets (Redis password, DB credentials) in
.envand restrict file permissions to600. - Run Supervisor and PHP‑FPM as non‑root
www-datauser. - Audit Composer dependencies regularly:
composer audit.
Bonus Performance Tips
- Batch DB writes inside jobs using
DB::transaction()to reduce lock contention. - Leverage Laravel Horizon for real‑time queue monitoring and auto‑scaling.
- Place frequently accessed configs in Redis with
Cache::rememberForever(). - Enable HTTP/2 on Nginx:
listen 443 ssl http2;for faster API responses. - Use
php artisan schedule:workinstead of cron if you need sub‑minute precision.
FAQ
Q: My queue still times out after the changes. What next?
A: Check storage/logs/laravel-queue.log for SQLSTATE[HY000] [2006] MySQL server has gone away. Increase net_read_timeout on MySQL and verify no firewall drops.
Q: Can I use the same config on a shared hosting plan?
A: Shared hosts often block Supervisor and restrict php.ini. Switch to php artisan queue:listen and use the database driver. You’ll lose some performance but keep reliability.
Q: Is Redis mandatory?
A: Not mandatory, but it isolates queue traffic from MySQL, eliminates the timeout race, and gives you atomic job handling. For low‑traffic sites, the database driver with proper wait_timeout tuning is sufficient.
Final Thoughts
Laravel queue failures on a VPS rarely stem from a single mis‑configuration. They are the sum of MySQL timeouts, PHP‑FPM limits, Supervisor settings, and missing caching layers. By aligning every piece—from wait_timeout to Redis persistence—you turn a fragile nightly crash into a resilient, auto‑scaling pipeline ready for growth.
Implement the checklist, monitor your queue:failed table, and you’ll never again wonder why the nightly sync exploded.
No comments:
Post a Comment