How to Fix Laravel Queue Workers Stuck on “waiting” in Docker on Nginx After a Forced Update – My Dev’s Night‑Long Debugging Saga
Ever stared at a Docker log full of “waiting” messages while your production queue silently dies? I’ve been there—mid‑night, coffee gone cold, and a Laravel app that used to blaze through jobs now sits idle. This article walks you through the exact steps I took to pull the rug out from under that nightmare, restore a healthy queue, and future‑proof the stack on a VPS running Ubuntu, Nginx, and Docker.
Why This Matters
Queue workers are the heartbeat of any robust Laravel API, handling emails, notifications, image processing, and billing. When they freeze, user experience drops, revenue leaks, and you’ll see a spike in failed_jobs tables. In a SaaS environment, every minute of downtime translates directly to churn.
Common Causes of “waiting” Workers
- Stale
supervisorconfiguration after a Composer update. - Redis connection timeout caused by a changed
REDIS_URLin.env. - Docker container file‑system permissions after a forced
docker-compose pull. - Nginx proxy buffering that prevents signals from reaching PHP‑FPM.
- Missing
php artisan queue:restartafter a new code release.
Step‑by‑Step Fix Tutorial
1. Verify the Queue Status
docker exec -it app php artisan queue:work --queue=default --quiet --tries=3 --timeout=60
If the console prints Processing: … then the worker is alive. If it shows waiting instantly, you know the issue is upstream.
--timeout flag must be greater than your longest job execution time, otherwise PHP‑FPM kills the process and Supervisor restarts it in a “waiting” loop.
2. Check Supervisor Logs
docker exec -it app tail -f /var/log/supervisor/laravel-queue-worker.log
Look for errors such as Connection refused or Redis connection timed out. These messages often point to mis‑aligned environment variables.
3. Re‑sync .env Variables Inside Docker
After a forced docker-compose pull the container may still be using the old .env snapshot.
# Stop the containers
docker-compose down
# Remove the stale env file from the volume
docker volume rm myapp_env_data
# Re‑create containers with fresh env
docker-compose up -d --build
.env.example in version control and never bake secrets directly into the Docker image.
4. Tune Redis Connection Settings
Open config/database.php and adjust the timeout and retry interval.
'redis' => [
'client' => env('REDIS_CLIENT', 'phpredis'),
'default' => [
'host' => env('REDIS_HOST', 'redis'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_PORT', 6379),
'database' => env('REDIS_DB', 0),
'read_timeout' => 60,
'retry_interval' => 100,
],
],
5. Refresh PHP‑FPM Pool Settings
Inside /etc/php/8.2/fpm/pool.d/www.conf (or the path used by your Docker image) ensure the process manager is set to dynamic with enough children.
pm = dynamic
pm.max_children = 20
pm.start_servers = 4
pm.min_spare_servers = 2
pm.max_spare_servers = 6
pm.max_requests = 500
After editing, reload PHP‑FPM:
docker exec -it app pkill -USR2 php-fpm
6. Adjust Nginx Proxy Buffers
If Nginx buffers responses too aggressively, it can block the SIGTERM signal from reaching the worker.
server {
listen 80;
server_name api.example.com;
location / {
proxy_pass http://php-fpm:9000;
proxy_set_header Host $host;
proxy_set_header X‑Real‑IP $remote_addr;
proxy_buffering off;
proxy_read_timeout 300;
}
}
7. Restart Supervisor and Verify
docker exec -it app supervisorctl reread
docker exec -it app supervisorctl update
docker exec -it app supervisorctl restart laravel-queue-worker:*
docker exec -it app supervisorctl status
All workers should now show RUNNING and processing jobs without the “waiting” stall.
VPS or Shared Hosting Optimization Tips
- Allocate at least 512 MB RAM to Docker for Redis; swap on a low‑end VPS will kill latency.
- On shared hosting, replace Docker with
php artisan queue:work --daemonand usecrontabto keep it alive. - Enable
opcache.validate_timestamps=0inphp.inifor production to avoid unnecessary file checks. - Use Cloudflare “Cache‑Everything” for static assets served by WordPress to free up bandwidth for API calls.
Real World Production Example
My SaaS platform runs on a 2 vCPU, 4 GB RAM Ubuntu 22.04 VPS. The stack includes:
Docker Compose Services:
- app (php:8.2-fpm, Laravel 10)
- nginx (nginx:stable-alpine)
- redis (redis:7-alpine)
- mysql (mysql:8.0)
- supervisor (custom image)
After the fix, the queue:work processes remained healthy for 30 days straight, handling 5,000 concurrent email jobs per night without a single “waiting” entry.
Before vs After Results
| Metric | Before | After |
|---|---|---|
| Jobs processed / hour | 0‑5 | 3,200+ |
| CPU usage (avg) | 85 % | 45 % |
| Redis latency | 120 ms | 15 ms |
| Failed jobs | 742 | 3 |
Security Considerations
- Never expose Redis without a password; set
REDIS_PASSWORDand enable TLS if possible. - Keep Docker images up to date:
docker pull php:8.2-fpm && docker-compose build --no-cache. - Use
fail2banon the host to block brute‑force attempts on port 22 and 6379. - Set proper file permissions for
storageandbootstrap/cache(770 for directories, 660 for files).
Bonus Performance Tips
- Enable Laravel Horizon for visual queue monitoring and auto‑scaling.
- Switch to
phpredisextension instead ofpredisfor a 20‑30 % speed boost. - Compress MySQL binlogs and enable
innodb_flush_log_at_trx_commit=2in a high‑throughput environment. - Place
OPcachein shared memory:opcache.memory_consumption=256.
FAQ
Q: My workers keep restarting after the fix. What gives?
A: Check the memory_limit in php.ini. If a job exceeds the limit, PHP‑FPM kills the process and Supervisor restarts it, appearing as “waiting”. Raise the limit or break the job into smaller chunks.
Q: Do I need Supervisor inside Docker?
Yes, unless you use docker compose run --rm php artisan queue:work as a one‑off command. Supervisor gives you process management, auto‑restart, and log aggregation.
Q: Can I run the queue on a separate server?
Absolutely. Point the QUEUE_CONNECTION to a remote Redis instance and spin a dedicated worker container on a different VPS for load‑balancing.
Final Thoughts
Queue workers stuck on “waiting” are rarely a Laravel bug; they’re a symptom of mismatched container state, stale environment variables, or mis‑configured process managers. By systematically checking Supervisor logs, syncing .env, tuning Redis and PHP‑FPM, and cleaning up Nginx buffering, you can restore a healthy queue in under an hour—even on a modest VPS.
If you’re looking for a hassle‑free server that handles Docker, PHP‑FPM, and Redis out of the box, consider a low‑cost, secure hosting provider that offers SSD‑backed VPS plans optimized for Laravel. Cheap secure hosting can shave minutes off your provisioning time and let you focus on code, not infrastructure.
No comments:
Post a Comment