Laravel Queue Workers Dead‑locking on Docker: How a Missing “chmod 775 storage” and Improper php‑fpm “pm.max_children” Setting Are Killing Your Prod App ⚡️
You’ve just pushed a fresh Laravel release to your Docker‑based production server. The API endpoint that used to return JSON in 30 ms now sits there, waiting forever. Your queue:work processes are stuck, Redis logs show “connection refused”, and the error logs are a sea of Permission denied. Sound familiar? You’re not alone. This article breaks down the two silent killers—wrong folder permissions and a mis‑tuned pm.max_children—and gives you a battle‑tested, copy‑paste‑ready fix.
Why This Matters
In a SaaS environment a single stalled queue can blow up response times, cause double‑billing, and break webhook callbacks. In a WordPress + Laravel hybrid, the same issue can stall scheduled posts, break email newsletters, and trash your SEO rankings. The cost is real: lost revenue, angry customers, and wasted dev‑time.
Common Causes of Queue Dead‑locks
- Docker volume permissions that default to
root:rooton the host. - Missing
chmod 775 storageafter a freshcomposer install. - PHP‑FPM
pm.max_childrenset too low for the number of workers you’re spawning. - Supervisor using the same
USERfor all processes, causing file lock contention. - Redis not reachable because the container is on a different Docker network.
Step‑by‑Step Fix Tutorial
1. Verify Storage Permissions
Info: Laravel needs write access to storage and bootstrap/cache. If these directories are owned by root the queue workers can’t create lock files.
# Inside your Dockerfile (or entrypoint.sh)
RUN set -eux; \
chown -R www-data:www-data /var/www/html/storage /var/www/html/bootstrap/cache; \
find /var/www/html/storage -type d -exec chmod 775 {} \;; \
find /var/www/html/bootstrap/cache -type d -exec chmod 775 {} \;
Alternatively, run it manually after deployment:
docker exec -it mylaravel_app bash
cd /var/www/html
sudo chown -R www-data:www-data storage bootstrap/cache
chmod -R 775 storage bootstrap/cache
2. Tune PHP‑FPM pm.max_children
Tip: Calculate pm.max_children based on your server RAM and average PHP memory usage.
# /usr/local/etc/php-fpm.d/www.conf
[www]
pm = dynamic
pm.max_children = 40 ; adjust for your VPS (e.g., 4GB RAM ≈ 40)
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 15
After editing, restart PHP‑FPM:
docker exec mylaravel_app pkill -o php-fpm
docker exec mylaravel_app php-fpm -D
3. Align Supervisor Config with PHP‑FPM
Success: Workers now respect the new pm.max_children limit and no longer compete for file locks.
# /etc/supervisor/conf.d/laravel-queue.conf
[program:laravel-queue]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work redis --sleep=3 --tries=3 --timeout=90
autostart=true
autorestart=true
user=www-data
numprocs=8
priority=100
stdout_logfile=/var/log/laravel/queue.log
stderr_logfile=/var/log/laravel/queue_error.log
Reload Supervisor:
docker exec mylaravel_app supervisorctl reread
docker exec mylaravel_app supervisorctl update
docker exec mylaravel_app supervisorctl restart laravel-queue:*
4. Confirm Redis Connectivity
Warning: A mis‑configured Docker network will still cause dead‑locks even after fixing permissions.
# docker-compose.yml snippet
services:
app:
image: mylaravel/app:latest
networks:
- backend
depends_on:
- redis
redis:
image: redis:6-alpine
networks:
- backend
networks:
backend:
driver: bridge
Test the connection from inside the app container:
docker exec -it mylaravel_app redis-cli -h redis ping
# Expected output: PONG
VPS or Shared Hosting Optimization Tips
- Use
ufwto allow only trusted IPs to Redis (port 6379). - Enable
opcache.enable=1and setopcache.memory_consumption=256inphp.ini. - Set
realpath_cache_size=4096Kfor faster class loading. - On shared hosting, replace Supervisor with
crontabbased “queue:listen” and keeppm.max_childrenlow (5‑10). - Configure Nginx fastcgi buffers:
fastcgi_buffers 8 16k;andfastcgi_buffer_size 32k;.
Real World Production Example
Acme SaaS runs a 4‑CPU, 8 GB Ubuntu 22.04 VPS behind Cloudflare. Before the fix the average API latency was 820 ms with occasional 30‑second timeouts. After applying the permission fix, tuning pm.max_children to 32, and reducing Supervisor processes to 6, latency dropped to 140 ms and queue throughput rose from 120 jobs/min to 540 jobs/min.
Before vs After Results
| Metric | Before | After |
|---|---|---|
| API Avg. Response | 820 ms | 140 ms |
| Queue Throughput | 120 jobs/min | 540 jobs/min |
| CPU Utilization | 85 % | 45 % |
| Memory Footprint | 7.8 GB | 4.3 GB |
Security Considerations
- Never set
chmod 777onstorage. Use775with proper group ownership. - Run PHP‑FPM and Supervisor as
www-dataor a dedicated non‑root user. - Restrict Redis to the Docker backend network; avoid exposing it to the public internet.
- Enable
fail2banfor SSH and Nginx brute‑force protection. - Regularly rotate Laravel
APP_KEYand Redis passwords.
Bonus Performance Tips
- Use
php artisan config:cacheandroute:cachein production. - Leverage Redis’s
STREAMfor event‑driven processing instead of standard queues. - Turn on
opcache.preloadto preload frequently used classes. - Compress Nginx responses with
gzip on;and setgzip_typesto includeapplication/json. - Deploy with zero‑downtime strategies:
docker compose up -d --no-deps --buildand rolling restarts.
FAQ
Q: My queue still stalls after fixing permissions. What else could be wrong?
A: Check the Docker memory limit. If the container hits the memory cgroup limit, PHP‑FPM will silently kill workers. Increase mem_limit in docker-compose.yml or move Redis to a dedicated host.
Q: How do I find the optimal pm.max_children?
Run top or htop while a load test is active. Divide available RAM (in MB) by average PHP memory usage (check memory_get_peak_usage()). Keep a 20% safety buffer.
Q: Can I use Laravel Horizon instead of Supervisor?
Yes. Horizon internally manages PHP‑FPM workers, but you still need correct chmod on storage/framework and a proper pm.max_children value. Horizon also gives you a live dashboard for quick diagnostics.
Final Thoughts
Missing a single chmod 775 storage and an over‑constrained pm.max_children can turn a healthy Laravel Docker stack into a production nightmare. By applying the steps above you’ll restore queue throughput, lower CPU load, and keep your API blazing fast. Remember: permissions and PHP‑FPM tuning are the foundation of any high‑scale Laravel or WordPress‑backed SaaS.
Looking for cheap, secure VPS hosting that plays nicely with Docker and Laravel? Check out Hostinger – reliable US‑based servers, 24/7 support, and a 30‑day money‑back guarantee.
No comments:
Post a Comment