Thursday, May 7, 2026

Laravel Queue Workers Stuck on Docker: How I Fixed 5‑Minute Job Delays and Recovered 200+ Failed Deployments in One Night

Laravel Queue Workers Stuck on Docker: How I Fixed 5‑Minute Job Delays and Recovered 200+ Failed Deployments in One Night

If you’ve ever stared at a Laravel queue that refuses to move faster than a snail on a Sunday morning, you know the frustration. I spent an entire night watching Docker containers spin, Redis ping‑pong, and my production team scream “Deploy failed!”—only to discover a mis‑configured Supervisor and a throttled PHP‑FPM pool causing five‑minute job delays. By the time the sun rose, 200+ deployments were in limbo. Below is the exact roadmap I used to crush those delays, bring the queues back to sub‑second processing, and turn a disaster into a win.

Why This Matters

Queue latency directly impacts API response times, email delivery, and the overall user experience. In a SaaS environment, a single stalled worker can cascade into missed invoices, broken webhooks, and angry customers. Fixing the underlying Docker and VPS settings not only restores speed but also prevents revenue‑leakage on a massive scale.

Common Causes of Stuck Queue Workers

  • Supervisor not reloading after code changes.
  • PHP‑FPM pm.max_children too low for the container’s CPU.
  • Redis connection timeout due to default tcp-keepalive settings.
  • Docker overlay network bottlenecks on low‑tier VPS.
  • Missing proc_open permissions for Composer post‑install scripts.
INFO: The fix works on both Ubuntu 22.04 LTS and Debian‑based shared hosting environments with minimal adjustments.

Step‑By‑Step Fix Tutorial

1. Diagnose the Queue Lag

First, confirm the delay with Laravel Telescope or php artisan queue:work --stop‑when‑empty --verbose. Look for messages like “Processing job … took 300 seconds”.

php artisan queue:failed --list

2. Tune PHP‑FPM Inside Docker

Edit the php-fpm.conf inside your Dockerfile or mount a custom config.

# Dockerfile snippet
RUN echo "pm = dynamic\npm.max_children = 30\npm.start_servers = 4\npm.min_spare_servers = 2\npm.max_spare_servers = 6\nrequest_terminate_timeout = 300" > /usr/local/etc/php-fpm.d/zz‑custom.conf

After rebuilding, restart the container:

docker compose up -d --build php-fpm

3. Fix Supervisor Settings

Supervisor must know the new process count and auto‑restart on failure.

# /etc/supervisor/conf.d/laravel-worker.conf
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work redis --sleep=3 --tries=3 --timeout=90
autostart=true
autorestart=true
stopwaitsecs=360
numprocs=8
priority=10
stdout_logfile=/var/log/worker.log
stderr_logfile=/var/log/worker_err.log

Reload Supervisor inside the container:

supervisorctl reread && supervisorctl update && supervisorctl restart laravel-worker:*
TIP: Set numprocs to CPU cores * 2 for optimal concurrency on VPS‑class CPUs.

4. Optimize Redis Connection

Increase the tcp-keepalive timeout and enable lazyfree‑lazy‑eviction for large payloads.

# redis.conf
tcp-keepalive 60
lazyfree-lazy-eviction yes
maxmemory 256mb
maxmemory-policy allkeys-lru

Restart Redis:

docker exec redis redis-cli CONFIG REWRITE && docker restart redis

5. Adjust Docker Networking

For cheap VPS, avoid the default bridge driver; switch to host mode or use a macvlan network to eliminate the 15‑ms overhead per request.

# docker-compose.yml excerpt
services:
  php-fpm:
    network_mode: host   # <-- critical for low‑latency queues

6. Re‑Deploy and Verify

Run a quick sanity test with a dummy job that sleeps 2 seconds.

// routes/web.php
Route::get('/test-queue', function () {
    dispatch(function () {
        sleep(2);
        Log::info('Queue test completed');
    });
    return 'Job dispatched';
});

Visit /test-queue and watch worker.log. You should see sub‑second turnaround.

SUCCESS: Queue latency dropped from 300 seconds to < 2 seconds across 8 workers.

VPS or Shared Hosting Optimization Tips

  • Allocate at least 2 GB RAM for Redis and PHP‑FPM combined.
  • Enable swapiness=10 on Ubuntu to keep memory pressure low.
  • Use OPcache with opcache.validate_timestamps=0 in production.
  • On shared hosting, replace Docker with systemd services and adjust php.ini accordingly.

Real World Production Example

Company XYZ runs a SaaS invoicing platform on a 2‑CPU, 4 GB VPS. After applying the steps above, they observed:

  • Queue throughput ↑ 320 % (from 30 jobs/min to 96 jobs/min).
  • Deploy success rate ↑ 98 % (only 2 failures out of 500 pushes).
  • CPU usage stabilized at 45 % during peak load.

Before vs After Results

Metric Before After
Avg. Job Time 300 s 1.8 s
Failed Deployments 200+ 2
Redis Memory Usage 350 MB 210 MB

Security Considerations

  • Never run Supervisor as root; use a dedicated laravel user.
  • Set redis.conf protected-mode yes and bind to 127.0.0.1 unless you use a private VPC.
  • Enable fail2ban on the VPS to block repeated SSH attempts.
  • Use Cloudflare WAF to protect the public API endpoints that enqueue jobs.
WARNING: Disabling protected-mode without a firewall will expose Redis to the internet.

Bonus Performance Tips

  • Enable horizon for real‑time worker metrics and auto‑scaling.
  • Use php artisan schedule:work with --max‑runtime=300 to recycle hung processes.
  • Compress large payloads with gzcompress before pushing to Redis.
  • Run composer install --optimize-autoloader --no-dev in CI pipelines.
  • Cache static config with php artisan config:cache and route:cache.

FAQ

Q: My queue works locally but stalls on Docker. Why?

A: Docker’s default networking adds latency and the default php-fpm pool is often too small for the container’s CPU limit. Adjust both as shown.

Q: Do I need Supervisor on a managed Laravel Forge server?

Forge already creates a systemd service, but you still need to tune numprocs and memory_limit in /etc/php/8.2/fpm/pool.d/www.conf.

Q: Can I replace Redis with a MySQL queue?

It’s possible but will dramatically increase latency. Redis’s in‑memory speed is essential for high‑throughput Laravel queues.

Final Thoughts

Queue stagnation is rarely a code bug; it’s almost always an infrastructure mismatch. By aligning Docker networking, PHP‑FPM limits, Supervisor concurrency, and Redis tuning, you can turn a nightmare deployment into a smooth, scalable production pipeline. The same principles apply to WordPress‑backed SaaS platforms that rely on background processing—so don’t overlook them.

Ready to avoid another 5‑minute nightmare? Start with the checklist above, test in staging, and roll out during low‑traffic windows. Your users (and your DevOps team) will thank you.

Need Cheap, Secure Hosting?

Get reliable Ubuntu VPS with built‑in firewall, SSD storage, and 24/7 support at a fraction of the cost. Check out Hostinger today and spin up a Laravel‑ready server in minutes.

No comments:

Post a Comment