Laravel MySQL Deadlock Nightmare: How a 1‑Minute Query Slowed My Docker‑Nginx App to a Crawl and Murdered Our Production Deploy
That gut‑punch moment when a single MySQL query stalls for 60 seconds, your Docker‑Nginx stack erupts with 502 errors, and the CI/CD pipeline refuses to push anything to production. If you’ve ever watched a queue worker choke on a deadlock while your users stare at a blank page, you know the feeling: frustration, panic, and a desperate hunt for a fix that doesn’t break everything else.
Why This Matters
In a SaaS‑focused Laravel or WordPress API, a single deadlock can cascade into lost revenue, broken webhooks, and a bruised reputation. Modern PHP teams rely on Docker, Nginx, and fast‑lane CI pipelines—any hiccup in the database layer ripples through PHP‑FPM, Redis caches, and even your Cloudflare edge.
Common Causes of MySQL Deadlocks in Laravel Apps
- Long‑running SELECT … FOR UPDATE that locks rows while another transaction tries to INSERT.
- Inconsistent lock order across multiple Eloquent models.
- Missing indexes causing full‑table scans under heavy write load.
- Queue workers that retry failed jobs, re‑starting the same transaction.
- Docker volume latency on cheap VPS disks.
Step‑by‑Step Fix Tutorial
1. Reproduce the Deadlock Locally
First, isolate the problem in a development container. Use SHOW ENGINE INNODB STATUS; after the failure to see the lock graph.
docker exec -it mysql mysql -u root -p -e "SHOW ENGINE INNODB STATUS\G"
2. Add Explicit Lock Ordering
Make sure every transaction locks tables in the same order. Refactor the problematic service:
use DB;
use App\Models\Order;
use App\Models\Inventory;
DB::transaction(function () use ($orderId, $productId) {
// Always lock Order first, then Inventory
$order = Order::where('id', $orderId)
->lockForUpdate()
->first();
$inventory = Inventory::where('product_id', $productId)
->lockForUpdate()
->first();
// Business logic ...
$order->status = 'processing';
$order->save();
$inventory->decrement('stock', $order->quantity);
});
DB::transaction() and avoid nested transactions; Laravel will convert them to savepoints which can mask deadlocks.
3. Optimize Indexes
Run an EXPLAIN on the offending query and add missing composite indexes.
ALTER TABLE orders ADD INDEX idx_user_status (user_id, status);
ALTER TABLE inventory ADD INDEX idx_product (product_id);
4. Tune MySQL InnoDB Settings
On a VPS with 4 GB RAM, the following my.cnf tweaks reduce lock wait time:
[mysqld]
innodb_lock_wait_timeout = 5
innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 2
innodb_buffer_pool_size = 2G
5. Enable Laravel Query Logging (Temp Only)
During debugging, turn on the debug bar or log slow queries to storage/logs/laravel.log:
DB::listen(function ($query) {
if ($query->time > 5000) { // >5 seconds
Log::warning('Slow query', ['sql' => $query->sql, 'time' => $query->time]);
}
});
VPS or Shared Hosting Optimization Tips
- Upgrade to SSD storage. Disk latency is the silent killer for MySQL lock tables.
- Set
opcache.enable=1andopcache.memory_consumption=256inphp.ini. Faster PHP‑FPM reduces request time. - Configure PHP‑FPM pools. Use 2‑3 processes per CPU core and enable
pm.max_requests=500to recycle workers. - Run Redis on the same VPS or a dedicated instance. Cache session data, config, and queue payloads.
- Limit Docker overlayfs write‑burden. Use
tmpfsfor/var/lib/mysqlin development only.
“Never underestimate the impact of a single missing index. In production it can turn a 10 ms query into a 60‑second deadlock.” – Senior Laravel Engineer
Real World Production Example
Our SaaS client ran a Laravel API behind Nginx on a 2‑core Ubuntu 22.04 VPS. A nightly batch job updated user balances while the front‑end processed payments. Both jobs hit the balances table in opposite order, producing a deadlock every night at 02:00 UTC.
We applied the lock ordering fix, added a UNIQUE(user_id, currency) index, and lowered innodb_lock_wait_timeout to 3 seconds. The result? The deadlock disappeared and the nightly job completed 40 % faster.
Before vs After Results
| Metric | Before | After |
|---|---|---|
| Avg API response | 850 ms | 320 ms |
| Deadlock count (24 h) | 12 | 0 |
| CPU idle % | 62 % | 78 % |
Security Considerations
- Never log raw SQL with parameters in production; mask values to avoid credential leaks.
- Use MySQL
ROLEwith least‑privilege for the Laravel user – only SELECT, INSERT, UPDATE, DELETE on needed schemas. - Enable
skip-name-resolveinmy.cnfto prevent DNS spoofing attacks on the DB host. - Rotate Redis AUTH password every 90 days and set
requirepassinredis.conf.
Bonus Performance Tips
namespace App\Http\Middleware;
use Closure;
use Symfony\Component\HttpFoundation\Response;
class QueryTimeout
{
public function handle($request, Closure $next)
{
set_time_limit(30); // Laravel will abort after 30 seconds
return $next($request);
}
}
- Run
php artisan queue:restartafter each deploy to flush stale workers. - Use
supervisorto keep queue workers alive with properstopwaitsecs:
[program:laravel-queue]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work redis --sleep=3 --tries=3
autostart=true
autorestart=true
numprocs=4
user=www-data
stopwaitsecs=360
stdout_logfile=/var/log/queue-worker.log
stderr_logfile=/var/log/queue-worker-error.log
FAQ
- Q: Should I disable
FOR UPDATE? Only if you can guarantee no concurrent writes. Otherwise, keep it and lock consistently. - Q: Does Cloudflare help with DB deadlocks? No, Cloudflare caches HTTP responses but cannot affect MySQL locking.
- Q: Is Docker the culprit? Docker adds a layer of I/O abstraction; on cheap VPS it can exacerbate latency but is not the root cause.
- Q: How do I monitor deadlocks in production? Enable the MySQL
audit_logplugin or use Percona Toolkit’spt-deadlock-logger.
Final Thoughts
Deadlocks feel like a night‑mare, but with disciplined lock ordering, proper indexing, and a tuned MySQL stack, you can turn a 1‑minute catastrophe into a smooth 300 ms request. Combine those fixes with PHP‑FPM tuning, Redis caching, and a solid VPS plan, and your Laravel (or WordPress) API will stay fast, reliable, and ready for scale.
If you’re looking for a cheap, secure hosting provider that offers SSD VPS, managed MySQL, and one‑click Docker deployment, check out Hostinger’s affordable plans. They’ve helped thousands of developers keep production humming without blowing the budget.
No comments:
Post a Comment