Performance Impact

Incident Report for Storj DCS

Postmortem

On May 22, 2025 from 00:12 to 02:54 UTC Storj experienced a service disruption causing elevated error rates and increased request latency that affected the US1 API, Gateway, Linksharing, and US1 - Select services. We discovered lock contention triggered by high-load delete operations. In the short term, we are artificially slowing down some of these intensive requests to leave needed headroom. We will eliminate the lock contention going forward.

Posted May 23, 2025 - 17:20 UTC

Resolved

This incident has been resolved.
Posted May 22, 2025 - 05:13 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted May 22, 2025 - 02:47 UTC

Update

We are continuing to investigate this issue.
Posted May 22, 2025 - 02:09 UTC

Investigating

We are investigating reports of performance impacts to one or more components of our system.
Posted May 22, 2025 - 01:53 UTC
This incident affected: AP1 (AP1 - API, AP1 - Gateway), EU1 (EU1 - API, EU1 - Gateway), and US1 (US1 - API, US1 - Gateway, US1 - Select).