Queuing database issues
Incident Report for Labelbox
Postmortem

The database that backs our queuing service encountered a spike in operations and with response latency, leading to a backlog of requests and heightened latency with queue-related operations (including labeling). We were able to scale our services and reduce latency from these requests, which cleared out the backlog sooner, and operations resumed. We’ve made some configuration changes to the database, more granular monitoring and alerting, and have updated the database size and scale itself to handle a greater load.

Posted Feb 19, 2020 - 04:08 UTC

Resolved
This incident has been resolved.
Posted Feb 12, 2020 - 17:20 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Feb 12, 2020 - 10:38 UTC
Investigating
We are currently investigating this issue.
Posted Feb 12, 2020 - 10:26 UTC
This incident affected: Platform (common services).