We've replaced the cloud service (load balancer) that was failing. We've monitored it for over an hour, and the metrics look stable. We will still be closely monitoring the situation, so please don't hesitate to reach out if you see anything unusual. Our deepest apologies for the recent disruption.
Posted Jul 29, 2020 - 23:54 UTC
Update
We are still observing intermittent instability. We continue to actively investigate and work on restoring stability to the service.
Posted Jul 29, 2020 - 20:40 UTC
Identified
We are investigating a problem with our cloud provider, who applied an automatic update to our load balancer late last night that resulted in connection issues. We have engaged their support to troubleshoot the issue. In the mean time, our Engineering team continues to investigate alternative mitigation steps to restore stable service.
Posted Jul 29, 2020 - 19:47 UTC
Investigating
We're seeing residual effects from the outage last night, the team is continuing to monitor the issue as we work towards root cause
Posted Jul 29, 2020 - 16:59 UTC
This incident affected: Annotate and Platform (common services).