Outage 10-4-19 - OTT Platform

Incident Report for Vimeo OTT

Postmortem

On October 4th, 2019, Vimeo OTT platform experienced two outages, the first between 12:00 and 12:05 EDT and the second between 14:00 and 15:34 EDT.

What was the issue? An internal test inadvertently and adversely affected our production database. The test caused database degraded performance, eventually resulting in the outages.

When was this resolved? The issue was resolved at 15:34pm EDT on October 4th, 2019, following quick response by the engineering teams who identified the root cause and initiated remediation actions. As per our process, the customer facing status page was updated every 30 minutes here: status.vhx.tv

*What are we doing to prevent this from happening in the future?
*
All test procedures have been reviewed and changes made to prevent a similar recurrence. We will continue to work with our vendors to review and deploy the industry's best solutions on our platforms.

If there are any follow up questions or concerns in regards to this issue, please contact the Support team via vimeoott.zendesk.com for assistance.

Vimeo Enterprise Support

Posted Nov 01, 2019 - 18:34 EDT

Resolved

This incident has been resolved. Thank you for your patience. If you have any further questions or concerns please contact our Support Team at ottsupport@vimeo.com.
Posted Oct 04, 2019 - 16:29 EDT

Monitoring

We have identified the issue and have made some system updates that are allowing the platform to stabilize. We will continue to monitor and will update with any changes.
Posted Oct 04, 2019 - 15:56 EDT

Update

We are experiencing intermittent site outages. We are continuing to investigate this issue, and will update within 30 minutes or as soon as the issue is resolved.
Posted Oct 04, 2019 - 15:18 EDT

Update

We are continuing to investigate this issue. We will update within 30 minutes or as soon as the issue is resolved.
Posted Oct 04, 2019 - 14:43 EDT

Investigating

Our platform is currently experiencing a service-wide outage. We are actively investigating the issue and will provide updates as soon as possible. Thank you for your patience.
Posted Oct 04, 2019 - 14:11 EDT
This incident affected: Vimeo OTT Platform.