tremendous-status - AWS outage – Incident details

AWS outage

ResolvedMajor outage23 hours 51 minutes

Affected

API

Major outage from 12:27 PM to 12:18 PM

Reward redemptions

Major outage from 12:27 PM to 12:18 PM

Dashboard

Major outage from 12:27 PM to 12:18 PM

  • ResolvedUTC

    This incident has been resolved.

  • UpdateUTC

    We are no longer experiencing issues. Our upstream providers haven't marked their issues as resolved, so we're continuing to monitor ourselves.

  • UpdateUTC

    We are continuing to monitor for any further issues.

  • UpdateUTC

    Our production servers are back and we'll keep monitoring them to make sure it's stable again as our providers are still working on the issue.

  • MonitoringUTC

    Our production servers are back and we'll keep monitoring them to make sure it's stable again as our providers are still working on the issue.

  • UpdateUTC

    OK, AWS seems to have lost power to one of their data centers in the US-East-1 region. They're working to restore power now, and are starting to see servers come back.

  • IdentifiedUTC

    Confirmed that the issue is related to AWS, and so large parts of the internet are down too. We've already contacted our downstream providers, but unfortunately we're going to have to wait for them to figure this out.

  • InvestigatingUTC

    Production servers are returning 503 - Application Errors. On first glance, this appears to be related to downstream issues with our hosting provider, unrelated to the Tremendous application. We are investigating.