Tremendous - Site availability issues – Incident details

At about 12:50 UTC there was a spike in the number of database connections used, to the point of exhausting the maximum number of available database connections. Upon investigation we realized that this was caused by a rapid succession of new app releases, which caused our infrastructure provider Heroku to launch lots of new app dynos. However, the old dynos are not shut down immediately. So with each new release the number of used connections went up. At this time we had already stopped issuing new releases so the issue resolved itself as the old app dynos started being shut down. As remediation, we are updating our process to prevent the issuing of this many new releases in such a short timeframe.