Monitoring - A fix has been implemented and we are monitoring the results.
Apr 27, 20:55 EDT
Identified - Radar test results are experiencing slow or no updates at the moment and being queued up, we've identified the cause and are working to mitigate it.
Apr 27, 18:23 EDT
Monitoring - Normal system operation resumed at 9:01AM Pacific Time after test runs paused at 8:50AM this morning.
Apr 27, 12:05 EDT
Identified - We've detected another service disruption that is causing paused test runs as of 8:40am.
Apr 27, 11:49 EDT
Monitoring - Radar tests and the Runscope Dashboard became operational at 8:06 AM Pacific Time. All throughput has returned to normal as of 8:20 AM. At this time all systems are back to normal operation. We are continuing to research the root cause of these problems and improve throughput of test runs.
Apr 27, 11:21 EDT
Identified - Radar tests and the Runscope Dashboard began running slowly again at 7:40 AM Pacific Time. We are gathering more data about the issues and hope to have both components fully functional shortly.
Apr 27, 10:55 EDT
Monitoring - Radar tests and the Runscope Dashboard became operational at 6:11 PM Pacific Time. All throughput returned to normal by about 6:20 PM. At this time all systems are back to normal operation. We are continuing to research the root cause of these problems and improve throughput of test runs.
Apr 26, 21:38 EDT
Identified - Radar tests and the Runscope Dashboard began running slowly again at 5:50 PM Pacific Time. We are gathering more data about the issues and hope to have both components fully functional shortly.
Apr 26, 20:59 EDT
Monitoring - Radar tests, test results, and the Radar API have recovered and have been operating close to normal capacity since 1:14 PM Pacific Time. Dashboard performance has also improved and we are continuing to monitor availability. We'll continue to put measures in place to avoid outages and improve performance.
Apr 25, 16:54 EDT
Identified - We are still working on resolution to the issues causing tests not to run and results not to load. We have put some measures in place which is improving throughput of test runs, but they are not yet running at full capacity.
Apr 25, 16:06 EDT
Investigating - We're investigating an issue causing test runs to execute slowly or not at all as well as causing the dashboard to be non-responsive. Will update as we know more.
Apr 25, 15:11 EDT
Dashboard - runscope.com ? Operational
Runscope Radar Degraded Performance
Runscope URL Gateway (US East) ? Operational
Runscope URL Gateway (US West) ? Operational
Runscope URL Gateway (US Midwest) ? Operational
Runscope URL Gateway (US Texas) ? Operational
Runscope URL Gateway (Australia) ? Operational
Runscope URL Gateway (Brazil) ? Operational
Runscope URL Gateway (Ireland) ? Operational
Runscope URL Gateway (Singapore) ? Operational
Runscope API ? Operational
Passageway Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Past Incidents
Apr 28, 2015

No incidents reported today.

Apr 25, 2015
Resolved - All systems are fully operational. Radar tests are running on normal schedules across all regions. Ultimately we found that some Radar tests did not run between 1:15 AM and 1:38 AM Pacific time. You may see some missing or "expired" test runs during that time.
Apr 25, 05:13 EDT
Monitoring - Radar tests are operating normally. We are continuing to monitor the situation.
Apr 25, 04:53 EDT
Investigating - We're investigating an issue causing tests to execute slower than normal and potentially 'Expire' (exceeding the maximum execution time allowed for a test). Updates will follow as we know more.
Apr 25, 04:32 EDT
Apr 24, 2015

No incidents reported.

Apr 23, 2015
Resolved - The issues have been resolved and the system has stabilized. The backlog of traffic was processed, and Radar traffic is available again in the Traffic Inspector. We are performing a full root cause analysis on the underlying issues and putting additional monitoring and infrastructure in place to reduce the likelihood of recurrences of this issue.
Apr 23, 17:52 EDT
Monitoring - We are continuing to process the backlog of traffic. Radar traffic is still excluded from the Traffic Inspector. New requests made through the Runscope URL gateways are showing up. Radar tests have fully recovered and have operated normally since 10:10am PT.
Apr 23, 14:03 EDT
Update - Traffic Inspector is loading data again, but there are delays processing new traffic being handled by the Runscope URL gateways. There is a large backlog of traffic that we are processing. We are adding additional resources to speed up how quickly the backlog can be cleared. Radar tests continue to operate normally.
Apr 23, 13:37 EDT
Update - We are continuing work on resolving the issues, however test runs and results have been performing normally for the past 20 minutes. It's possible that expirations may reoccur but we are monitoring the situation closely.
Apr 23, 13:30 EDT
Update - As a temporary measure to improve performance of Radar tests, requests executed during a test will be not be stored or displayed in the Traffic Inspector. *Radar traffic will continue to be available, with full request/response detail in the Radar test results detail.* This exclusion only affects Radar traffic being viewed in the Traffic Inspector. All other traffic generated via Runscope URLs, request capture, etc. will continue to be displayed in the Traffic Inspector.
Apr 23, 13:19 EDT
Identified - We have identified a potential source of the problem and are spinning up additional capacity to address the issue. As the capacity is added you will begin to see test run performance improve and the Traffic Inspector restored.
Apr 23, 13:04 EDT
Update - The Traffic Inspector is currently unable to load request data. Runscope URLs are functioning normally. We are continuing to investigate the issue causing test run slowness/expiration along with the Traffic Inspector issue.
Apr 23, 12:49 EDT
Investigating - We're investigating an issue causing tests to execute slower than normal and potentially 'Expire' (exceeding the maximum execution time allowed for a test). You may also experience intermittent issues loading the dashboard and test results. Updates to follow as we know more.
Apr 23, 12:37 EDT
Apr 22, 2015
Resolved - All systems are operating normally, a post-mortem analysis will be after we complete a thorough investigation and determine corrective actions.
Apr 22, 21:07 EDT
Monitoring - All systems have returned to normal, we are continuing to monitor and investigate all affected systems
Apr 22, 20:35 EDT
Identified - We've identified the cause of inconsistent Radar and Dashboard page loads and are monitoring our systems as they recover
Apr 22, 19:57 EDT
Investigating - We're investigating inconsistent loading of Dashboard and Radar pages
Apr 22, 18:55 EDT
Apr 21, 2015

No incidents reported.

Apr 20, 2015
Resolved - All systems are operational, Radar tests are running on normal schedules across all regions
Apr 20, 20:37 EDT
Monitoring - Radar tests are operating normally but we are continuing to monitor
Apr 20, 20:24 EDT
Identified - We've identified the cause of the instability with Radar tests and are working to mitigate and resolve the issue.
Apr 20, 19:10 EDT
Investigating - We're investigating instability in display of Radar test results in Runscope Dashboard.
Apr 20, 18:26 EDT
Apr 19, 2015

No incidents reported.

Apr 18, 2015

No incidents reported.

Apr 17, 2015

No incidents reported.

Apr 16, 2015

No incidents reported.

Apr 15, 2015

No incidents reported.

Apr 14, 2015

No incidents reported.