Our internal monitoring system altered us to a delay in sending test results data to customers making use of the data push destinations feature. We began to investigate this and attempted a few interventions to resolve the issue. The nature of the problem meant we needed to monitor the backlog of data for a while before we would be sure our solution was resolving the issue. Once we identified the root cause, we were able to allocate more computing resources to process the backlog of data. The recovery was still a slow process which is why it took a few days to fully recover.

The delay affected a small number of customer's sensors with large delays and most sensors had minor delays. During the last 24 hours of recovery, the number of affected sensors diminished drastically.

We have put extra computing resources in place to prevent this type of failure from happening again.