Elevated latencies and error rates in US East PoP
Incident Report for PubNub
Postmortem

Incident Summary

In our US-EAST point of presence third party networking issues resulted in errors and latency to a small portion of our traffic.

Root Cause

Our third party service provider lost network connectivity to some of our instances. Our automatic health checks removed these failed instances from service shortly after the incident began. Latency and connectivity issues returned to normal very quickly even whilst the network issues continue. This incident is now fully resolved with our provider and our service operates normally.

Posted Oct 17, 2017 - 22:15 UTC

Resolved
All errors rates and latencies have returned to normal.
Posted Oct 13, 2017 - 23:21 UTC
Update
Everything is still looking good with our services. We are continuing to monitor.
Posted Oct 13, 2017 - 23:08 UTC
Monitoring
We have identified a network outage on a subset of us-east-1 PoP. We have rerouted around those machines and we are seeing error rates and latencies fall back to normal.
Posted Oct 13, 2017 - 22:38 UTC
Investigating
We have noticed higher latencies and error rates in our US East Point of Presence. We are currently investigating the cause.
Posted Oct 13, 2017 - 22:13 UTC