Elevated latency and errors for History service in US-East

Incident Report for PubNub

Postmortem

Problem Description, Impact, and Resolution 

On June 10, 2025 at 18:33 UTC, we observed elevated error rates and increased latency for a subset of traffic for the History service in the US East region.

Customers using the History API in this region may have encountered read failures or timeouts during this window. We identified transient connection issues with infrastructure supported by a third-party provider service. We quickly escalated the issue, working with the provider to identify and mitigate the root cause.

Service performance returned to normal levels, and the issue was resolved by 19:07 UTC the same day.

Mitigation Steps and Recommended Future Preventative Measures 

To improve the resilience of the History service and prevent such issues from recurring in the future, we have upgraded backend storage systems and deployed updated versions of our History mechanisms globally. These updates improve compatibility with recent infrastructure changes and enhance the reliability of backend connectivity.

In the coming weeks, we are also evaluating a modernization of the History service architecture to further improve long-term stability and performance.

Posted Jun 23, 2025 - 17:43 UTC

Resolved

Beginning at 18:33 UTC we observed increased latency and errors for our History service in one of our North America regions. Users in the affected region may have experienced failures with error or latency when making calls to the History endpoint. The issue has been resolved as of 19:07 UTC.

We will publish a root cause analysis within the coming days. We sincerely apologize for any impact caused to your PubNub services. If you feel you have been impacted by the issue and wish to report it, please contact support@pubnub.com.
Posted Jun 10, 2025 - 18:30 UTC