Presence Server Errors and Elevated Latency

Incident Report for PubNub

Postmortem

Problem Description, Impact, and Resolution

On March 10, 2025, we observed elevated latency and errors in the Presence service across our global points of presence from 18:26 to 18:40 UTC and again from 20:05 to 20:27 UTC.

After investigating, we found that a short spike in traffic with an unusual pattern caused a downstream service provider to respond with increased latency, which affected our own response times. While this incident resolved quickly, we are working with the third-party provider to better handle this scenario in the future.

Posted Mar 13, 2025 - 16:58 UTC

Resolved

With no further issues observed, the incident has been resolved. We will follow up soon with a root cause analysis.

If you believe you experienced impact related to this incident, please report them to PubNub Support at support@pubnub.com.

Posted Mar 10, 2025 - 21:38 UTC

Update

We continue to monitor the situation to guarantee that stability is fully restored.

Posted Mar 10, 2025 - 21:08 UTC

Monitoring

We have identified the issue and have taken effective remediation actions, and our engineers are diligently monitoring the situation to guarantee that stability is fully restored.

Posted Mar 10, 2025 - 20:38 UTC

Investigating

At about 6:26 PM UTC, Presence service started to experience elevated latencies and server errors in all PoPs. PubNub Technical Staff is currently investigating and more updates will follow once available.

If you are experiencing issues and believe them to be related to this incident, please report them to PubNub Support at support@pubnub.com.

Posted Mar 10, 2025 - 20:25 UTC

This incident affected: Points of Presence (North America Points of Presence, European Points of Presence, Asia Pacific Points of Presence, Southern Asia Points of Presence) and Realtime Network (Presence Service).