On June 1, 2023, we observed two subscribe latency spikes in our Europe Point of Presence. The spikes occurred from 08:18 AM UTC through 09:06 AM UTC, and from 09:44 AM UTC through 09:54 AM UTC. During these times, users in the Europe region may have experienced slower than normal responses on subscribe calls. The higher than normal latency affected one of multiple access zones in the region. Shortly after detecting the increase in latency, the cause of the issue was identified, and a fix was deployed, restoring the region to normal operational status by 09:54 AM UTC on June 1.
This issue occurred when a code deployment to the region overwrote a configuration previously deployed, resulting in a lack of resources in the access zone.
To prevent a similar issue from occurring in the future, we are applying a fix to all clusters. Additionally, we are improving alerting around publish to subscribe latency so we are quickly notified if a similar issue were to occur.