Around 14:45 UTC, March 1, 2024, we observed the Presence service started experiencing missed Presence webhook calls. The missed webhook calls were due to a migration of the Presence webhooks across multiple customers. We rolled back the migration for all customers in case the issue was broader and created a status page for transparency. Further investigation showed the missed webhook calls were isolated to a small subset of customers.
To prevent a similar issue from occurring, we added and deployed redirect functionality in our Events & Actions service and added monitoring for future webhook migrations.