The remediation is complete; app and account day and month aggregates should now be correct again. Apologies again for the inconvenience.
(For nearly all customers at least; for customers on enterprise dedicated clusters it might take a few more hours before the month aggregrate for the current month shows correctly).
Posted Sep 25, 2024 - 18:49 UTC
Identified
We have been alerted to an issue with stats aggregation (app stats through our stats api and account stats through the website dashboard) since 8th September that can lead to messages being double-counted in day and month aggregates.
We have identified the problem and are in the process of remediating it. The issue only affects day and month aggregates; minute and hour aggregates are correct.
We will be remediating day and month aggregates for all accounts and apps from the known-good data (the hour aggregates). We hope to have this done by the end of the 25th. Since billing is done at the end of the month, we do not expect any customers to be billed incorrectly as a result of this.
In the meantime, we advise anyone using the stats api to query for hourly stats instead of daily or monthly. We realise this is inconvenient, and deeply apologise, especially for how long it took us to notice the issue.
If you have any further questions or concerns, please contact our support team at https://ably.com/support .
Posted Sep 24, 2024 - 17:35 UTC
This incident affected: Ably Global Service Availability, Ably Control-API, and Ably Website.