This incident has been resolved, and platform metrics have returned to normal. Some store-specific queues may still need time to fully catch up. If you need a hand, email team@usemechanic.com. ❤️❤️
Posted Mar 09, 2024 - 19:27 UTC
Update
Store-level queues continue to recover normally.
Posted Mar 09, 2024 - 18:04 UTC
Update
The following message is a duplicate, to ensure that status page subscriber notifications are sent.
A fix has been introduced, and runs are flowing once more. Many stores will experience a backlog of their own events; as the backlog clears, store-level throughput will return to normal. Stores with less active event streams will recover much more quickly, and some are already back to normal.
For context and clarity, this issue arose ~5hrs ago, and resulted in a complete cessation of runs for those 5 hours. Yes, we are learning from this. I'm really sorry, y'all. -Isaac
Posted Mar 09, 2024 - 17:29 UTC
Monitoring
A fix has been introduced, and runs are flowing once more. Many stores will experience a backlog of their own events; as the backlog clears, store-level throughput will return to normal. Stores with less active event streams will recover much more quickly, and some are already back to normal.
For context and clarity, this issue arose ~5hrs ago, and resulted in a complete cessation of runs for those 5 hours. Yes, we are learning from this. I'm really sorry, y'all. -Isaac
Posted Mar 09, 2024 - 17:26 UTC
Identified
The issue has been identified and a fix is being implemented.