Something triggered a deadlock/slowlock in the agent scheduling queue. The issue definitely affected HTTP in and out. It might have affected wakeups.
We grabbed a core dump and restarted the daemon. We’ll continue to investigate today.
This incident only affected the developer server. Unfortunately, there are some production devices on that server for historical reasons. We’ll be looking to resolve that today or tomorrow.
We’re also examining why we didn’t get paged when the incident started. It looks like some required monitoring wasn’t in place. We’ll be resolving that this morning.