The incident on March 11th was triggered by an excessive load on the SQL server caused by an inordinate number of queries. This resulted in degraded performance and impacted the environments for other tenants on the NL3C11 hosting location. Customers reported degraded performance around 08:00, and after a thorough investigation, we reset the connection to the SQL server and contacted customers to take necessary measures to prevent similar issues from occurring in the future. We immediately observed the SQL server returning to normal functionality. The major incident was closed at 11:00 after continuously monitoring normal behavior. We have revised our internal procedures regarding investigation to expedite the troubleshooting and resolution of similar incidents. Additionally, we have engaged our TechTeam to implement measures aimed at preventing a recurrence of this issue.
Posted Mar 19, 2024 - 10:07 CET
Resolved
This morning, it was discovered that some environments on our NL3 hosting location were experiencing connection issues. It was quickly discovered that the reason was due to an increased CPU load on one of our containers. The cause of the CPU load was pinpointed and measures were implemented to lower the CPU levels to a normal state and prevent this issue from reoccurring.
We will now conclude that the issue has been resolved. If you still experience issues, please reach out to our Support department.
Posted Mar 11, 2024 - 11:24 CET
Monitoring
Our monitoring has now shown that CPU levels have returned to normal and connections to environments should be returning.
We will continue to monitor the situation and investigate the cause. We will update our statuspage (https://status.topdesk.com/) when the reason has been uncovered.
Our apologies for any inconvenience this issue may have caused.
Posted Mar 11, 2024 - 10:07 CET
Update
It was discovered that the CPU usage on one of our SQL servers was overloaded, resulting in failed connections to environments.
We are currently investigating the cause.
Posted Mar 11, 2024 - 09:51 CET
Investigating
We are currently experiencing issues on our NL3 hosting location, as a result environments may not be reachable.
We are aware of the problem and are investigating the cause.