On the morning of July 22nd, 2024, at approximately 08:25 CET, we began receiving reports from our customers stating that their TOPdesk environments were unavailable or unreachable. An initial check by our Support department indicated that these reports were all originating from customer environments hosted in the NL3 datacenter, specifically within containers 8 and 26.

Upon further investigation, our engineers determined that the reported issues were potentially caused by a misbehaving host machine. We promptly contacted our provider, Leaseweb, to confirm this suspicion. In the meantime, all environments within containers 8 and 26 were failed over to backup infrastructure at approximately 09:15 CET to mitigate the impact on our customers.

In collaboration with Leaseweb, the faulty host machine was placed into maintenance mode. At around 10:15 CET, after careful consideration, the decision was made to move customers back to the main infrastructure in batches while closely monitoring the situation. The final batch was completed at approximately 15:30 CET.

We are pleased to report that the system remained stable throughout the entire restoration process, and the major disruption was resolved by 16:01 CET. The entire disruption, from the initial report to the completion of the restoration process, lasted approximately 7 hours.

Moving forward, we will continue to work closely with our provider, Leaseweb, as we did during this incident. Their prompt response was crucial in mitigating the disruption, and this level of collaboration will be maintained to swiftly address any future disruptions.

Your understanding and patience are greatly appreciated as we continue to work towards providing a more reliable service.