RESOLVED: SaaS disruption NL3

Incident Report for TOPdesk SaaS Status page

Postmortem

Compiled RCA from our Hosting Provider:
On 7 February 2024, a report was made regarding an unexpected service disruption on a particular server.
Upon investigation, it was discovered that during a request from the cloud management team to replace the RAM in a piece of equipment, there was a discrepancy in the equipment IDs, resulting in the replacement being conducted on the wrong piece of equipment.
Due to a human error, there was a mismatch in the Equipment ID when requesting the RAM replacement. Following the completion of the replacement on the correct equipment, the server came back online as expected.

Posted Mar 11, 2024 - 14:48 CET

Resolved

At 11:15, our monitoring systems detected multiple environment failures at our NL3 hosting location.
We promptly responded to these alerts and assembled a team to ascertain the root cause.

Upon immediate investigation, it was determined that the underlying issue was due to the physical host being offline.
All impacted environments were efficiently transferred to operational hardware while the affected host underwent a reboot.

Subsequent to this, the hardware resumed functionality and the affected environments were evenly distributed to ensure a balanced load on the hardware.

Furthermore, we've asked our hosting provider to perform a Root Cause Analysis which will be posted on our status page https://status.topdesk.com/ when acquired.

Our apologies for the inconvenience.

Posted Feb 07, 2024 - 13:11 CET

Monitoring

The hardware failure has been fixed and the affected environments should now be returning to a normal state.

Since the hardware issue has been resolved, we will monitor the status of all the affected environments and make sure that everything returns to normal again.

Our apologies for any inconvenience this issue has caused.

Posted Feb 07, 2024 - 11:50 CET

Update

We are currently experiencing an issue in our NL3 hosting location.

The preliminary cause is hardware failure and our Cloud Engineers are working with our hosting provider to restore functionality as quickly as possible

Posted Feb 07, 2024 - 11:30 CET

Investigating

We are currently experiencing problems on our NL3 hosting location. As a result your TOPdesk environment may not be available.

We are aware of the problem and are working on a solution.

Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved.

E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates.

To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR24 02 2111.

Posted Feb 07, 2024 - 11:24 CET

This incident affected: NL3 SaaS hosting location.