SaaS disruption NL3

Incident Report for TOPdesk SaaS Status page

Postmortem

Time line (CEST)

09:15 Received the first tickets.
10:02 Created a ticket with our infrastructure supplier.
10:15 The issue appeared to be caused by two physical hosts.
10:30 Migrated first set of VMs to different physical host.
10:35 Monitoring indicated that host names of TOPdesk environments were reachable.
10:47 Infrastructure Supplier confirms possible hardware issues.
11:05 Migrated second set of VMs to different physical host.
11:30 Migrated firewall node to different host server.
14:30 Finished migrating first set of VMs back to original physical host.
15:50 Finished migrating second set of VMs back to original physical host.
16:24 All physical hosts were available.

Root cause

Memory issues were found on two physical hosts. The memory modules were replaced by our infrastructure supplier.

Identified points of improvement

Investigate if internal major procedure can be improved.
Communicate with our infrastructure supplier to see if monitoring of failing memory modules can be improved.

Posted Nov 25, 2021 - 11:38 CET

Resolved

The disruption on the NL3 hosting location was caused by two physical hosts which malfunctioned. The disruption has been resolved by moving our virtual machines to new physical hosts.

Please contact TOPdesk Support if your TOPdesk environment is still unavailable.

We're still investigating the root cause of this problem and what can be done to prevent this from occurring. When a full Root Cause Analysis is available, we'll update our status page.

Posted Oct 18, 2021 - 12:01 CEST

Update

We have currently mitigated the disruption on the NL3 hosting location. Environments should be available again.
We are monitoring the situation to ensure all environments are running as expected.

Posted Oct 18, 2021 - 11:31 CEST

Update

Our engineers are still investigating the disruption on our NL3 hosting location.
In order to resolve this issue, together with our hosting provider, we are migrating our virtual machines to other physical hosts.

We aim to update this status page every 30 minutes until the issue has been resolved.

Posted Oct 18, 2021 - 11:10 CEST

Update

Our engineers are still investigating the disruption on our NL3 hosting location and are working on a solution.

Posted Oct 18, 2021 - 10:31 CEST

Update

Our engineers are still investigating the disruption on our NL3 hosting location and are working on a solution.

Posted Oct 18, 2021 - 10:03 CEST

Investigating

We are currently experiencing problems on the NL3 hosting location. As a result your TOPdesk environment may not be available.

We are aware of the problem and are working on a solution.

Our apologies for the inconvenience. We aim to update this status page every 30 minutes until the issue has been resolved.

E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates.

To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR21 10 4571.

Posted Oct 18, 2021 - 09:34 CEST

This incident affected: NL3 SaaS hosting location.