Environments hosted on our US datacenter unreachable
Incident Report for TOPdesk SaaS Status page
Postmortem

During a scheduled security/firewall configuration update, it was discovered that the interface configuration of a critical network component was incorrect, resulting in unexpected behavior that left all environments inaccessible at our US1 hosting location. 

We checked all other hosting locations for similar misconfigurations and found none. 

The configuration update was implemented in at 10:12 CET, and by 10:25 CET, we initiated an investigation into the root cause. The corrective action was implemented at 11:00 CET, at which point the incident was transitioned to monitoring status.

Only a few customers reported issues with their environment being down, and this can be primarily attributed to the configuration occurring within the maintenance window of the US1 hosting location.

We apologize for the inconvenience.

Posted Nov 22, 2023 - 16:20 CET

Resolved
Earlier today, a configuration error impacting the traffic routing towards the US1 datacenter was identified by our engineers. This caused environments to be unreachable. This configuration error was swiftly resolved, and all US1 environments are accessible again. If you experience any difficulties, don't hesitate to get in touch with TOPdesk Support.

We will provide an RCA once the evaluation is completed.
Posted Nov 17, 2023 - 11:47 CET
Monitoring
We've successfully identified and resolved a configuration error.

Environments are now reachable again.

Our team will continue to monitor the situation closely, and the next update is scheduled in 30 minutes.

Thank you for your patience and understanding.
Posted Nov 17, 2023 - 11:12 CET
Investigating
We are currently experiencing problems on our US hosting location

As a result your TOPdesk environment may not be available.

We are aware of the problem and are working on a solution.

Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved.

E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates.

To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR23 11 4944.
Posted Nov 17, 2023 - 10:48 CET
This incident affected: US1 SaaS hosting location.