Update - We are monitoring the situation and we see no problems with performance or the reporting service.
We are planning to close the major around noon. If you encounter any problems, please update your ticket.
Sep 25, 10:09 CEST
We are planning to close the major around noon. If you encounter any problems, please update your ticket.
Sep 25, 10:09 CEST
Update - Our SaaS Operations team has been working on a fix. Currently the performance is back to what it should be and we are closely monitoring the Duration Reports & Odata feed functionalities. Tomorrow we will evaluate the situation and determine the strategy for the next few days.
We ask you to not use any heavy Odata feed extractions until we can safely say that the fix works.
We strive to have all functionalities fully operational with solid performance by Monday. However we are cautious and cannot guarantee this yet.
Sep 24, 17:10 CEST
We ask you to not use any heavy Odata feed extractions until we can safely say that the fix works.
We strive to have all functionalities fully operational with solid performance by Monday. However we are cautious and cannot guarantee this yet.
Sep 24, 17:10 CEST
Update - Hereby more information regarding the current problems with performance and the reporting (Odata) services.
The root cause of the problem is that the virtual machines which are hosting our new services architecture have become overloaded. The is due to an increase in the overall traffic and an increase in the usage of the reporting layer. To remedy the performance issues we have temporarily disabled the reporting service (Duration reporting and the Odata feed).
We realize that the Odata feed can be a vital part of your process and therefor we are researching if we can reactivate the reporting service.
For a long term solution we will evaluate the resources allocation in the new services architecture, more on this will follow in the RCA.
Sep 24, 13:55 CEST
The root cause of the problem is that the virtual machines which are hosting our new services architecture have become overloaded. The is due to an increase in the overall traffic and an increase in the usage of the reporting layer. To remedy the performance issues we have temporarily disabled the reporting service (Duration reporting and the Odata feed).
We realize that the Odata feed can be a vital part of your process and therefor we are researching if we can reactivate the reporting service.
For a long term solution we will evaluate the resources allocation in the new services architecture, more on this will follow in the RCA.
Sep 24, 13:55 CEST
Update - Our SaaS Operations team continues to work on the problem.
It seems that multiple causes have put a too high load on the hardware of the services architecture.
Currently we are in contact with the supplier of the hardware to find a fitting solution.
Later today we will publish a more detailed update on the situation, the cause and the steps we will take to remedy this.
Sep 24, 10:55 CEST
It seems that multiple causes have put a too high load on the hardware of the services architecture.
Currently we are in contact with the supplier of the hardware to find a fitting solution.
Later today we will publish a more detailed update on the situation, the cause and the steps we will take to remedy this.
Sep 24, 10:55 CEST
Update - The SaaS Operations team has decided to disable the Odata feed for a longer period of time. We will enable the Odata feed as soon as possible but stability and performance have our priority. When we have an estimated time we will post this on the status page.
Along with closely monitoring the situation, our Operations team has contacted our supplier to make sure that the SaaS environments will remain stable after we enable all the features.
Sep 23, 17:22 CEST
Along with closely monitoring the situation, our Operations team has contacted our supplier to make sure that the SaaS environments will remain stable after we enable all the features.
Sep 23, 17:22 CEST
Monitoring - Our SaaS team has been closely monitoring the resources on the NL3 data center. We have implemented a temporary fix and the performance should be back to normal.
A part of this temporary fix is to disable the reporting service. The reporting service is part of the new architecture and will affect Duration reports and Odata reports, the standard reports in TOPdesk will continue to work.
We are planning to enable the reporting service again after working hours with reduced resources.
A long term solution will involve changes made by our development teams, these changes will take more time and will be mentioned in the RCA.
Sep 23, 13:05 CEST
A part of this temporary fix is to disable the reporting service. The reporting service is part of the new architecture and will affect Duration reports and Odata reports, the standard reports in TOPdesk will continue to work.
We are planning to enable the reporting service again after working hours with reduced resources.
A long term solution will involve changes made by our development teams, these changes will take more time and will be mentioned in the RCA.
Sep 23, 13:05 CEST
Update - We are closely monitoring the services and the resources and we are seeing that the impact on performance has been reduced. We will continue to monitor the situation.
We are not enabling the reporting service yet (for Duration reports and the Odata feed), we want to make sure that the issues are resolved before enabling these services. Later today we will post a decision regarding the re-enabling of the reporting service.
Sep 23, 11:06 CEST
We are not enabling the reporting service yet (for Duration reports and the Odata feed), we want to make sure that the issues are resolved before enabling these services. Later today we will post a decision regarding the re-enabling of the reporting service.
Sep 23, 11:06 CEST
Update - The SaaS Operations team is closely monitoring the resources of the services.
Within an hour we will give an update.
Sep 23, 09:56 CEST
Within an hour we will give an update.
Sep 23, 09:56 CEST
Identified - A summary of the problems we have been having on our NL3 data center:
On Monday we noticed one of our services was requiring a lot more resources than normal, which negatively affected the performance of your TOPdesk environment. During the day it seemed like the problems were hardware related and the major incident was closed.
Later on this same day the problems started coming back around noon. It was not clear whether the issues had the same cause, thus we created a new major incident. We investigated until the end of the day and continued to monitor the resources.
On Tuesday around peak times (10:00 AM) we noticed the same problems with the services and the resources. A more in depth investigation found a likely culprit, the reporting service. We planned to disable this service on Wednesday during the next peak times, however later on this Tuesday the problems came back.
Around 14:00 our SaaS Operations team noticed the issues were affecting the authentication service. To resolve this, resources have been re-balanced to have this key service function again. Furthermore, we have disabled the reporting service for now. This means that Duration reports as well as the Odata feed will not work at this moment.
The plan to monitor the new situation closely on Wednesday morning still stands, we will closely monitor the resources and we will determine a strategy to bring this service back online as soon as possible with stability as the most important factor.
Sep 22, 17:02 CEST
On Monday we noticed one of our services was requiring a lot more resources than normal, which negatively affected the performance of your TOPdesk environment. During the day it seemed like the problems were hardware related and the major incident was closed.
Later on this same day the problems started coming back around noon. It was not clear whether the issues had the same cause, thus we created a new major incident. We investigated until the end of the day and continued to monitor the resources.
On Tuesday around peak times (10:00 AM) we noticed the same problems with the services and the resources. A more in depth investigation found a likely culprit, the reporting service. We planned to disable this service on Wednesday during the next peak times, however later on this Tuesday the problems came back.
Around 14:00 our SaaS Operations team noticed the issues were affecting the authentication service. To resolve this, resources have been re-balanced to have this key service function again. Furthermore, we have disabled the reporting service for now. This means that Duration reports as well as the Odata feed will not work at this moment.
The plan to monitor the new situation closely on Wednesday morning still stands, we will closely monitor the resources and we will determine a strategy to bring this service back online as soon as possible with stability as the most important factor.
Sep 22, 17:02 CEST
Update - We have now briefly experienced problems with the authentication service, this will have resulted in temporary downtime.
In order to swiftly react we are disabling the service responsible for reporting. This means that the duration reporting functionalities will not be working in order to make sure that our other services stay online. The standard TOPdesk reports will continue to work.
We will update you with a more detailed plan.
Sep 22, 14:55 CEST
In order to swiftly react we are disabling the service responsible for reporting. This means that the duration reporting functionalities will not be working in order to make sure that our other services stay online. The standard TOPdesk reports will continue to work.
We will update you with a more detailed plan.
Sep 22, 14:55 CEST
Update - Our SaaS operations team suspects that the problems from today are originating from specific key services which are affecting the performance.
If you have information about a certain time period of degraded performance, please let us know by updating your ticket so we can correlate this to our monitoring.
Sep 22, 11:57 CEST
If you have information about a certain time period of degraded performance, please let us know by updating your ticket so we can correlate this to our monitoring.
Sep 22, 11:57 CEST
Update - The performance problems which started yesterday (2020-09-21) still appear to affect environments on the NL3 data center.
We will contact the callers of newly created tickets to investigate the cause of the problems.
If your environment is affected, please create a new ticket from the major on my.topdesk.com.
Sep 22, 10:53 CEST
We will contact the callers of newly created tickets to investigate the cause of the problems.
If your environment is affected, please create a new ticket from the major on my.topdesk.com.
Sep 22, 10:53 CEST
Update - We are continuing to investigate this issue.
Sep 21, 18:22 CEST
Sep 21, 18:22 CEST
Update - We are continuing to investigate this issue.
Sep 21, 17:54 CEST
Sep 21, 17:54 CEST
Update - To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR20 09 6276.
Sep 21, 16:33 CEST
Sep 21, 16:33 CEST
Investigating - A few customers in our NL3 hosting location reported performance issues. We're investigating if a common cause can be found for these problems.
It is different from the problem experienced this morning and we expect to give more updates as soon as we have more information or find out the root cause of it.
Sep 21, 16:20 CEST
It is different from the problem experienced this morning and we expect to give more updates as soon as we have more information or find out the root cause of it.
Sep 21, 16:20 CEST
About This Site
This page shows the current status of our TOPdesk SaaS infrastructure.
You can subscribe for updates for your relevant hosting locations at the top right of the page. To see which hosting location is used for your environment, see the SaaS hosting details on My TOPdesk.
This page displays disruptions affecting groups of customers. For detailed information regarding your TOPdesk SaaS environment, see the My TOPdesk monitoring page.
Also see
TOPdesk Support contact details | TOPdesk SaaS information website | TOPdesk product release notes
NL2 SaaS hosting location
?
Operational
NL3 SaaS hosting location
?
Operational
US1 SaaS hosting location
Operational
UK1 SaaS hosting location
Operational
CA1 SaaS hosting location
Operational
AU1 SaaS hosting location
?
Operational
EU1 SaaS hosting location
Operational
BR1 SaaS Hosting location
Operational
My TOPdesk
?
Operational
General notifications
?
Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Scheduled Maintenance
Hosting location NL2 will be deprecated
Jul 12, 23:00 - Jul 13, 02:00 CEST
As part of our commitment to continuously improve the level of service we provide here at TOPdesk, we would like to inform you about an upcoming move of TOPdesk environments in the NL2 hosting location.
The infrastructure used to host environments in the NL2 datacenter no longer meets our demands, and with the introduction of new datacenters in the Netherlands we've improved our services and management options. To continue providing the best possible service to all TOPdesk SaaS customers, we've decided to deprecate the NL2 hosting location.
Environments hosted on the NL2 datacenter will be relocated to our recently deployed EU1 hosting location (Netherlands based). For more details about the hosting locations that are available, see https://page.topdesk.com/saas-information#hostinglocations
With this move to the EU1 location you will get access to new functionalities including Smart Categorization, Call Affiliation, OData reporting, Chat as well as a number of additions within TOPdesk Labs. The new datacenter set-up provides improved stability and enhanced security.
Communication and planning
We will inform the SaaS Main Contact(s) of all customers with an environment in the NL2 datacenter by e-mail with the exact date of the move later on.
The planning is currently:
12th - 24th July: We will move all TOPdesk Lite environments and start moving all TOPdesk test environments to EU1.
26th July - end of August: We will move all TOPdesk Professional and TOPdesk Enterprise Production environments to EU1.
We will require your help in order to make this transition as smooth as possible. Please carefully read our knowledge item about preparation steps before moving to a new datacenter; KI 12978.
https://my.topdesk.com/tas/public/ssp/content/detail/knowledgeitem?unid=9a312e94181d48e0815acb8e2d69d098
If you'd like your TOPdesk environment to be moved to a different hosting location, or if you have any remaining questions about this upcoming move, please contact TOPdesk support.
Posted on Jun 18, 16:14 CEST
The infrastructure used to host environments in the NL2 datacenter no longer meets our demands, and with the introduction of new datacenters in the Netherlands we've improved our services and management options. To continue providing the best possible service to all TOPdesk SaaS customers, we've decided to deprecate the NL2 hosting location.
Environments hosted on the NL2 datacenter will be relocated to our recently deployed EU1 hosting location (Netherlands based). For more details about the hosting locations that are available, see https://page.topdesk.com/saas-information#hostinglocations
With this move to the EU1 location you will get access to new functionalities including Smart Categorization, Call Affiliation, OData reporting, Chat as well as a number of additions within TOPdesk Labs. The new datacenter set-up provides improved stability and enhanced security.
Communication and planning
We will inform the SaaS Main Contact(s) of all customers with an environment in the NL2 datacenter by e-mail with the exact date of the move later on.
The planning is currently:
12th - 24th July: We will move all TOPdesk Lite environments and start moving all TOPdesk test environments to EU1.
26th July - end of August: We will move all TOPdesk Professional and TOPdesk Enterprise Production environments to EU1.
We will require your help in order to make this transition as smooth as possible. Please carefully read our knowledge item about preparation steps before moving to a new datacenter; KI 12978.
https://my.topdesk.com/tas/public/ssp/content/detail/knowledgeitem?unid=9a312e94181d48e0815acb8e2d69d098
If you'd like your TOPdesk environment to be moved to a different hosting location, or if you have any remaining questions about this upcoming move, please contact TOPdesk support.
Posted on Jun 18, 16:14 CEST