Performance Degradation Compute FRA
Incident
Report for IONOS Cloud
Resolved
We have successfully completed the rollout to all remaining hosts and are closing this incident. A Root Cause Analysis is currently being conducted by the Compute Team and will be shared here upon completion.
Update
Our Compute Team has confirmed that the fix has been rolled out to the majority of affected hosts. We are currently finishing the rollout and will provide an update once the remaining hosts on affected clusters are covered
Update
The second configuration update rollout is currently in progress, and we have confirmed initial improvements related to CPU performance. Due to the size of the fleet, we expect the rollout to take some time to complete. Throughout the process, customers will see performance gains as soon as the specific hosts supporting their workloads have been updated. We will provide a final update once the rollout is finished.
Monitoring
The adjustment was rolled out. Our Compute team is seeing dropping CPU steal time. We are monitoring the situation. Our Tech Teams are preparing another rollout that should improve the performance further.
Update
Our compute team has successfully tested the proposed fix for the CPU core affinity and is preparing a rollout. We will monitor the results.
Update
Our compute team has found another factor negatively impacting CPU performance for affected VMs. We are currently testing a potential transparent resolution for the problematic CPU affinity setting.
Identified
We have identified an increase in CPU steal time on affected hosts. Our Compute team has identified a likely culprit and is currently testing a potential mitigation to ensure its effectiveness before a rollout.
Investigating
We are currently investigating performance degradation affecting compute components in our FRA DC. This issue is impacting a subset of Virtual Machines (VMs) and Kubernetes Clusters. We will provide further updates as our investigation progresses.
This incident affected: Location DE/FRA (Compute, Managed Kubernetes).