Out-of-band network partition in STO
Gestión fuera de banda — STO
Cronología
-
Resuelto
Out-of-band access in STO has been stable since 20:33 UTC. Duration: 25 minutes. Any reinstall or power action queued during the window has been retried and completed.
-
En observación
Consoles are reachable again. We are restoring the state sync between the two firewall members.
-
Causa identificada
The firewall pair in front of the STO management network lost its state synchronisation and both members went passive. We have forced one member active.
-
Investigando
No IPMI or KVM console in STO is reachable since 20:08 UTC. Production traffic is unaffected. Reinstalls and power actions from the customer area will fail for machines in STO until this is fixed.
Revisión posterior al incidente ·
Qué ocurrió y qué cambió
Causa
A firmware update on the management-network firewall pair in STO reset the state-synchronisation key on one member. When the heartbeat failed, both members concluded the other was active and went passive.
Impacto
IPMI, KVM, reinstalls and remote power actions were unavailable in STO for 25 minutes. Production connectivity, workloads and data were unaffected.
Qué cambiamos
- Firewall firmware updates are now applied one member at a time with an explicit failover test between the two.
- The split-brain guard was changed so that a member with a healthy upstream link stays active rather than going passive.
Las horas se muestran en su zona horaria ().
Páginas relacionadas
- Estado del servicioEstado en vivo de gpuserver.io: componentes y centros de datos sondeados cada 60 s desde 5 ciudades, con disponibilidad de 90 días e incidentes desde 2022.
- Historial de incidentesTodos los incidentes y ventanas de mantenimiento de gpuserver.io desde septiembre de 2022, mes a mes, con cronología y revisión posterior al incidente.
- Acuerdo de nivel de servicioEl compromiso del 99,9 %: qué cuenta como inactividad, cómo se mide desde fuera, cinco minutos de plazo por cada minuto perdido, y las exclusiones completas.
- RedPuertos sin contador hasta 25 Gbit/s, dos operadores y un IX por sede, filtrado DDoS permanente, IPv6 enrutado — y las cuatro cosas que no ofrecemos, dichas ya.