Core switch line-card failure in ZRH, row C

时间线

1. 已解决2025年3月16日 07:12 UTC

   Every machine in row C has been reachable since 07:12 UTC. Duration of the outage: 50 minutes. The failed card is being replaced under an existing maintenance window; the replacement does not require another interruption. Affected machines receive the SLA extension automatically.

2. 观察中2025年3月16日 06:58 UTC

   All 13 affected ports are up on the surviving card. We are checking each machine individually before closing this.

3. 已确定原因2025年3月16日 06:42 UTC

   A line card in one of the two core switches in ZRH failed. Row C was dual-homed, but the failover on 13 ports did not complete because the standby links were still training. We are forcing those ports over to the surviving card.

4. 调查中2025年3月16日 06:28 UTC

   Machines in row C of ZRH lost network connectivity at 06:22 UTC. Rows outside C are unaffected. Engineers are on the floor.

事故复盘会在解决后 5 天内发布于此。