Core switch line-card failure in REY, row B

时间线

1. 已解决2026年3月13日 22:22 UTC

   Every machine in row B has been reachable since 22:22 UTC. Duration of the outage: 54 minutes. The failed card is being replaced under an existing maintenance window; the replacement does not require another interruption. Affected machines receive the SLA extension automatically.

2. 观察中2026年3月13日 22:13 UTC

   All 14 affected ports are up on the surviving card. We are checking each machine individually before closing this.

3. 已确定原因2026年3月13日 21:50 UTC

   A line card in one of the two core switches in REY failed. Row B was dual-homed, but the failover on 14 ports did not complete because the standby links were still training. We are forcing those ports over to the surviving card.

4. 调查中2026年3月13日 21:31 UTC

   Machines in row B of REY lost network connectivity at 21:28 UTC. Rows outside B are unaffected. Engineers are on the floor.

事故复盘会在解决后 5 天内发布于此。