Forums » General

Fiber cut outage, Mark II (No Game Impact).

Apr 03, 2025 incarnate link
Note: The only public-facing impacts of this are the Players Online graph is down, and the Test Server is offline.

We're coming up on 10 hours into another fiber cut, from the same provider as last year (Lumen). Different location, different reasons for the failure, but seemingly it's still taking them forever to fix. This is another "large" capacity outage, impacting way more than just us. This is not a "residential" type outage, but dense wave at a major regional facility (and collapsing another fiber ring).

Anyway, people eventually tend to start asking about the graphs and stuff, so FYI.
Apr 03, 2025 incarnate link
Cut is seemingly resolved and everything's back up.
Apr 04, 2025 Whistler link
I work for a largish healthcare organization. Years ago, when we transitioned to an electronic healthcare record, we contracted with one of the few major companies providing this service. A big deal was made about the installation of TWO data lines so that there would be redundancy in the event of damage to one of the lines. Good thinking, right? Unfortunately, the lines were laid in parallel a few feet apart, and a backhoe working on an unrelated project managed to take them both out in a matter of seconds.

Now the lines come in from opposite compass points.
Apr 04, 2025 incarnate link
Yeah, "rights of way" are limited in the fiber space, and it's extremely common for several providers to be in the same trench together. But, at the same time, they will egregiously lie about their "route diversity" and claim they have all kinds of independent geographical paths to support their rings. Often they're actually leasing glass from one another, and aren't even in a separate cable.

That was what happened in the cut last year, with Spectrum, Lumen and AT&T in the same trench, when the auger drilled through and ripped them all out. AT&T wouldn't even admit that they were there, and refused to explain why they were down, heh.

The irony is that even for your building, there's a good chance they come in from opposite points, and then converge to a single buried-next-to-the-road or aerial (pole) route within a short distance.
Apr 05, 2025 incarnate link
Aaand.. Lumen is down again. Since 7:54am this morning, across the whole region.

No explanation or ETA yet. Kind of unusual to see a major fiber outage on a weekend.
Apr 05, 2025 incarnate link
2025-04-05 19:17:51 GMT - Local provider personnel are on-site, working to prepare the tails of the fiber. Splicers are en route with an ETA of 19:40 GMT. An ETR of 21:00 GMT has been advised.

Fingers crossed.
Apr 05, 2025 incarnate link
Came back online around 6pm.