1. 20
  1. 3

    …the nodes along the fiber network were so flooded, they could not be reached by their administrators to troubleshoot the issue…

    Does CenturyLink not reserve some bandwidth or have a dedicated network to access their servers, just in case normal bandwidth is saturated for some reason? I’m not a network engineer or system administrator, so correct me if I’m wrong, but this seems like something that would be fairly easy to do.

    1. 8

      The storm was over their own management link, so I think this was their dedicated control network. As to why malformed packets would ever be forwarded onto their management network as they apparently describe, that’s a mystery to me.

      1. 2

        It would seem that the whole issue is that these packets were not “malformed”, hence, they continued to be forwarded around.

        How on earth were they not malformed is a better question. In IPv4 and IPv6, the byte-sized time-to-live (TTL) field is a very standard feature precisely for this reason, and it’ll ensure that these loops aren’t really possible, or, at worst, are self-constrained. Why are they using a protocol without such basic reliability features is, perhaps, the root question I’d have here.

        1. 2

          I was interested in this part too, so I checked the FCC report (linked in the article). Three interesting things I found there - this is a proprietary protocol created by the vendor of the affected nodes, the vendor wasn’t able to explain how the packets were generated and they don’t know which node generated them, and the report says ‘At the time of the outage, the affected network used nodes supplied by Infinera Intelligent Transport Networks (Infinera)’. I don’t want to read too much into that but it sounds like they might not be using that protocol any more.

          1. 2

            I’d be interested in what, if any, benefits this proprietary protocol had over what is commonly available but there is unlikely to be any detailed write up on that.

        2. 1

          Good catch, I didn’t read the article carefully enough.