Of Course the Network Diagrams are Bad!

As security professionals we know network diagrams are critical to providing security.  It's the top control in the SANS CIS CSC top 20 controls. Yet, almost every organization we go to has network diagrams that are convoluted, out-of-date, missing things, or just plain wrong.  Our pen tests produce better network diagrams than what the organization has in the span of the engagement.  Why is that?

1. Laying out networks is HARD!  Networks are really just graphs (in the mathematical sense) and graph layout consumes a lot of trees in the pursuit of academic publication.  Honestly, I have been laying graphs out for years, up to the 100's of 1,000's of nodes (where visualizations tools tend to top out) and  any graph with over just a dozen or two nodes is no longer self-explanatory.  Just look at Figure 15 in the PHIDBR.  It's pretty, but let's be honest; you can't really draw any conclusions simply by looking at the graph.
2. In reality, you'd need an artist.  Someone skilled in data visualization and with good artistic prowess to build useful network diagrams.  Yet, how's that going to work?  Do you hire an artist who knows your network?  Do you train your network guys in visual arts?  Do you hire a full time position simply to draw beautiful network diagrams?
3. And the network is always changing.  Those diagrams are likely to be obsolete as soon as they are completed.  Does the artist maintain them?  Do you hold back network changes for updates to the network diagram?
4. And even if you do get a good set of network diagrams that your artist-in-residence keeps up-to-date, what level of detail are you creating them at? Are you creating block diagrams that generally show the top level of the system in the abstract?  Are you creating wiring diagrams for the racks down to the power and ground cables?  Are you creating every potential view in the DODAF?  The reality is when people say "Show me the network diagram." what they mean is "Show me the network diagram showing me exactly the things I'm interested in at the level of detail that I think is correct but that I have never communicated."
5. And none of this even begins to touch on the issue of determining ground-truth in how your network is connected.  It's hard when you know all the devices and can dump all the configurations.  It's damn near impossible in a practical network when people add things without saying, and use equipment that is not centrally managed.

The reality is it'd be surprising to find someone keeping great network diagrams, simply because of the amount of effort involved.  There are automated tools to help, but if a human can't easily make the network visually understandable, the software is not going to do better.  Also, the software suffers from the same problems related to level-of-detail, being up-to-date, and accurately discovering the true network that a person manually doing the job would.

So are there solutions?  I don't know.  Probably not.  I think that a real-time interactive visualization system rather than static pieces of paper is better.  A system designed with a certain amount of artificial intelligence to learn and explore the network would probably help.  However, we simply may need to accept that we won't know our network fully and that the situation is more like:

*

Knowing this, sympathize with organizations doing their best, and help plan a defense that accepts this reality.

* I saw this on twitter but can't find it again.  If anyone has proper attribution I'd be happy to add it.