Diagrams as Code Part 1 — Automatic Diagraming of Cloudflare DNS Zones

I have been been working on Diagrams as Code. As someone who uses diagrams to explain concepts and manage infrastructure this is an area which adds a lot of value for me.

The Concept:

Diagram as Code allows you to track the architecture diagram changes in any version control system.

An article that Sparked my interest:

The Diagrams Library:

Why Diagrams:

  • It’s a python library so I can reuse existing auditing code I have
  • There are a range of layouts and customizations
  • Most of the common Cloud Platforms have icons
  • You can use the Custom Icons for nodes that are not in the library
  • You can create dynamic diagrams to represent your current state.

The working example:

  • I am going to show you how you can use the Diagrams library to automatically generate a diagram of all the A and CNAME records in Cloudflare.
  • I am going to focus on the harder problems using a Cluster concept to group all records that point to one A or CNAME record.
  • The Second concept will be how to create a flow (without knowing ahead of time) by manipulating the Diagrams objects.

Note: for security reasons I have obfuscated the actual records.

Part 1: Getting the Cloudflare Data

This is fairly boiler plate code that returns a list of dictionaries containing the DNS record (eg. a.xyz.com b.xyz.com) and the value of the DNS entry (for A records this is an IPV4 address (e.g. 1.1.1.1)

There are two helper funcitons:

  • is_cname_an_azure_sevice_of_interest — this function narrows down the scope of CNAME records we are looking at to Azure Servcies
  • def hash_dns_records_for_demo — this function hashes the raw names as for demo purposes I don’t want to expose our DNS records and Values
  • I tried using the Cloudflare Python SDK however filtering and documentation were a bit cumbersome so I chose to use requests instead
cloudflare_api_key=”fill me in”cloudflare_email = “change me”zone_id = “use your zone ID here”dns_record_types_of_interest = [“A”,”AAAA”,”CNAME”]azure_service_domain_suffixes = [“azurefd.net”,”azurewebsites.net”,”trafficmanager.net”]def is_cname_an_azure_sevice_of_interest(cname):for suffix in azure_service_domain_suffixes:if suffix in cname:return Truedef hash_dns_records_for_demo(dns_record):return hashlib.sha512(dns_record.encode()).hexdigest()[0:16]headers = {‘X-Auth-Email’: cloudflare_email,‘X-Auth-Key’: cloudflare_api_key,‘Content-Type’: ‘application/json’}url = f”https://api.cloudflare.com/client/v4/zones/{zone_id}/dns_records"def audit_cloudflare_return_data(hash_results=True):a_records = []aaaa_records = []cname_records = []for dns_record_type in dns_record_types_of_interest:query_parameters = {“type”:dns_record_type,”per_page”:100}records_response = requests.request(“GET”, url, headers=headers, params=query_parameters,data={}).json()for dns_record in records_response.get(“result”):if dns_record_type == “CNAME” and is_cname_an_azure_sevice_of_interest(dns_record.get(“content”)):if hash_results:dns_name = hash_dns_records_for_demo(dns_record.get(‘name’))dns_value = hash_dns_records_for_demo(dns_record.get(‘content’))else:dns_name = dns_record.get(‘name’)dns_value = dns_record.get(‘content’)print(f” Found CNAME Record \t DNS Record: {dns_name} \t DNS Value: {dns_value}”)cname_records.append({“name”:dns_name,”value”:dns_value})if dns_record_type == “A”:if hash_results:dns_name = hash_dns_records_for_demo(dns_record.get(‘name’))dns_value = hash_dns_records_for_demo(dns_record.get(‘content’))else:dns_name = dns_record.get(‘name’)dns_value = dns_record.get(‘content’)print(f” Found A Record \t DNS Record: {dns_name} \t DNS Value: {dns_value}”)a_records.append({“name”:dns_name,”value”:dns_value})return a_records,cname_records

This code block when run will return something like this:

Part 2: Grouping Records by A or CNAME Record using Clusters

  • I like using clusters because the Diagrams library is smart enough to know if a cluster with a given label has been previously generated.
  • I am also using a custom Icon
def diagram_cloudflare_data_grouped_by_value(a_records,cname_records):with Diagram(“A Records Grouped By Value”, show=True,direction=”BT”,outformat=”png”):for a_record in a_records:with Cluster(a_record.get(“value”),direction=”TB”):records = [Custom(a_record.get(“name”),icon_path=”.\dns.png”)]with Diagram(“CNAME Grouped By Value”, show=True,direction=”BT”,outformat=”png”):for cname_record in cname_records:with Cluster(cname_record.get(“value”),direction=”LR”):records = [Custom(cname_record.get(“name”),icon_path=”.\dns.png”)]

The output looks like this and gives us some insights

  • In the CNAMEs we can see a few records point to one CNAME (a load balancer perhaps???)
  • In the A Records we can see a similar pattern with one IP being the frontend for multiple records (another load balancer perhaps?)

Part 3: Creating a Record Flow Diagram so I can add more information later

The diagrams in part 2 are useful for insights however the longer term goal is to hunt down the public IPs and CNAME records in Azure. To do this I am going to change the diagram to be a flow so I can expand it later.

  • I need to keep a track of seen A or CNAME records so I don’t add them twice to the diagram
  • I also need to keep a track of the Nodes I create to represent unique A and CNAME records (so I can link them to DNS records later)
  • This part of the code shows that you can use the Diagrams library without explicitly declaring variables.
def diagram_cloudflare_data_grouped_by_flow(a_records,cname_records):seen_records = []node_mapper = {}with Diagram(“A Records Grouped By Flow”, show=True,direction=”LR”,outformat=”png”):confused = Custom(“Where in Azure are they pointing?”,icon_path=”.\problem.png”)for a_record in a_records:if a_record.get(“value”) in seen_records:passelse:record = PublicIpAddresses(a_record.get(“value”))record >> Edge(minlen=”5") >> confusedseen_records.append(a_record.get(“value”))node_mapper[a_record.get(“value”)] = recordfor a_record in a_records:record = Custom(a_record.get(“name”),icon_path=”.\cf-facebook-card.png”)record >> Edge(minlen=”5") >> node_mapper[a_record.get(“value”)]with Diagram(“CNAME Records Grouped By Flow”, show=True,direction=”LR”,outformat=”png”):confused = Custom(“Where in Azure are they pointing?”,icon_path=”.\problem.png”)for cname_record in cname_records:if cname_record.get(“value”) in seen_records:passelse:record = DNSZones(cname_record.get(“value”))record >> Edge(minlen=”5") >> confusedseen_records.append(cname_record.get(“value”))node_mapper[cname_record.get(“value”)] = recordfor cname_record in cname_records:record = Custom(cname_record.get(“name”),icon_path=”.\cf-facebook-card.png”)record >> Edge(minlen=”5") >> node_mapper[cname_record.get(“value”)]

The results look something like this.

  • I can see there are 7 and 9 records pointing to 2 unique CNAMEs (most likely a load balancer)
  • I can also see some other 1 to 1 mappings (web services?)

Todo and Next Steps

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Beat the Java bottleneck

What’s New at Cardstack?

How we are using Slack + Asana to bring our workforce together

Numerical and Categorical Data

JAVA Volatile Variables

How to show your current Firebase project name on the command line prompt to prevent dangerous…

Docker compose with Node.js and MongoDB

Creating a DynamoDB Table Part One

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Andrei Hawke

Andrei Hawke

More from Medium

What is Hashicorp Vault, and why you should care if you are a developer?

G Cloud Backup

G Cloud Backup

How To Setup Nginx Reverse Proxy Server On Ubuntu 20.04

Using Yubico Security Key on MacOS for SSH