Skip to content

Turning CLI output into structured data

Several articles on this site reach for the same trick without dwelling on it: the Python article passes use_textfsm=True to turn show output into a list of dictionaries, and change validation with pyATS leans on Genie to parse a command into a dictionary you can assert on. This article pulls that thread into one place — how to turn a device's human-readable CLI output into data a program can act on, which tool to use for it, and, just as important, when you shouldn't be parsing CLI at all.

Why raw CLI output is a liability

show commands were designed to be read by a person at a terminal, not consumed by a program. The columns line up, the headers are words, and the layout can change between software releases. A script that pulls a value out of that text with string slicing or a hand-rolled regular expression is brittle: it works until the vendor adds a column, renames a field, or wraps a long line, and then it silently returns the wrong thing. Fragile parsing is a leading cause of the kind of confident, wrong output covered in diagnosing failures from logs and output.

The fix is to hand the parsing to a maintained library that already knows the shape of each command's output and returns a predictable data structure — a list of dictionaries or a nested dictionary — that you index by field name instead of by character position. Three libraries dominate, and they sit at different points on the effort/coverage curve.

TextFSM and the NTC templates

TextFSM is Google's template-based text parser. You describe the output once as a small state machine of named values and regular expressions, and TextFSM turns any matching text into rows of records. You rarely write those templates yourself: ntc-templates is a community library of over a thousand ready-made templates indexed by platform and command, and Netmiko ships with it. Passing use_textfsm=True to send_command looks up the right template and returns structured data instead of a string:

from netmiko import ConnectHandler

device = {"device_type": "cisco_ios", "host": "192.0.2.11",
          "username": "automation", "password": PASSWORD}

with ConnectHandler(**device) as conn:
    facts = conn.send_command("show version", use_textfsm=True)

# facts is a list of dicts, e.g. [{'version': '17.9.1', 'serial': ['FXS...'], ...}]
print(facts[0]["version"])

Two things to know. First, if no template matches the platform/command pair, Netmiko falls back to returning the raw string — so code that expects a list has to handle a str too, or it will break on the first unsupported command. Second, you can point Netmiko at your own template directory with the NET_TEXTFSM environment variable (it must contain an index file) when you need a template the community library doesn't ship.

Genie parsers

Genie, the parsing companion to pyATS, takes the same idea further. It ships over a thousand parsers that normalise output to OS-agnostic data models, so show ip interface brief returns the same dictionary shape whether the box runs IOS XE, NX-OS, or IOS XR — you assert on fields, not on raw text. The quickest way in is the pyats parse CLI against a testbed file:

pyats parse "show ip interface brief" --testbed-file testbed.yaml --devices r1

{
  "interface": {
    "GigabitEthernet1": { "ip_address": "10.0.0.1", "status": "up", "protocol": "up" },
    "GigabitEthernet3": { "ip_address": "unassigned", "status": "down", "protocol": "down" }
  }
}

From Python, every device in a loaded testbed has a .parse() method returning that same dictionary, which is exactly what makes scripted validation readable — you check a value in a dict instead of grepping a string:

from genie.testbed import load

tb = load("testbed.yaml")
dev = tb.devices["r1"]
dev.connect()

parsed = dev.parse("show ip interface brief")
if parsed["interface"]["GigabitEthernet1"]["status"] != "up":
    raise SystemExit("GigabitEthernet1 is down")

Genie's deeper coverage is why pyATS's learn/diff change validation works: it can parse whole features into a structured snapshot and diff two of them. (Netmiko can also invoke Genie with use_genie=True if you'd rather drive it through a Netmiko session.)

NAPALM getters: structured by contract

NAPALM comes at the problem from the other direction. Instead of parsing arbitrary commands, it defines a fixed set of getters — get_facts, get_interfaces, get_bgp_neighbors, and so on — each with a documented, vendor-neutral return schema. The same call returns the same keys on every supported platform:

from napalm import get_network_driver

driver = get_network_driver("ios")
dev = driver(hostname="192.0.2.11", username="automation", password=PASSWORD)
dev.open()

facts = dev.get_facts()   # same keys on IOS XE, EOS, JUNOS, ...
print(facts["os_version"], facts["serial_number"])
dev.close()

get_facts() always returns the same keys — hostname, vendor, model, serial_number, os_version, uptime, interface_list — so a cross-vendor inventory or asset-management job is the same code everywhere. The trade-off is coverage: a getter only exists for data NAPALM has modelled, and support varies by driver, so check each platform's support matrix before relying on one. Where it fits, it's the least fragile of the three, because you're coding against a contract rather than a template.

The best parse is no parse

Every option above is still, underneath, extracting data from text meant for humans — and CLI output is unstructured by design, so a template or parser can break when the format shifts. When the device can give you the data already structured, take it. RESTCONF, NETCONF, and gNMI return YANG-modelled JSON or XML straight from the box — no template to maintain, no column to misread, and a schema that defines every field. A RESTCONF GET hands you a dictionary directly; there is nothing to parse.

The practical rule: prefer model-driven data (RESTCONF/NETCONF/gNMI) when the platform and feature support it, and fall back to CLI parsing only for legacy gear or the features that still appear in the CLI first. Screen-scraping isn't wrong — plenty of production automation depends on it — but it should be the fallback, not the default.

Making parsing robust

When you do parse CLI, treat it like any other fragile boundary:

  • Handle the fallback. Expect a raw string when no template matches, rather than assuming a list or dict always comes back.
  • Assert on fields, not text. The whole point is to check row["status"], never to grep the original output again.
  • Pin your versions. ntc-templates and Genie parsers evolve; pin them (see secure coding practices) so a dependency bump can't silently change your parsed shape.
  • Expect parser bugs. Both ntc-templates and Genie are maintained libraries — a command that won't parse is often a template/parser bug to report upstream, not a flaw in your code.
  • Test against captured output. Save real show output as a fixture and parse it offline in CI, so you catch a broken parse without touching a live device.

Key takeaways

  • Raw CLI output is unstructured text for humans; parsing it by hand is brittle. Hand the job to a maintained library that returns data you index by field name.
  • TextFSM + ntc-templates (via Netmiko use_textfsm=True) gives per-command templates and the widest community coverage; it falls back to a raw string when no template matches.
  • Genie parses to OS-agnostic models and powers pyATS validation; pyats parse or a device's .parse() returns a structured dictionary.
  • NAPALM getters return a fixed, vendor-neutral schema — the least fragile option where a getter exists, bounded by per-driver support.
  • The best parse is no parse: prefer YANG-modelled data over RESTCONF/NETCONF/gNMI when the device supports it, and keep CLI parsing as the fallback for legacy or CLI-first features.

Sources: TextFSM, ntc-templates, Netmiko, Genie / pyATS parsers, NAPALM supported getters, Ivan Pepelnjak — Using Screen Scraping in Network Automation.