Skip to content

Scaling network automation with concurrency

The Python article ends on a problem it doesn't fully solve: a loop that logs into devices one at a time is fine for ten switches and painfully slow for five hundred. This article is about closing that gap — why the fix is concurrency rather than faster code, how to fan a job out across a fleet with a thread pool, and how to do it without turning a small mistake into a large outage.

Why a serial loop doesn't scale

Walk through what a for loop spends its time on. Opening an SSH session, sending a command, and waiting for the device to answer are almost entirely waiting — the control node's CPU sits idle while packets cross the network and the box does its work. A single show version might take a second of wall-clock time but a millisecond of CPU. Run that serially across 500 devices and the job takes roughly 500 seconds, almost none of it productive.

The workload is I/O-bound: the bottleneck is network round-trips, not computation. That matters because the fix for an I/O-bound job is not a faster language or a tighter loop — it's doing the waiting in parallel. If each device-call blocks on the network, you want many of those waits happening at once.

Threads, because the work is I/O-bound

Python's global interpreter lock (GIL) means threads don't help a CPU-bound job much, which is why threading has a mixed reputation. But blocking I/O calls — a socket read in Netmiko, an HTTP request in requests — release the GIL while they wait, so other threads run during that idle time. For network automation that is exactly the shape of the work, so a thread pool is usually all you need. You don't reach for multiprocessing (built for CPU-bound work, and heavier) or rewrite everything around asyncio (which needs async-native libraries; Netmiko and requests are synchronous).

A thread pool with concurrent.futures

The standard-library tool is concurrent.futures.ThreadPoolExecutor. It manages a fixed set of worker threads and hands them your tasks. The simplest form maps a function over an iterable of hosts and collects the results in order:

from concurrent.futures import ThreadPoolExecutor
from netmiko import ConnectHandler

def grab(host):
    with ConnectHandler(device_type="cisco_ios", host=host,
                        username=USER, password=PASSWORD) as conn:
        return host, conn.send_command("show version", use_textfsm=True)

with ThreadPoolExecutor(max_workers=20) as pool:
    for host, facts in pool.map(grab, inventory):
        save(host, facts)

Twenty devices are now queried at once instead of one, and the job that took minutes finishes in seconds. Note the credentials come from variables read from the environment, not hard-coded — the same rule the secure coding practices article makes, and one that matters more here because a leaked script now reaches the whole fleet.

One thing not to rely on: the pool's default size. If you omit max_workers, current CPython uses min(32, os.cpu_count() + 4) — a value tuned for light I/O overlap, not for fanning out hundreds of long-lived SSH sessions. Set it explicitly for device work.

Isolate failures so one bad device doesn't sink the run

pool.map has a sharp edge: it yields results in input order, and if one call raised, the exception surfaces when you reach that item in the loop — stopping you from collecting the results that did succeed. Across 500 devices, one unreachable box should not cost you the other 499.

The robust pattern is submit plus as_completed: submit every task, then process each future as it finishes, calling .result() inside a try/except so a failure is recorded against its host instead of propagating:

from concurrent.futures import ThreadPoolExecutor, as_completed
from netmiko import (ConnectHandler, NetmikoTimeoutException,
                     NetmikoAuthenticationException)

def grab(host):
    with ConnectHandler(device_type="cisco_ios", host=host,
                        username=USER, password=PASSWORD) as conn:
        return conn.send_command("show version", use_textfsm=True)

results, errors = {}, {}
with ThreadPoolExecutor(max_workers=20) as pool:
    future_to_host = {pool.submit(grab, h): h for h in inventory}
    for future in as_completed(future_to_host):
        host = future_to_host[future]
        try:
            results[host] = future.result()
        except (NetmikoTimeoutException, NetmikoAuthenticationException) as exc:
            errors[host] = str(exc)

print(f"{len(results)} ok, {len(errors)} failed")

as_completed yields futures in the order they finish, not the order you submitted, which also lets you report progress as the fleet responds. Catching NetmikoTimeoutException (reachability) and NetmikoAuthenticationException (credentials) separately mirrors the failure signatures in diagnosing failures from logs and output — the error map tells you whether to retry or stop.

Sizing the pool and bounding the blast radius

Concurrency is a throttle, and the right setting is rarely "as high as possible." Several real limits cap it from below the CPU count:

  • The devices. A switch has a finite number of VTY lines; open too many sessions and new ones are refused.
  • The controller or API. Hitting a DNA Center or Meraki endpoint with 100 parallel requests invites 429 Too Many Requests. Pair a bounded pool with the backoff discipline from consuming REST APIs at scale.
  • Your own change-control. This is the one that bites hardest. Twenty workers pushing config means a bad change lands on twenty devices before you notice.

So concurrency for reads can be generous; concurrency for writes must be deliberate. Prove the change on a canary first — ideally in a CML lab, then on one production device — and roll the rest out in batches, validating after each:

def in_batches(seq, size):
    for i in range(0, len(seq), size):
        yield seq[i:i + size]

for batch in in_batches(production, size=10):
    deploy(batch)      # push concurrently within the batch
    verify(batch)      # pyATS health check before the next batch

Because the change is idempotent and already proven, a batch that fails verification stops the rollout with most of the fleet untouched — speed you can actually afford.

When to reach for a framework

Hand-rolling a thread pool is fine, and it keeps you in plain Python. But collecting per-host results and errors, tagging them, and feeding an inventory is boilerplate that a framework already solves. Nornir is the common choice: it pairs a pluggable inventory with a threaded runner whose num_workers (default 20) is the same knob as max_workers, and it returns a results object keyed by host so success and failure are structured for you. If you genuinely need thousands of concurrent sessions, an async transport such as Scrapli's asyncio driver scales further than threads — but for most fleets, a bounded ThreadPoolExecutor or Nornir is the right amount of machinery.

Key takeaways

  • Network automation is I/O-bound — a serial loop wastes almost all its time waiting on the network. The fix is concurrency, not faster code.
  • Use threads (ThreadPoolExecutor): blocking I/O releases the GIL, so threaded SSH/API calls overlap. Reserve multiprocessing for CPU-bound work and asyncio for async-native libraries.
  • Prefer submit + as_completed with a try/except on future.result() over pool.map, so one failed device is recorded, not fatal.
  • Set max_workers explicitly — the default is tuned for light overlap, and device VTY lines and API rate limits cap it from below.
  • Reads can be concurrent freely; writes must be bounded — canary first, then batch and validate, so concurrency never multiplies the blast radius.

Sources: Python concurrent.futures, Netmiko, Nornir, Scrapli.