Building a conversational agent with LLMs

A conversational agent lets an engineer ask the network a question in plain language — "is Gi1 up on r1?" — and get a grounded answer, with the agent calling real tools to find out. This article shows how such an agent actually works: the tool-calling loop at its core, how to reuse the tools you already exposed over MCP, and the guardrails that keep a chatty interface from becoming a liability.

It is a loop, not magic

Underneath the natural-language veneer, an agent is a short loop. The user speaks; the model interprets the request and decides whether it needs live data; if so it emits a tool call; your code runs the real function and feeds the result back; the model turns that structured data into an answer. The model proposes — your code validates and executes. The model never touches a device directly.

An agent has five parts:

Part	What it does
System prompt	Sets the agent's role and guardrails ("You are a read-only network assistant; never invent data; confirm before any change").
The LLM	A chat model that supports tool calling (also called function calling).
Tool definitions	The schemas of the functions the model may call — for example, the tools from your MCP server.
The agent loop	Send the conversation to the model; if it requests a tool, execute it, append the result, and call the model again; repeat until it returns a plain answer.
Memory	The running list of messages, so the conversation has context across turns.

The tool-calling loop

The loop below is deliberately provider-neutral — it shows the pattern every tool-calling API follows. The model is given the user's message and the available tools; when it asks to call one, your code runs the real function and feeds the result back; the loop ends when the model produces a normal answer instead of a tool call.

SYSTEM = (
    "You are a read-only network assistant. "
    "Use tools for live data. Never fabricate. "
    "Ask for explicit confirmation before any change."
)

# Reuse the read-only tools from the MCP server
tools = {
    "get_interface_status": get_interface_status,
    "list_interfaces": list_interfaces,
}

def agent(user_text, history):
    history.append({"role": "user", "content": user_text})
    while True:
        reply = llm.chat(system=SYSTEM, messages=history, tools=tool_schemas)
        if reply.tool_call:                               # model wants live data
            args = validate(reply.tool_call.args)         # schema-check first
            result = tools[reply.tool_call.name](**args)  # run the real tool
            history.append({"role": "tool", "content": result})
            continue                                      # loop back to the model
        history.append({"role": "assistant", "content": reply.text})
        return reply.text                                 # final answer

A turn in practice: the engineer asks "Is Gi1 up on r1?" The model emits a tool call get_interface_status(host="r1", interface="GigabitEthernet1"); your code validates the arguments, runs the tool, and returns structured status; the model replies "GigabitEthernet1 on r1 is up/up." The model requested a vetted function — it never ran a command on the device.

Real APIs implement exactly this shape, so the pattern ports directly. In the Anthropic Messages API the loop is keyed on the response's stop_reason: keep looping while it is tool_use, and exit on end_turn. OpenAI's function calling carries the request in a tool_calls field and you return each result tagged with its matching tool_call_id. In both, the model can request several tools at once — run them, then return all the results together before the next call. The durable rule: let the stop condition be "the model returned a plain answer," not a text-matching heuristic or a hard iteration count.

Guardrails are the product, not an add-on

The loop is easy. What makes an agent safe to point at production is everything around it. A conversational agent is a new, powerful path into the network, and the same risks that apply to any AI-driven automation apply here — amplified, because the input is free-form natural language that may itself be hostile.

Build these in from the start:

• Read-only by default. An agent that can only query can, at worst, leak information if subverted. An agent that can configure can change the network from a single injected instruction. Keep the default tool surface read-only.
• Validate every argument before executing. The model proposes arguments; treat them as untrusted input. Schema-check them and match names and IDs against an allowlist, exactly as you would in any secure automation code, before they reach a device.
• Keep secrets out of the prompt. Credentials live in the environment or a vault, never in the system prompt or conversation history — a model can be coaxed into repeating its own context.
• Log every tool call. Each call, with its arguments and result, is an audit record. Route it the same way you route any automation log, and attach a run or correlation ID so a conversation can be reconstructed.

Adding a write action safely

Sometimes you do want the agent to propose a change. The safe pattern is two-step confirmation: the agent prepares the change and shows exactly what it will do, and a human must approve before the write runs. The model never executes the change on its own.

def apply_config(host, commands, confirmed=False):
    """Apply config to a device — only with explicit human approval."""
    if not confirmed:
        return {"status": "needs_approval",
                "preview": {"host": host, "commands": commands}}
    # human approved -> validate against an allowlist, then push
    safe = validate_commands(commands)
    conn = _connect(host)
    try:
        out = conn.send_config_set(safe)
    finally:
        conn.disconnect()
    return {"status": "applied", "result": out}

The agent first calls apply_config without confirmation, gets the preview, and shows it to the engineer — "I will run these 3 commands on r1 — approve?" Only after explicit approval does your code re-invoke the tool with confirmed=True. A human stays firmly between the model and the network. Before anything lands, validate the generated change the way you would any other: lint it, dry-run it, and prove it in a lab with Cisco Modeling Labs or pyATS.

Managing conversation memory

The message history grows with every turn and every tool result, and a model has a finite context window. A practical agent trims or summarizes older turns, keeps the system prompt pinned, and retains only what is still relevant. That prevents both context overflow and the slow leak of stale or sensitive data into later prompts.

One more lever on accuracy: ground the agent with retrieval. Supplying authoritative material — a device inventory, your source of truth, vendor documentation — in context, and asking the model to answer from that, keeps it anchored to real data instead of plausible-sounding invention. It does not replace validation, but it narrows the gap the model has to fill. The same discipline that makes AI-assisted development trustworthy applies: treat the model as a fast draft, and verify.

Key takeaways

• An agent is a loop: user → model interprets → (optional) tool call → tool result → model answers, with message history for context.
• The key parts are a system prompt (role and guardrails), a tool-calling model, tool definitions (reuse your MCP tools), the loop, and memory.
• Real APIs follow the same shape — loop while the stop reason is a tool call, return every tool result tagged to its request, and exit on a plain answer.
• The model proposes; your code validates and executes. Validate every tool argument before running it, and never let the model change a device directly.
• Make guardrails intrinsic: read-only by default, confirm before any change, secrets out of prompts, and log every call.