The stack (top = intent and governance, bottom = boxes)
Foundationeverything leans on it
Git / CI-CDSource of Truth (NetBox / Nautobot)Secrets (Vault / OpenBao / Conjur)Identity — ISE TACACS+/RADIUS, SAML, PIV, DuoPKI / SUDI / mTLSNTP · DNS · DHCPSmart Licensing (CSSM / SSM On-Prem)DevNet (sandboxes, certs)ITSM (ServiceNow)
▼ pipelines, AuthN/AuthZ, certificates, time ▼
L5 Test / Validatepre- and post-change assurance
pyATS + Genie (Blitz, Clean, Robot)Cisco Modeling Labs (CML)XPRESSONSO netsim / dry-run / commit-queueNDI Pre-Change AnalysisCatalyst Center Compliance / Drift / SWIM checksCrosswork Change Automation checksBatfish · SuzieQ · Containerlab (OSS)
◄► gates the pipeline ►
L4 IaC / Config Mgmtdeclarative + procedural
Ansible cisco.* collections (AAP / AWX / EDA)Terraform / OpenTofu CiscoDevNet providersNexus-as-Code / Network-as-CodePython SDKs (dnacentersdk, meraki, ciscoisesdk, catalystwan, intersight, cobra, fmcapi)Nornir · NAPALM · Netmiko · Scrapli · ncclient · gnmicSaltStack proxy minionsCisco RADKit
▼ REST / RESTCONF / NETCONF / JSON-RPC to orchestrators & controllers — or straight to boxes ▼
L3 Multi-Domain Orchestrationcross-domain, multi-vendor
NSO — CDB · FASTMAP · NEDs · HA Raft · LSACrosswork Network Controller (+SR-PCE, COE)Crosswork Data GatewayCrosswork Change Automation / Health InsightsCrosswork Workflow ManagerCrosswork Hierarchical Controller / Trust InsightsNexus Dashboard OrchestratorIntersight Cloud OrchestratorSplunk SOARCisco XDR AutomationEvent-Driven Ansible
▼ NEDs / REST / webhooks ▼
L2 Domain Controllersone domain each
Catalyst Center (Design · Policy · Provision · Assurance · Platform)Meraki DashboardAPIC / ACINexus Dashboard (NDFC · NDI · NDO)Catalyst SD-WAN Manager / Controller / ValidatorIntersight (SaaS / CVA / PVA)FMC / Security Cloud ControlISE (ERS · OpenAPI · pxGrid · TrustSec)Cyber Vision · IoT Operations Dashboard · INDSpaces · Secure Workload · SNA · Umbrella · Duo · Webex · CUCM AXL · 9800 WLC
▼ PnP · NETCONF · DME/OpFlex · OMP/DTLS · Device Connector · sftunnel · RADIUS/TACACS+ ▼
L1 Data Modelsthe contract
Cisco native YANG (XE/XR/NX)OpenConfigIETF YANGACI / NX-OS DME object treeYANG Suite · pyang · yanglint
L0 Devices & Interfaceswhat the box exposes
NETCONF :830RESTCONF :443gNMI / gNOI :57400 / 9339 / 50051NX-APIModel-Driven Telemetry (dial-out)EEMGuest Shell · on-box Python · IOxZTP · PnP · POAP · claimASA / FDM RESTIOS XE · NX-OS · IOS XR · ASA/FTD · UCS · Meraki · IE/IR
▼ telemetry (MDT / gNMI / NetFlow / syslog / SNMPv3) and events ▼
L6 Observability / Event-Driventelemetry in, action out
Splunk Enterprise · ITSI · Observability CloudThousandEyesAppDynamicsNexus Dashboard InsightsCatalyst Center AI Network AnalyticsSecure Network Analytics · Cyber VisionTelegraf · Kafka · Cisco Telemetry BrokerWebex bots / webhooks
▲ closed loop: event → orchestrator (L3) → change (L4) → validate (L5) → ticket/commit ▲
L0 devicesL1 modelsL2 controllersL3 orchestrationL4 IaCL5 testL6 observabilityfoundation
Closed-loop remediation — one concrete flow
Cat9K MDT (gRPC)→Telegraf→Splunk HEC alert→Webhook (DMZ receiver)→EDA rulebook→pyATS pre-snapshot→NSO re-deploy / CC template / cisco.ios→pyATS post-diff→pass: ServiceNow close + Git commit/fail: NSO reverse-FASTMAP rollback + Webex alert
Who talks to whom (primary southbound paths)
| From | To | Protocol | Notes |
|---|
| Catalyst Center | IOS XE / 9800 | PnP (HTTPS), NETCONF, SSH, SNMPv3, MDT | ISE via pxGrid + ERS for SGT/AAA |
| NDFC | NX-OS | NX-API, POAP, SSH | NDI consumes NX-OS software/hardware telemetry |
| APIC | N9K (ACI mode) | OpFlex | NDO → APIC REST; NDI → APIC |
| SD-WAN Manager | cEdge / vEdge | NETCONF over DTLS/TLS (via Controller/Validator) | Validator must be reachable from untrust → lives in DMZ |
| Intersight | UCS / IMC / HyperFlex | Device Connector (outbound 443) or PVA | Intersight Assist brokers vCenter/storage |
| FMC | FTD | sftunnel 8305 | ASA via REST agent / SCC |
| NSO | everything | NEDs: CLI-SSH, NETCONF, REST(generic), SNMP | incl. NEDs for CC, Meraki, ACI, FMC, ISE, SD-WAN |
| Ansible / Terraform / pyATS | controllers or boxes | REST/RESTCONF/NETCONF/SSH/NX-API/gNMI | EEs pinned; secrets from Vault |
| Boxes | collectors | MDT gRPC dial-out, NetFlow, syslog, SNMP traps | collector in trust; dual destinations for HA |
Comparison — controllers vs orchestrators vs IaC
| Catalyst Center | Meraki | APIC / NDFC | SD-WAN Mgr | Intersight | NSO | Crosswork | Ansible | Terraform |
|---|
| Scope | Campus | Cloud campus/WAN | DC | WAN | Compute | Any vendor/domain | SP | Any | Any (provider) |
| Transactional | Partial | No | ACI yes | Per push | Per profile | Network-wide | Via NSO | No | State-based |
| Brownfield reconcile | Drift reports | — | Import | Attach | Claim | sync-from/check-sync | Via NSO | gathered/parsed | import |
| Multi-vendor | No | No | No | No | Partial | ~400 NEDs | Partial | Yes | Yes |
| Air-gap | Yes | No | Yes | Yes | PVA | Yes | Yes | Yes | Yes |
| HA | 3-node + DR | SaaS | 3–7 APIC / 3 ND | 3 + DR | 3-node VA | Raft 3 / LSA | 3 hybrid + workers | AAP cluster | stateless |
| Rollback | Template | Change log | Snapshot | Template | Profile | Auto reverse-FASTMAP | Playbook | Manual | Prior state |
Security zones — where things live (all tiers)
UNTRUST
- WAN/ISP simulation, dual upstream
- Remote branch edges (SD-WAN cEdge, Meraki MX) onboarding over "internet"
- External DNS stub (ztp.viptela.com / devicehelper.cisco.com)
- Attacker box (validates ZTA assumptions)
- SaaS endpoints: Meraki, Intersight, ThousandEyes, Splunk Cloud — reached only via DMZ proxy
DMZ
- Perimeter FTDv (HA pair) / OPNsense CARP
- SD-WAN Validator (vBond) ×2 — the one Cisco controller that must face untrust
- Webhook receivers (EDA) with HMAC validation
- Egress proxy / reverse proxy / WAF (F5, NGINX)
- Bastion with MFA; AAP hop node
- Splunk HEC ingress, cloud connectors
TRUST
- MGMT-CTRL: Catalyst Center, ND/NDFC/NDI, APIC, SD-WAN Manager/Controller, Intersight PVA, FMC, NSO, Crosswork, SSM On-Prem
- TOOLS: AAP/AWX, OpenTofu, pyATS/XPRESSO, YANG Suite, CML, GitLab, NetBox, OpenBao
- IDENTITY: ISE, AD/FreeIPA, PKI, NTP, DNS
- DATA: Splunk, Telegraf/Kafka, Grafana
- MGMT-OOB: device management VRF — only reachable from MGMT-CTRL/TOOLS
Lab build-out tiers
Cisco T1 · Bare minimum
5 VMs
edge FW · bastion · tools · NSO+netsim · CML; DevNet sandbox for the controller. ~58 GB RAM.Cisco T2 · One of every role
24–29 VMs
CC VA, ISE, FMC, SD-WAN trio, ND, ACI sim, Intersight PVA, NSO, Crosswork+CDG, AWX, NetBox, OpenBao, AD/PKI, Splunk, Telegraf, CML, SSM On-Prem. ~930 GB RAM.Cisco T3 · Pragmatic HA (teaches)
≈38 VMs
HA where it teaches: NSO HA Raft ×3, FTDv HA, Validator ×2, vSmart ×2, ISE 2-node, AWX on RKE2 ×3, OpenBao Raft, Kafka ×3, AD ×2. ~1.1 TB RAM.Cisco T4 · Full vendor HA (costs learning)
≈100 VMs
CC 3+3+witness, ISE 6, vManage 3+DR 3, ND 3, APIC 3, PVA 3, NSO CFS+RFS Raft, Crosswork 5+2 CDG, AAP 8, Splunk 8, F5 pair. ~3.5 TB RAM, 4 hosts.OSS T1
4 VMs
OPNsense · bastion · tools (Ansible/OpenTofu/pyATS/NetBox/Gitea/Grafana) · Containerlab (FRR/SONiC/VyOS/Netopeer2/cEOS).OSS T2
17 VMs
NetBox/Nautobot, AWX+EDA, GitLab CE, OpenBao, FreeIPA+FreeRADIUS+tac_plus+Keycloak+step-ca, Prometheus/Grafana/Loki/SuzieQ/Redpanda, Wazuh+Shuffle, Containerlab+Batfish.OSS T3 · Pragmatic HA
≈32 VMs
OPNsense CARP, RKE2 ×3 for AWX, OpenBao Raft, Patroni PG, Keycloak ×2, Redpanda ×3, Thanos pair, clab ×2.
Compliance spine
FIPS / CNSA
- FIPS mode on RHEL 9, IOS XE/XR, NX-OS, ASA/FTD, CC, ISE, FMC; NSO FIPS build; EEs on UBI9-FIPS
- CNSA 1.0 ciphers everywhere (P-384, AES-256-GCM, SHA-384); track PQC (ML-KEM/ML-DSA) as IOS XE/XR land it
NIST 800-53 / 800-171 / CMMC / CSF 2.0
- CM-2/3/6/8 = IaC + Git + NetBox + drift; AU = Splunk + controller audit; IA = ISE/Vault/MFA; SC-7 = zones; SI-2 = PSIRT → SWIM
DISA STIG / SRG
- Product STIGs: Catalyst Center, ISE, FMC/FTD, ASA, IOS XE, NX-OS, IOS XR, ACI, RHEL 9, Windows 2022, PostgreSQL, Container Platform SRG, NDM SRG
- Automate with Ansible STIG roles, pyATS STIG tests, CC/NDI compliance, Evaluate-STIG in CI
Zero Trust (800-207 / DoD ZT)
- Controllers = network-pillar PDPs; automation identities get short-lived creds; device identity via SUDI + Trust Insights; mTLS gNMI; mgmt VRF micro-segmentation
- This stack is the DoD ZT "Automation & Orchestration" pillar
Vanta 38 crosswalk
- SOC 2 CC6/7/8, ISO 27001 A.8.9/8.15/8.32, PCI 1/2/6.4/10, HIPAA 164.312, GDPR 25/32, DORA/NIS 2 resilience, ISO 42001 / EU AI Act for AI-driven assurance (keep human-in-the-loop)
- Evidence = Git + Splunk + NetBox + ISE + nightly compliance job
Impact levels
- IL5: SD-WAN for DoD (GovCloud), CC appliance, on-prem ND/NSO/Crosswork
- IL6/SIPR: on-prem only; Intersight PVA; no Meraki/TE/Splunk Cloud
- Cross-domain only through approved CDS