I was recently tasked with converting a VPN WAN mesh with 19
nodes on ASAs over to routers. The client initially wanted statically defined
VPNs, similar to what they were used to in the ASA scheme. However, that is a
lot of keys and configurations. Thankfully, we were able to convince them to
allow us to build a DMVPN mesh to avoid all of that static assignment and just
let the protocol sort it all out.
As an aside, the calculation to find out how many
connections there would be in a VPN mesh is n*(n-1)/2. For 19, that means we
were nearly tasked with manually configuring 171 static profiles. Not something
I’d relish.
Anyway, the client’s security team asked us to investigate
building the dmvpn mesh with certificates to authenticate peers, rather than
passwords. A CA infrastructure can be much more secure and manageable than
passwords, so we said sure, how hard can it be?
Here’s what the design looks like:
Because DMVPN relies on a “hub and spoke” design, let’s make
sure the hub is redundant. Unfortunately there isn’t a good way on Cisco
routers to define two hubs under a single interface, so let’s use two tunnel
interfaces – tunnel1 (DC1 router is the hub) and tunnel2 (DC2 router is the
hub). To allow the hubs to be aware of each other, let’s establish a
‘backplane’ connection between the routers via DMVPN over the WAN. Effectively,
this creates two redundant dmvpn networks over the same WAN infrastructure, and
then connects them – a redundant, self-healing WAN infrastructure.
So let’s build it!
1. Crypto – let’s use some SuiteB strong encryption for our WAN (all routers)
crypto isakmp policy 5
encr aes 256
hash sha512
! authentication rsa-sig <-- Note, this is the default, no need to configure it
group 16
crypto ipsec transform-set dmvpnPhase2 esp-aes 256
esp-sha512-hmac
mode tunnel
crypto ipsec profile dmvpnCrypto
set transform-set
dmvpnPhase2
2. DMVPN
IP information:
Tunnel1 subnet: 10.255.254.0/24
Tunnel2 subnet: 10.255.255.0/24
Tunnel subnets, host IP: DC1: X.X.X.1 / DC2: X.X.X.2 /
Site1: X.X.X.3 / Site2: X.X.X.4
DC1 router public IP: 1.1.1.1
DC2 router public IP: 2.2.2.2
! --- DC1 router – Hub for Tunnel1 DMVPN cloud
! Hub tunnel
int tun 1
description Tunnel1 -
Hub
ip add 10.255.254.1 255.255.255.0
ip mtu 1440
ip nhrp map multicast
dynamic
ip nhrp network-id 1
tunnel key 1
tunnel source
Ethernet1/1
tunnel mode gre
multipoint
tunnel protection
ipsec profile dmvpnCrypto shared
! Client tunnel
int tun 2
description Tunnel2 -
Client
ip add 10.255.255.1 255.255.255.0
ip mtu 1440
ip nhrp map multicast
dynamic
ip nhrp network-id 2
ip nhrp map 10.255.255.2
2.2.2.2
ip nhrp map multicast
2.2.2.2
ip nhrp nhs
10.255.255.2
tunnel key 2
tunnel source
Ethernet1/1
tunnel mode gre
multipoint
tunnel protection
ipsec profile dmvpnCrypto shared
! --- DC2 router – Hub for Tunnel2 DMVPN cloud
! Client tunnel
int tun 1
description Tunnel1 -
Client
ip add 10.255.254.2
255.255.255.0
ip mtu 1440
ip nhrp map
10.255.255.1 1.1.1.1
ip nhrp map multicast
1.1.1.1
ip nhrp nhs
10.255.255.1
ip nhrp map multicast
dynamic
ip nhrp network-id 1
tunnel key 1
tunnel source
Ethernet1/1
tunnel mode gre
multipoint
tunnel protection
ipsec profile dmvpnCrypto shared
! Hub tunnel
int tun 2
description Tunnel2 -
Hub
ip add 10.255.255.2
255.255.255.0
ip mtu 1440
ip nhrp map multicast
dynamic
ip nhrp network-id 2
tunnel key 2
tunnel source
Ethernet1/1
tunnel mode gre
multipoint
tunnel protection ipsec
profile dmvpnCrypto shared
! --- Client site routers
int tun 1
description Tunnel1 -
Client
ip add 10.255.254.XXX
255.255.255.0
ip mtu 1440
ip nhrp map
10.255.255.1 1.1.1.1
ip nhrp map multicast
1.1.1.1
ip nhrp nhs
10.255.255.1
ip nhrp map multicast
dynamic
ip nhrp network-id 1
tunnel key 1
tunnel source
Ethernet1/1
tunnel mode gre
multipoint
tunnel protection
ipsec profile dmvpnCrypto shared
! Client tunnel
int tun 2
description Tunnel2 -
Client
ip add 10.255.255.XXX
255.255.255.0
ip mtu 1440
ip nhrp map multicast
dynamic
ip nhrp network-id 2
ip nhrp map
10.255.255.2 2.2.2.2
ip nhrp map multicast
2.2.2.2
ip nhrp nhs
10.255.255.2
tunnel key 2
tunnel source
Ethernet1/1
tunnel mode gre
multipoint
tunnel protection
ipsec profile dmvpnCrypto shared
3. Routing – EIGRP
Pro tip #1: Make sure to “no next-hop-self” (to allow
phase2 direct pathing to occur) and “no-split-horizon” (to allow DMVPN clients
to learn about other other client site routes)
Pro tip #2: If you want to redistribute ‘connected’ routes into
the routing process, use a route-map to filter the tunnel’s “source” interface,
or you’ll end up with eigrp looping errors due to stacked routers – the neighbors
will think the best way to get to the tunnel source interface is through the
tunnel, which doesn’t make any sense and will break things.
router eigrp ECorp
!
address-family ipv4
unicast autonomous-system 5
!
af-interface
TunnelXX
no next-hop-self
no split-horizon
exit-af-interface
!
topology base
redistribute static
exit-af-topology
network 10.0.0.0
exit-address-family
!--- Client sites
router eigrp ECorp
!
address-family ipv4
unicast autonomous-system 5
!
topology base
redistribute static
exit-af-topology
network 10.0.0.0
exit-address-family
4. Certificate infrastructure
Okay, now we have a respectable DMVPN infrastructure. We could either add some static pre-shared keys, or we could build the certificate infrastructure on the routers to allow dynamic, flexible management. Let’s do it.
Pro tip: On all routers, make sure to set up ntp to an
accurate time-source. Certificates are time-stamped, and if your clocks drift
too far, your infrastructure might fall apart for what appears to be no reason.
! – NTP, clocks (all routers)
ntp server 132.163.4.101
clock timezone MST -6
clock summer-time MDT recurring
!-- To build certs we need a hostname and domain (all
routers)
hostname Hub1
ip domain-name ecorp.com
!-- To allow clients to download their certs (CA server
only)
ip http server
!—Certificate server (Only on CA server – we’ll use Hub1
router)
Pro tip: Because of a limitation of GNS3 to store files in
flash, I’m storing the certs in nvram. You’re able to store them with ftp,
tftp, direct to flash, and many other ways if you’d like.
crypto key generate rsa label hub1-ca modulus 2048
exportable
crypto pki server hub1-ca
issuer-name
CN=Hub1.ecorp.com
grant auto
hash sha512
auto-rollover
database url nvram:
database level
complete
no shut
!—Register the router to the CA and get a signed identity
certificate.
Pro tip #1: Even the CA server (If participating in DMVPN, like we
are doing here) needs to check in with the CA (itself) to get an identity
certificate.
Pro tip #2: This doesn’t have to occur over a local network or
over a VPN link on the internet – it works just fine over the internet. And the
routers are only exchanging public keys, never their private signing keys, so
there is little risk here.
crypto key generate rsa label Hub1-dmvpn modulus 2048
exportable
crypto pki trustpoint dmvpn-client
enrollment url
http://1.1.1.1:80
subject-name CN=XXXX.ecorp.com <-- Use a unique name, like the router’s hostname
revocation-check crl
rsakeypair Hub1-dmvpn
storage nvram:
exit
!—Once the trustpoint is built, we need to activate it to
accept the CA as valid and get our own router identity certificate to be used
for dmvpn authentication.
crypto pki authenticate dmvpn-client
! Provided your router can reach the IP of the CA (public
IPs work just fine), the router will download the CA certificate and store it
locally. You are asked to accept it.
crypto pki enroll dmvpn-client
! You are asked several questions about downloading your
identity certificate. Make sure to say “yes” to downloading your own cert, and
the routers will negotiate the rest.
5. Debugging
If you have any issues, you can use the
following commands to troubleshoot the issues. In my experience, these aren’t
incredibly useful, but it might help!
! Debug hub
debug crypto pki messages
debug crypto pki server
debug crypto pki transactions
! Debug spokes
debug crypto pki messages
debug crypto pki server
debug crypto pki transactions
debug crypto isakmp
debug crypto ipsec
debug crypto engine
Profit!
And that’s it. Clients will be able to authenticate to the
hubs and to each other. Clients will
automatically roll-over to new certificates when the certs reach their
expiration point, and no more user interaction is required at any point.
Also, any routes injected into the routing domain by any
client or hub devices will propagate through the dmvpn clouds and direct
routing will take over nearly immediately.
Possible venues for making this situation more redundant are more hubs, redundant CA infrastructures, and regionalizing initial routing.
To regionalize WAN routing so the shortest path is used for the initial connection (before phase2 routing utilized p2p connections), lessen the delay (default 50,000u sec) on the tunnel interface pointed at the local DMVPN hub site.
Download and play with the topology here.
Possible venues for making this situation more redundant are more hubs, redundant CA infrastructures, and regionalizing initial routing.
To regionalize WAN routing so the shortest path is used for the initial connection (before phase2 routing utilized p2p connections), lessen the delay (default 50,000u sec) on the tunnel interface pointed at the local DMVPN hub site.
Download and play with the topology here.
Have fun and good luck!
Kyler
