Ansible Automation on Ubuntu 20.04 in GNS3

Ansible is a useful tool for automating tasks on various systems, including servers and network devices. It can be a little tricky to set up, but it’s ok once you get the hang of it. Today we’ll just try to get ansible installed on an Ubuntu 20.04 server and able to connect to another Ubuntu server, a Rocky Linux server, and a Cisco IOSv router.

Topology

Ansible in GNS3

Our simple subnet of 172.16.0.0/24 will hold a Ubuntu 20.04 server at the top at 172.16.0.1 acting as the ansible controller. The others will all be managed nodes, they don’t have ansible installed. Ansible uses python3 on Linux servers to execute various commands, and it’s not installed by default on Rocky Linux 8.5, so I installed it. I also added a firewall to the Rocky node to allow SSH in, which is how ansible connects to nodes. The SSH server has been enabled on the Cisco IOSv router as well.

Installation

Following ansible’s official installation instructions, this quick script will get it installed. Comments are inline:

#Create apt file for ansible
touch /etc/apt/sources.list.d/ansible.list

#Add ppa to above file
echo '
deb http://ppa.launchpad.net/ansible/ansible/ubuntu focal main
' >> /etc/apt/sources.list.d/ansible.list

#Add key for ppa
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 93C4A3FD7BB9C367

#Update and install
apt-get update
apt-get install ansible

Ansible should now be installed!

Create inventory

The next thing to do is create an inventory file. Ansible keeps it’s config files in /etc/ansible, so we’ll start by writing a config to /etc/ansible/hosts. This file supports multiple formats, we’ll use yaml because I like yaml (don’t judge me!). This simple script (it’s just an echo command and redirect) will set up the hosts file:

echo '

servers:
  hosts:
    ubuntu:
      ansible_host: 172.16.0.2
    rocky:
      ansible_host: 172.16.0.3
  vars:
    ansible_python_interpreter: /usr/bin/python3  #specify python3 for ubuntu and rocky


routers:
  hosts:
    cisco:
      ansible_host: 172.16.0.4
      ansible_network_os: cisco.ios.ios                     #OS is Cisco IOS!
      ansible_connection: ansible.netcommon.network_cli     #connect to IOS via CLI, no python
      ssh_args: -oKexAlgorithms=+diffie-hellman-group1-sha1 #key exchange algorithm to support older IOS

' > /etc/ansible/hosts

Verify inventory connections

The below command will check the integrity of your inventory (hosts) file:

ansible-inventory --list -y

Then once you have worked out any inventory file issues, “ping” your hosts. This is not an ICMP echo-request/echo-reply (commonly known as ping), ansible will actually try to log in to each host via SSH. Ansible assumes you have SSH public/private key pair authentication enabled, but since this is in a test environment in GNS3 we’ll just pass the user and password to ansible on the command line:

ansible all -m ping --extra-vars "ansible_user=james ansible_password=james"
---

ubuntu | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
rocky | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
cisco | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

With this, we’ve verified that we have configured our hosts and can properly connect to them from the controller.

Next we’ll try to make some changes with playbooks! Hope you liked this one.

Nagios Core 4.4.6 Monitoring on Ubuntu 20.04

Nagios is a time-tested network monitoring tool that network engineers and systems administrators alike have used since its creation in 1999 to monitor networks and alert engineers if something goes wrong. At some point around 2009, Nagios became Nagios Core so that the company could release some more products. Nagios Core remains free and open source, though.

It has many ways to monitor various network nodes, including ping checks, agents and SNMP. Today, we’ll see if we can just get the server and web interface installed, along with a couple basic ping checks.

Let’s get started!

Topology

Topology in GNS3

We’ll monitor across a simple subnet of 172.16.0.0/24. We’ll install Nagios Core on Ubuntu server at the top, and monitor the Rocky Linux and Cisco IOSv router at the bottom.

Installation

While it’s possible to install from the standard Ubuntu repositories, that version is very old and doesn’t seem to work very well out of the box. Building the most recent version from source works the best.

There are a number of steps, so I will include comments inline about what commands we’re entering. To install:

#Install necessary libraries
apt-get update
apt-get install -y autoconf gcc libc6 make wget unzip apache2 php libapache2-mod-php7.4 libgd-dev

#Download Nagios Core
cd /tmp
wget -O nagioscore.tar.gz https://github.com/NagiosEnterprises/nagioscore/archive/nagios-4.4.6.tar.gz

#Extract and enter directory
tar xzf nagioscore.tar.gz

#Compile
cd /tmp/nagioscore-nagios-4.4.6/
./configure --with-httpd-conf=/etc/apache2/sites-enabled
make all

#Set up users
make install-groups-users
usermod -a -G nagios www-data

#Install
make install
make install-daemoninit
make install-commandmode

#Install a sample script config
make install-config

#Install Apache config files
sudo make install-webconf
sudo a2enmod rewrite
sudo a2enmod cgi

#Create a user account for nagios
sudo htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

#Restart apache
systemctl restart apache

#Prepare for plugins installation. Need plugins to do anything at all with nagios
sudo apt-get install -y autoconf gcc libc6 libmcrypt-dev make libssl-dev wget bc gawk dc build-essential snmp libnet-snmp-perl gettext

#Download and extract plugins
cd /tmp
wget --no-check-certificate -O nagios-plugins.tar.gz https://github.com/nagios-plugins/nagios-plugins/archive/release-2.3.3.tar.gz
tar zxf nagios-plugins.tar.gz

#Compile plugins and install
cd /tmp/nagios-plugins-release-2.3.3/
./tools/setup
./configure
make
make install

That was a lot! Hopefully you got it installed ok. The nagios web interface can be accessed from http://<fqdn or ip>/nagios. It should look something like this:

Nagios Web Interface

Now that we have it installed, it’s time to set up some monitoring. You’d think this could be done via the web interface but it cannot. It needs to be done from nagios config files.

Monitoring Configuration

There are a number of configuration files, they should all be located in /usr/local/nagios/etc/objects. The nagios program itself is located at /usr/local/nagios/bin/nagios, but the installation process registered a service with systemd, so we’ll mostly be using that. We need to create “hosts”, which are other severs, workstations or routers to monitor. We’ll run this script, I have commented in-line:

#Edit the main nagios config file at /usr/local/nagios/etc/nagios.cfg, add line to add a config file called "hosts.cfg"
echo "cfg_file=/usr/local/nagios/etc/objects/hosts.cfg" >> /usr/local/nagios/etc/nagios.cfg

#Write a config to ping hosts
echo "

define host{
    host_name                       rocky
    alias                           rocky
    address                         172.16.0.2
    check_command                   check-host-alive
    max_check_attempts              5
    check_period                    24x7
    notification_interval           30
    notification_period             24x7
}
define host{
    host_name                       cisco_iosv
    alias                           cisco_iosv
    address                         172.16.0.3
    check_command                   check-host-alive
    max_check_attempts              5
    check_period                    24x7
    notification_interval           30
    notification_period             24x7
}

" >> /usr/local/nagios/etc/objects/hosts.cfg

Before you restart nagios and apply the configuration, you can check to see if there’s any errors using a built-in tool that nagios has:

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
---
<Edited for brevity>

Total Warnings: 2
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check

If there’s any errors, it should give you some advice on how to fix them. Otherwise, restart nagios:

systemctl restart nagios

After the checks run, you should be seeing that your hosts are up by click on “Hosts” on the nav bar to the left:

Nagios is a sizeable piece of software and very extensible. There are many things you can do with it far beyond what we’ve done here. However we’ve gotten the core functionality working – pinging stuff.

We’ll take a look at how to do more than that in the next post. Stay tuned!

Syslog Server on Ubuntu 20.04

Running a syslog server that can collect logs from various devices on your network is really simple with Ubuntu Server 20.04. Using built-in software Rsyslog, you can quickly configure it to be either a syslog client or a server. Since most network devices have the capability to send logs to an external server, you can quickly set up your Ubuntu server act as a central log collection point.

What many folks don’t know is that syslog is actually a standard application-layer network protocol, not just software. It is defined in RFC 5424. It’s because of this standard protocol that network devices and servers alike are able to easily send and store logs. Without a standard protocol, it would be much more difficult to pull that off.

Let’s set up syslog on Ubuntu 20.04!

Topology

Topology in GNS3

The Ubuntu server at 10.0.0.1 will act as our syslog server while the other Ubuntu server and Cisco router will act as clients, sending their logs to the server.

Server Configuration

Since Rsyslog is already installed on Ubuntu (and others), there’s no installation. First we need to edit /etc/rsyslog.conf and uncomment these lines:

module(load="imudp")
input(type="imudp" port="514")

module(load="imtcp")
input(type="imtcp" port="514")

They will activate the server on TCP and UDP port 514 for incoming syslog messages. With just this configuration, the syslog server will work. But we’ll make one more modification – we want each IP address to have it’s own file. Otherwise all messages get dumped in the main file at /var/log/syslog.

We’ll create a file at /etc/rsyslog.d/30-custom.conf and place a couple of simple rules in it:

if $fromhost-ip startswith '10.0.0.2' then /var/log/network/10.0.0.2.log
& stop
if $fromhost-ip startswith '10.0.0.3' then /var/log/network/10.0.0.3.log
& stop

Create and change the ownership of the /var/log/network directory:

mkdir /var/log/network
chown syslog:adm /var/log/network

And restart Rsyslog:

systemctl restart rsyslog

And we’re done!

Client Configuration

For a Cisco IOSv device, the following command will turn on logging to a remote server:

logging host 10.0.0.1

For Ubuntu, just add the following line to /etc/rsyslog.conf:

*.* @@10.0.0.1:514

And restart the service:

systemctl restart rsyslog

Verification

To verify that syslog messages are in fact going to the server, we need to initiate an event.

For Cisco IOSv, shutting/no shutting any interface will do the trick. In config mode on the interface, just issue these commands:

Router(config-if)# shut
Router(config-if)# no shut

While you might be tempted to go check /var/log/network/10.0.0.2.log right away for syslog messages, it might be worth it to do a packet capture first to see if logs are indeed leaving the Cisco router and heading for the syslog server.

A capture between the two shows the following lone packet when we issue those shut commands:

Syslog protocol in Wireshark

Then check the /var/log/network/10.0.0.3.log on the syslog server to see if the message was properly written:

cat /var/log/network/10.0.0.3.log
---
Jan 27 13:48:17 10.0.0.3 45: *Jan 27 13:48:16.540: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to down

Initiating an event on the Ubuntu client is as easy as shutting down a service (I’m sure there’s others too). I happen to have Nginx web server running on this guy so I’ll stop it:

systemctl stop nginx

The in the file on the syslog server:

cat /var/log/network/10.0.0.2.log
---
Jan 27 23:21:21 u20vm systemd[1]: Stopping A high performance web server and a reverse proxy server...
Jan 27 23:21:21 u20vm systemd[1]: nginx.service: Succeeded.
Jan 27 23:21:21 u20vm systemd[1]: Stopped A high performance web server and a reverse proxy server.

Hope you liked this one.

Install GNS3 and Docker on Ubuntu 20.04 for Cisco and Linux Network Labs

Every major OS has its place, so I’m not hoping to get into that discussion, but I find that Ubuntu Linux works really well for creating network labs in GNS3. If you’re not familiar with GNS3, you’re missing out. It allows you to pull in real VM’s, and even Docker containers into an emulated network environment for testing and experimentation. You can run Cisco routers and switches, other vendor network vendors, Windows Desktop and Server, Linux and any other OS that is supported by Linux’s QEMU/KVM hypervisor which is pretty much anything. GNS3 has many features, but today we’ll just look at getting it installed, along with Docker.

Why is Ubuntu better to run GNS3? You may have noticed that on Windows or Mac versions of GNS3, the server has to run on a VM to work properly. That server VM runs a Linux OS, specifically Ubuntu. So using Ubuntu as your desktop OS means you’re cutting out all of that complexity with the server VM, not to mention the additional RAM consumed. Simply put, GNS3 runs the way it’s supposed to on Ubuntu. Not to knock the Windows and Mac versions, the GNS3 team worked hard on those. But in my humble and honest opinion, Ubuntu just works better for GNS3.

Most folks stick to using VM’s in GNS3, but the Docker integration is pretty awesome and has some very real benefits over VM’s. Any docker container you have installed on the same system as the GNS3 server can be pulled into GNS3, although whether it will work properly depends somewhat on what the container has installed in it.

GNS3 Installation

The official GNS3 Ubuntu releases can be found at their PPA at:

https://launchpad.net/~gns3/+archive/ubuntu/ppa

The PPA can be added and GNS3 installed with just a few quick commands, although it’s a relatively big download:

sudo add-apt-repository ppa:gns3/ppa
sudo apt-get update
sudo apt-get install gns3-gui

When you first run GNS3, you’ll notice that the default option is not a VM, it’s to run the server locally. No VM needed!

At this point, GNS3 is installed, although you may have to run this command to get wireshark captures working:

chmod 755 /usr/bin/dumpcap

Docker Installation

I’ll just be following the official Docker instructions here, they work great:

https://docs.docker.com/engine/install/ubuntu/

These are to install the repository, which is probably the “best” option. There is a convenience one-liner script, but we all know that’s not a good habit to get into, so we’ll avoid that.

First install dependencies:

 sudo apt-get update
 sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

Add the Docker official GPG key:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Add the stable repository:

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

And install:

 sudo apt-get update
 sudo apt-get install docker-ce docker-ce-cli containerd.io

To avoid getting permissions errors in GNS3, you’ll need to add your user to the docker group. You’ll need to log out/log in or restart for this to take effect:

sudo usermod -aG docker ${USER}

Add a Docker container to your GNS3

Now that Docker is installed, pulling a Docker image from the Docker Hub is easy. A popular one is Alpine Linux because it’s so small, but packs lots of popular tools and libraries:

docker pull alpine

Now you should be able to add this image to GNS3. Go into GNS3, go to preferences, and all the way at the bottom where it says “Docker containers”. Click on “new”, and you should be able to select the Alpine Linux image from the drop-down menu:

Click through and leave the defaults, but you might want two network adapters instead of one, in case you want it to be a router. Now just drag a couple containers out onto the canvas:

At this point, you should be able to double click on these and get a busybox shell, which will let you configure IP settings and the like. You may have noticed that the startup of these containers is near-instantaneous, and they consume very little RAM. One of the many perks of the lightweight nature of Docker containers. Enjoy!

Strongswan IPSec Mediation – Both Ubuntu Linux 20.04 Peers Behind Cisco NAT

Background

Strongswan is an open source project that implements the IKE protocol which is used for cryptographic key negotiation in the IPSec standard protocol. IPSec is used to build an encrypted network connection between two points on a network, usually the Internet but not always.

Most often, the two points where the encryption and decryption are happening know about each other’s IP address, on the Internet that means they have publicly routable IP addresses. IPSec has a built-in mechanism to handle the scenario if one of these points does not have an a public routable IP address, this is called NAT-Traversal. It looks something like this:

POINT A —> NAT ROUTER —> INTERNET —> POINT B

Point A must initiate the connection to Point B because it has a private IP address (10.x and the like) and that’s the only way the NAT router will let the connection through.

But what if BOTH peers are behind NAT routers with private IP addresses? Something like this:

POINT A —> NAT ROUTER —> INTERNET —> NAT ROUTER —> POINT B

Private-peer-to-private-peer connectivity is established in other network protocols such as VoIP using some sort of 3rd party mediator that performs a trick called UDP hole-punching, where both peers are told by the mediator the public IP address of the NAT Router that is in front of the peer they are trying to connect to. For VoIP this is usually a STUN server. Both peers attempt a connection at the same time, the NAT Routers write translations for the peers, and a connection is built.

While this technique is pretty ancient and many applications implement it, there is not a standard way to do it for IPSec. Strongswan, it seems, has a little known feature for IPSec peer mediation that allows for peer to peer NAT Traversal similar to STUN in VoIP. This feature requires that a third device have a public IP (can’t escape a public IP somewhere in the equation) and running the Strongswan mediation service. It only works with strongswan, although an RFC draft was created back in 2008 in the hopes that mediation for IPSec would be standardized.

Topology

The two Ubuntu 20.04 IPSec peers are both behind Cisco IOSv routers running a basic NAT in Port Address Translation (PAT) mode, which is a tcp/udp port-based one-to-many NAT that is running by default on many consumer routers and is the way the many devices today connect to the Internet. In other words, 192.168.0.3 can not ping 172.16.0.3, and vice versa. But they can both ping 13.0.0.2, where our secret weapon is located, the Strongswan IKE Mediator.

Installation

Installing Strongswan on Ubuntu 20.04 is easy as pie:

apt-get install strongswan

Configuration

UbuntuServer20.04-1

First edit /etc/ipsec.conf

config setup
conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        keyexchange=ikev2
        mobike=no
        dpdaction=restart
        dpddelay=60s
        left=%defaultroute
        leftfirewall=yes

conn medsrv
        leftid=bob@questioncomputer.com
        leftauth=psk
        right=13.0.0.2
        rightid=mediator@questioncomputer.com
        rightauth=psk
        mediation=yes
        authby=secret
        auto=start

conn peer
        leftid=bob@questioncomputer.com
        leftauth=psk
        leftsubnet=192.168.0.0/24
        right=%any
        rightid=alice@questioncomputer.com
        rightsubnet=172.16.0.0/24
        rightauth=psk
        authby=secret
        mediated_by=medsrv
        auto=start

Then edit /etc/ipsec.secrets to put your PSK in there:

mediator@questioncomputer.com : PSK "james"
bob@questioncomputer.com : PSK "james"
alice@questioncomputer.com : PSK "james"

UbuntuServer20.04-2

First edit /etc/ipsec.conf

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        keyexchange=ikev2
        mobike=no
        dpdaction=restart
        dpddelay=60s
        left=%defaultroute
        leftfirewall=yes
conn medsrv
        leftid=alice@questioncomputer.com
        leftauth=psk
        right=13.0.0.2
        rightid=mediator@questioncomputer.com
        rightauth=psk
        mediation=yes
        auto=add

conn peer
        leftid=alice@questioncomputer.com
        leftauth=psk
        leftsubnet=172.16.0.0/24
        right=%any
        rightid=bob@questioncomputer.com
        rightsubnet=192.168.0.0/24
        rightauth=psk
        mediated_by=medsrv
        authby=secret
        auto=start

Then edit /etc/ipsec.secrets to put your PSK in there:

mediator@questioncomputer.com : PSK "james"
bob@questioncomputer.com : PSK "james"
alice@questioncomputer.com : PSK "james"

Mediator (UbuntuServer20.04-4)

First edit /etc/ipsec.conf

conn %default
        ikelifetime=60m
        keylife=20m
        rekeymargin=3m
        keyingtries=1
        keyexchange=ikev2
        mobike=no
        dpdaction=clear
        dpddelay=60s

conn medsrv
        left=13.0.0.2
        leftid=mediator@questioncomputer.com
        leftauth=psk
        leftfirewall=yes
        right=%any
        rightauth=psk
        mediation=yes
        authby=secret
        auto=add

Then edit /etc/ipsec.secrets

mediator@questioncomputer.com : PSK "james"<br>bob@questioncomputer.com : PSK "james"<br>alice@questioncomputer.com : PSK "james"

Make sure to issue this on all three to load your configs:

ipsec restart

Verification

A healthy connection can be checked with the ‘ipsec’ command:

root@u2004:/home/james/strongswan# ipsec statusall
Status of IKE charon daemon (strongSwan 5.9.0, Linux 5.4.0-29-generic, x86_64):
  uptime: 103 minutes, since Oct 08 19:38:24 2020
  malloc: sbrk 1748992, mmap 0, used 902656, free 846336
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 14
  loaded plugins: charon agent connmark eap-mschapv2 aes des rc2 sha2 sha3 sha1 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp curve25519 chapoly xcbc cmac hmac gcm drbg attr kernel-netlink resolve socket-default socket-dynamic stroke vici updown xauth-generic lookip error-notify addrblock unity counters
Listening IP addresses:
  192.168.0.3
Connections:
      medsrv:  %any...13.0.0.2  IKEv2, dpddelay=60s
      medsrv:   local:  [left@questioncomputer.com] uses pre-shared key authentication
      medsrv:   remote: [mediator@questioncomputer.com] uses pre-shared key authentication
      medsrv:   child:  dynamic === dynamic TUNNEL, dpdaction=restart
        peer:  %any...%any  IKEv2, dpddelay=60s
        peer:   local:  [left@questioncomputer.com] uses pre-shared key authentication
        peer:   remote: [right@questioncomputer.com] uses pre-shared key authentication
        peer:   child:  192.168.0.0/24 === 172.16.0.0/24 TUNNEL, dpdaction=restart
Security Associations (2 up, 0 connecting):
        peer[6]: ESTABLISHED 2 minutes ago, 192.168.0.3[left@questioncomputer.com]...12.0.0.2[right@questioncomputer.com]
        peer[6]: IKEv2 SPIs: 400da3c783ad093d_i 35869903e6803225_r*, pre-shared key reauthentication in 52 minutes
        peer[6]: IKE proposal: AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/ECP_256
        peer{10}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: c01b6c68_i c56e26d5_o
        peer{10}:  AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 12 minutes
        peer{10}:   192.168.0.0/24 === 172.16.0.0/24
      medsrv[3]: ESTABLISHED 49 minutes ago, 192.168.0.3[left@questioncomputer.com]...13.0.0.2[mediator@questioncomputer.com]
      medsrv[3]: IKEv2 SPIs: 127a15068a32e07f_i* 703cba84aa11f49e_r, pre-shared key reauthentication in 2 minutes
      medsrv[3]: IKE proposal: AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/ECP_256

You’ll be looking to see that both the “medsrv” and “peer” connections are in an “Established” state.

And of course as always, make sure you ping across the tunnel to make sure it works.

As an added bonus, I was able to ping through the tunnel from l92.168.0.3 to the Cisco NAT Router at 172.16.0.1, I just need to add a route on the Cisco NAT Router by issuing “ip route 192.168.0.0 255.255.255.0 172.16.0.3”.

Troubleshooting

The log file where most logs go in Ubuntu is at ‘/var/log/syslog’, grepping for “charon” will help you sort them:

root@u2004:/home/james/strongswan# tail /var/log/syslog | grep charon
Oct  8 21:28:41 u2004 charon: 13[NET] sending packet: from 192.168.0.3[4500] to 12.0.0.2[4500] (80 bytes)
Oct  8 21:28:41 u2004 charon: 16[NET] received packet: from 12.0.0.2[4500] to 192.168.0.3[4500] (80 bytes)
Oct  8 21:28:41 u2004 charon: 16[ENC] parsed INFORMATIONAL response 8 [ ]

And as usual, do a packet capture to see if the two peers are trying to build a connection to each other. If packets are not being generated or dropped somewhere, you’ll know where to look for a problem. You can filter for “isakmp” packets to narrow things down:

In the above image you can see there is a connection between the NAT routers 11.0.0.2 and 12.0.0.2, but also connections to the mediator at 13.0.0.2.

IPsec on Linux – Strongswan Configuration w/Cisco IOSv (IKEv2, Route-Based VTI, PSK)

IPsec is a cool tool for encrypting connections between network nodes, usually over the Internet (but not always). There are many different ways to configure an IPsec tunnel. Many tunnels use a policy-based approach which means the traffic that is sent through the tunnel is pre-defined using a “policy” that is part of the configuration. That doesn’t always work, you may need to dynamically change what traffic goes through using a routing protocol like OSPF or EIGRP without having to bring the tunnel down and reconfigure it . Thus there is another option, a “route-based” tunnel.

There are two different methods for creating a route-based IPsec tunnel, the first using GRE, which inserts a second GRE IP header into packets going into the tunnel. The second is VTI, which operates in a similar manner to GRE but under the hood it’s quite a different implementation.

VTI was originally way to save IP space on point-to-point links in the early networking days before subnetting. It was adapted as a way to assign routes to an IPsec tunnel. Part of its legacy is a “numbered” or “unnumbered” mode. Originally, VTI could inherit an IP from another interface and save IP address space. This unnumbered mode is pretty strange, so today we’ll take a look at numbered mode to keep things familiar.

Topology

Pretty simple, we’re trying to get the Window10 box at the bottom left to ping the Ubuntu Server 18.04 at the bottom right to ping each other. We’ll configure a tunnel between the Ubuntu box at the top left and the Cisco IOSv router at the top right. The underlying 1.1.1.0/30 network serves to look like a WAN network, while the network inside the VTI tunnel will be 10.0.0.0/30.

Installation

Ubuntu18.04-FRR-1


Strongswan on Ubuntu 18.04 is pretty easy with apt-get:

apt-get install strongswan

Configuration

Ubuntu18.04-FRR-1

First you need to add a config to /etc/ipsec.conf, something that looks like this:

conn tunnel 
        leftupdown=/usr/local/sbin/ipsec-notify.sh #run this script on start
        left=166.0.0.5
        leftsubnet=0.0.0.0/0 #all traffic
        right=166.0.0.1
        rightsubnet=0.0.0.0/0 #all traffic
        ike=aes256-sha2_256-modp1024!
        esp=aes-sha2_256!
        authby=secret
        auto=start
        keyexchange=ikev2
        mark=32 # only accepts packets with this mark
        type=transport

Then configure the PSK in /etc/ipsec.secrets:

1.1.1.2 1.1.1.1 : PSK '12345'

Then create the tunnel script that is referenced in the config. For me the file will be located at /home/james/ipsec-notify.sh:

ip link add vti0 type vti local 1.1.1.2 remote 1.1.1.1 key 32
ip link set vti0 up
ip addr add 10.0.0.2/30 dev vti0
ip lin set dev vti0 mtu 1400

Note the “key 32” in the first line above. That identifies what traffic strongswan should encrypt and corresponds to the “mark” in the strongswan config. It’s important.

Next you need to add a line for your VTI interface in /etc/sysctl.conf that looks like this to disable kernel policy lookups, this is a routed interface:

net.ipv4.conf.vti0.disable_policy=1

Finally, you need to tell Charon (Strongswan’s IKE daemon) to not handle routing. We’ll handle routing on our own. In /etc/strongswan.d/charon.conf, find this line:

install_routes = no

Make sure it’s set to no. Tell strongswan to restart and the tunnel should attempt a connection:

ipsec restart

CiscoIOSv15.6(2)T-1

My running-config is abbreviated, but it looks like this:

crypto ikev2 proposal james-proposal 
 encryption aes-cbc-256
 integrity sha256
 group 2
!
crypto ikev2 policy james-policy 
 proposal james-proposal
!
crypto ikev2 keyring james-ring
 peer remote-router-james
  address 1.1.1.2
  pre-shared-key 12345
!   
crypto ikev2 profile james-profile
 match identity remote address 1.1.1.2 255.255.255.255 
 authentication local pre-share
 authentication remote pre-share
 keyring local james-ring
!
crypto ipsec transform-set james-trans esp-aes esp-sha256-hmac 
 mode transport
!
crypto ipsec profile james-protect-vti
 set transform-set james-trans 
 set ikev2-profile james-profile
!
interface Tunnel0
 ip address 10.0.0.1 255.255.255.252
 ip mtu 1400
 tunnel source 1.1.1.1
 tunnel mode ipsec ipv4
 tunnel destination 1.1.1.2
 tunnel protection ipsec profile james-protect-vti

Static routing

Ubuntu18.04-FRR-1

Pretty simple, just add a route for 172.16.0.0/24 pointing to 10.0.0.1

ip route add 172.16.0.0/24 via 10.0.0.1

CiscoIOSv15.6(2)T-1

Simple here too:

ip route 192.168.0.0 255.255.255.0 10.0.0.2

Optional – EIGRP Routing

Remove those static routes and we’ll try a protocol to achieve the aforementioned dynamic routing.

Ubuntu18.04-FRR-1

Checkout my post on setting up EIGRP with Free Range Routing on Linux for the installation. Assuming you have that done, just log into the FRR shell on Ubuntu, it’s in /snap/bin, or just use the full path:

/snap/bin/frr.vtysh
Hello, this is FRRouting (version 7.2.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
james# 
james# conf t
james(config)# router eigrp 10
james(config-router)# network 192.168.0.0/24
james(config-router)# network 10.0.0.0/30
james(config-router)# end
james# 

CiscoIOSv15.6(2)T-1

On a Cisco IOSv router, it’s pretty simple:

Router(config)#router eigrp 10
Router(config-router)#network 172.16.0.0 0.0.0.255
Router(config-router)#network 10.0.0.0 0.0.0.3
Router(config-router)#end

Verification

Ubuntu18.04-FRR-1

Verifying the tunnel on Ubuntu is done with “ipsec statusall”, although there are more specific commands. This will do since we only have one tunnel. I’ve abbreviated the output, but these lines say it all:

Security Associations (1 up, 0 connecting):
      tunnel[1]: ESTABLISHED 58 minutes ago, 1.1.1.2[1.1.1.2]...1.1.1.1[1.1.1.1]
      tunnel[1]: IKEv2 SPIs: 9a137e96ee332b6b_i* 1599c6291be65825_r, pre-shared key reauthentication in 110 minutes
      tunnel[1]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024
      tunnel{11}:  INSTALLED, TRANSPORT, reqid 1, ESP SPIs: c72f4e27_i de8fcbc0_o
      tunnel{11}:  AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 30 minutes
      tunnel{11}:   1.1.1.2/32 === 1.1.1.1/32

For routing, just issue “ip route” and you’ll see your static or EIGRP routes:

root@james:/snap/bin# ip route
1.1.1.0/30 dev eth1 proto kernel scope link src 1.1.1.2  
10.0.0.0/30 dev vti0 proto kernel scope link src 10.0.0.2  
172.16.0.0/24 via 10.0.0.1 dev vti0 proto 192 metric 20  
192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.1

CiscoIOSv15.6(2)T-1

Lots of ways to show what’s going on with an IPsec tunnel on an IOS device, and “show crypto ipsec sa peer” is one of them. I usually use “i” to just include the status:

Router#show crypto ipsec sa peer 1.1.1.2 | i Status
        Status: ACTIVE(ACTIVE)
        Status: ACTIVE(ACTIVE)
        Status: ACTIVE(ACTIVE)
        Status: ACTIVE(ACTIVE)

Also “show ip route” will show your static or EIGRP routes (D for EIGRP):

      1.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        1.1.1.0/30 is directly connected, GigabitEthernet0/1
L        1.1.1.1/32 is directly connected, GigabitEthernet0/1
      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.0.0.0/30 is directly connected, Tunnel0
L        10.0.0.1/32 is directly connected, Tunnel0
      172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C        172.16.0.0/24 is directly connected, GigabitEthernet0/0
L        172.16.0.1/32 is directly connected, GigabitEthernet0/0
D     192.168.0.0/24 [90/26882560] via 10.0.0.2, 00:07:56, Tunnel0

Finally, don’t forget to ping from Windows:

Troubleshooting

There are lots of tools here, including the strongswan “ipsec statusall”, Cisco debug commands, and others. But the one that always let’s me know what’s wrong the fastest is a packet capture. Look for IKE negotiation packets (ISAKMP filter in Wireshark) if you’re tunnel isn’t coming up, and make sure traffic goes through the tunnel (ESP filter in Wireshark) when it’s up:

BIND9 DNS Server on Ubuntu 18.04 Linux (w/ Cisco Routing)

Background

BIND9 is an open-source DNS (Domain Name System – the text-to-ip-address resolution system we all rely upon) server application that many of the world’s DNS servers use to change IP addresses (e.g. 1.2.3.4) into strings of characters such as “questioncomputer.com”. DNS is an area of expertise in which some people spend their entire careers and as such, there is much to know about it. I’m no DNS expert, but since anyone can fire up their own DNS server with BIND9, I figured I’d give it a go. BIND9 of couse runs on Linux, today we’ll be using Ubuntu 18.04.

BIND9 is one of several applications that a non-profit company called “Internet Systems Consortium” produces, you can find their page on BIND9 here. They rely on companies that use BIND9 to purchase professional support, so if you work for an organization that does, please consider reaching out to them.

Topology

This topology includes 3 different routers and 3 different servers. The routers, aptly-named Router1, Router2, and Router3 each have 1 server connected. Two of the routers are Cisco IOSv routers, and Router3 is an Ubuntu box with Free Range Routing installed just for kicks. Routing is set up using OSPF, so everything can already ping everything else. If you’re interested in how set up an Ubuntu 18.04 server as a router with OSPF, please check out my post on installing FRR. In that post I ran EIGRP but configuring OSPF is pretty simple once you get FRR installed.

The DNS server at the top is running Ubuntu 18.04, as well as the generic Ubuntu 18.04 server at the bottom right. There is a Windows 10 desktop at the bottom left to provide some diversity.

Installation

Installation of BIND9 is pretty easy using Ubuntu’s package manager:

apt-get install bind9

Configuration

There are three text files needed to get a basic BIND9 configuration up and running, these are:
/etc/bind/named.conf.options –> Configures BIND9 options
/etc/bind/named.conf.local –> Sets zone file name and gives its location
/etc/bind/zones/db.jamesmcclay.com –> The actual zone file with DNS records.

First order of business is edit /etc/bind/named.conf.options to have a very, very basic configuration:

options {
        directory "/var/cache/bind";
        listen-on { any; };
};

Second, add a zone configuration to named.conf.local that indicates where the zone file will be kept:

zone "jamesmcclay.com" {
    type master;
    file "/etc/bind/zones/db.jamesmcclay.com";
};

Lastly, we’ll create “db.jamesmcclay.com” under the “zones” directory, because that’s what I put in named.conf.local. Obviously you’ll want to probably substitute something else for “jamesmcclay.com”:

@               IN      SOA     dns-server.jamesmcclay.com    dns-server.localhost (
                                2               ; Serial
                                604800  ; Refresh
                                86400           ; Retry
                                2419200 ; Expire
                                604800 )        ; Negative Cache TTL
;
@               IN      NS      dns-server
@               IN      A       172.16.0.2
Router1         IN      A       10.0.0.1
Router2         IN      A       10.0.0.2
Router3         IN      A       11.0.0.1
Win10           IN      A       192.168.0.2
U1804           IN      A       192.168.1.2
dns-server      IN      A       172.16.0.2

The values towards the top are mostly just setting default values for DNS, while the A records at the bottom are more important. Those are the IP addresses that will be returned for hostnames within “jamesmcclay.com”. Gotta make sure those are correct. They all correspond with the IP’s listed in the topology.

Now restart BIND9:

systemctl restart bind9

It should show green if your config is good:

$systemctl status bind9

Configuring the servers and desktops is as simple as setting the DNS server IP address to 172.16.0.2. On Ubuntu linux, you can do it with Netplan. The default config file for Netplan is at /etc/netplan/50-cloud-init.yaml, and I configured UbuntuServer18.04-2 at the bottom right of the topology like this:

network:
    ethernets:
        eth0:
            addresses: [192.168.1.2/24]
            dhcp4: false
            optional: true
            gateway4: 192.168.1.1
            nameservers:
                search: [jamesmcclay.com]
                addresses: [172.16.0.2]
    version: 2

For Windows 10 (and most others, I think) you can sift through the control panel to get to network adapter settings, but I just hit “window key + R” and type “ncpa.cpl” and press enter which takes me to network adapter configuration. My configuration on the Windows 10 desktop at the bottom left of the topology looks like this:

In “Advanced” I also set the search domain to “jamesmcclay.com” which appends that domain to any non-fully qualified hosts entered. You’ll see how that’s helpful in the verification section:

Verification

Simple pings will verify that you’re able to reach various hosts via their DNS names instead of IP addresses. On the UbuntuServer18.04-2 at the bottom right, I pinged the whole group using their fully qualified domain names:

$ping router1.jamesmcclay.com -c 1
64 bytes from 10.0.0.1 (10.0.0.1): icmp_seq=1 ttl=254 time=1.45 ms
$ping router2.jamesmcclay.com -c 1
64 bytes from 10.0.0.2 (10.0.0.2): icmp_seq=1 ttl=254 time=1.03 ms
$ping router3.jamesmcclay.com -c 1
64 bytes from 11.0.0.1: icmp_seq=1 ttl=64 time=0.635 ms
$ping win10.jamesmcclay.com -c 1
64 bytes from 192.168.0.2 (192.168.0.2): icmp_seq=1 ttl=126 time=2.11 ms
$ping dns-server.jamesmcclay.com -c 1
64 bytes from 172.16.0.2 (172.16.0.2): icmp_seq=1 ttl=62 time=0.913 ms

And since I entered search domains in my DNS server configurations, I can ping hosts without fully qualified domain names. From my Windows 10 desktop:

And as usual, you can do a packet capture to verify that the DNS packet flow is working correctly. If I look at packets leaving my Windows 10 desktop, I’ll see a DNS query from the desktop, and a response from BIND9:

Troubleshooting

If you’re having trouble, make sure all of your config files are formatted correctly. They get angry if you don’t format them right. Be sure and do a packet capture if you get stuck, that will pinpoint the problem pretty quickly. Also you can check the logs in the BIND9 server for messages that might have a clue. The command I use is “cat /var/log/syslog | grep bind”.

Apr 19 03:42:16 uvm1804 named[15145]: running as: named -f -u bind<br>Apr 19 03:42:16 uvm1804 named[15145]: loading configuration from '/etc/bind/named.conf'<br>Apr 19 03:42:16 uvm1804 named[15145]: reading built-in trust anchors from file '/etc/bind/bind.keys'<br>Apr 19 03:42:16 uvm1804 named[15145]: set up managed keys zone for view _default, file 'managed-keys.bind'<br>Apr 19 03:42:16 uvm1804 named[15145]: configuring command channel from '/etc/bind/rndc.key'<br>Apr 19 03:42:16 uvm1804 named[15145]: configuring command channel from '/etc/bind/rndc.key'<br>root@dns-server:/etc/netplan#

Loopback Adapter on Ubuntu 18.04 (Like on Cisco)

Background

For many years, Linux users used the old familiar network configuration text file located (usually) at ‘/etc/network/interfaces’ to configure IP addresses, DNS settings, gateway settings, and the like. It was even possible to create a virtual (read: loopback) interface with no associated physical interface using this file. While desktop versions of Ubuntu have used Network Manager for a long time to make things easier from a GUI, the CLI-only nature of an Ubuntu Server doesn’t need all the overhead of Network Manager.

Recent versions of Ubuntu Server (starting with 18.04) have moved to a new network configuration method, using a software package called “Netplan” developed by Ubuntu’s maintainer, Canonical. I’ve read the Ubuntu developers’ reasoning for making this switch, and to make a long story short, their decision was…. justified. The old method had many shortcomings. And while Netplan has a cool new and modern feel including configuration from a YAML file, the one thing that’s missing from it (best I can tell) is a way to create a loopback adapter. I scoured the Intertubes looking for a way to create one via Netplan but finally came to the conclusion that it doesn’t really have one. Others in the Ubuntu community seemed to agree but I could find no official documentation… if you know of some please leave a comment or send me an email, I’d love to know. I’ll update this article if loopback functionality is added to Netplan.

In any case, I’ve been using a method to create loopback adapters on Ubuntu Server and Desktop well before Netplan came along, simply because it’s the way I learned and it seems to work no matter what distribution or version I’m using. Loopback adapters are useful for all sorts of stuff in networking. You can create one, give it an IP address and advertise it in a routing protocol like OSPF, for example. Unlike physical interfaces, loopback adapters are much, much less likely to “go down”.

Topology

Nothing mind-blowing, we have a simple 10.0.0.0/24 network here. My Ubuntu Server 18.04 box will have a physical IP of 10.0.0.1/24 and a loopback adapter with IP address of 1.1.1.1/32. The Cisco IOSv box will have a physical address of 10.0.0.2/24 and a loopback adapter with IP address 2.2.2.2/32. Remember, neither of these boxes “knows” about the other’s loopack address, so we’ll have to do some sort of routing. Static routes are a simple solution, but most networking peeps are going to be looking to do a routing protocol. I’ll do OSPF just for kicks.

Configuration

Since there’s no external software or packages to install, I’ll jump right into configuration. This blog post assumes you have IPv4 routing enabled on both the Ubuntu Server and the Cisco router, and you have configured the network interfaces for 10.0.0.0/24.

Ubuntu18.04-1

On Ubuntu 18.04, you can add a “dummy” adapter with a single command like this:

 $ip link add name loop1 type dummy

Then turn it on and give it an IP address:

$ip link set loop1 up
$ip addr add 1.1.1.1/32 dev loop1

And you’re done. Keep in mind these commands don’t stick when you reboot. You can put these commands in a startup script, I’ll cover that in a another post.

CiscoIOSv15.6(2)T-3

Creating a loopback adapter on a Cisco IOS is simple as well, although I will say it would be nice if it supported CIDR notation (like other Cisco OS’s). Go into config mode and issue these commands:

Router(config)#int loop1
Router(config-if)#ip add 2.2.2.2 255.255.255.255

Verification

Ubuntu18.04-1

A good old fashion ping from the Ubuntu box to its own loopback adapter will tell you if it’s up:

$ping 1.1.1.1
64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=0.038 ms                                                                                                                 
64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.049 ms

The “ip route” command won’t show your /32, so try issuing the “ip addr show” command. To just show your loop1 adapter, add that on the end:

$ip addr show loop1
4: loop1: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether de:9d:7c:37:7a:c2 brd ff:ff:ff:ff:ff:ff
    inet 1.1.1.1/32 scope global loop1
       valid_lft forever preferred_lft forever
    inet6 fe80::dc9d:7cff:fe37:7ac2/64 scope link 
       valid_lft forever preferred_lft forever

CiscoIOSv15.6(2)T-3

From enabled mode you can just issue “show ip int brief”:

Router#show ip int bri
Interface                  IP-Address      OK? Method Status                Protocol                                                                                                     
GigabitEthernet0/0         10.0.0.2        YES manual up                    up                                                                                                           
GigabitEthernet0/1         unassigned      YES NVRAM  administratively down down    
GigabitEthernet0/2         unassigned      YES NVRAM  administratively down down    
GigabitEthernet0/3         unassigned      YES NVRAM  administratively down down    
Loopback1                  2.2.2.2         YES manual up                    up

Optional – Static Routing

A static route is a quick and easy way (but not very scalable) to make the loopback adapters reachable across the network.

Ubuntu18.04-1

On Ubuntu, just issue a single command to add a static route to reach the Cisco router’s loopback. Make :

$ip route add 2.2.2.2/32 via 10.0.0.2

You can now ping it:

ping 2.2.2.2
64 bytes from 2.2.2.2: icmp_seq=1 ttl=255 time=1.36 ms
64 bytes from 2.2.2.2: icmp_seq=2 ttl=255 time=1.72 ms

CiscoIOSv15.6(2)T-3

It’s also a single command on the Cisco router’s configuration mode to add a static route to reach the Ubuntu box’s loopback adapter. You should be able to ping after adding it:

Router(config)#ip route 1.1.1.1 255.255.255.255 10.0.0.1
Router(config)#end
Router#ping 1.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

Optional – OSPF Routing

I have removed the static routes, so routing is no longer in place.

Ubuntu18.04-1

For setting up OSPF routing on Ubuntu, please see my post on that. It’s not too bad, just some commands to install Cumulus Linux Free Range Routing (FRR) from the Canonical Snap Store. Assuming you got that done, jump into the FRR CLI:

$/snap/bin/frr.vtysh
frr#conf t
frr(config)#router ospf
frr(config-router)#network 10.0.0.0/24 area 0
frr(config-router)#network 1.1.1.1/32 area 0
frr(config-router)#end
frr#show ip route
O   1.1.1.1/32 [110/10] via 0.0.0.0, loop1 onlink, 00:02:16
C>* 1.1.1.1/32 is directly connected, loop1, 00:19:40
O>* 2.2.2.2/32 [110/101] via 10.0.0.2, eth0, 00:00:08
O   10.0.0.0/24 [110/100] is directly connected, eth0, 00:03:14
C>* 10.0.0.0/24 is directly connected, eth0, 00:20:25

You’ll get the above route output as soon as the OSPF neighborship is up. You should be able to ping the Cisco’s loopback at 2.2.2.2.

CiscoIOSv15.6(2)T-3

On a Cisco IOS router, OSPF is of course already installed, firing it up is just a couple commands in config mode. Once again, CIDR notation would be nice. Does the world really need wildcard masks? They probably keep it that way to support automated systems that use it, but they could add CIDR notation support. Just sayin’.

Router(config)#router ospf 1
Router(config-router)#network 10.0.0.0 0.0.0.255 area 0
Router(config-router)#network 2.2.2.2 0.0.0.0 area 0
Router(config-router)#end
Router#show ip route
      1.0.0.0/32 is subnetted, 1 subnets
O        1.1.1.1 [110/11] via 10.0.0.1, 00:00:35, GigabitEthernet0/0
      2.0.0.0/32 is subnetted, 1 subnets
C        2.2.2.2 is directly connected, Loopback1
      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.0.0.0/24 is directly connected, GigabitEthernet0/0
L        10.0.0.2/32 is directly connected, GigabitEthernet0/0

You should now be able to ping 2.2.2.2 from the Ubuntu box, and ping 1.1.1.1 from the Cisco router.

As I mentioned earlier, your “ip” commands won’t survive a reboot.

Kea DHCP Server 1.6 on Ubuntu 18.04 with Cisco IOS

Background

The Internet Services Consortium is a 501 non-profit (according to Wikipedia) that has developed and maintains both BIND DNS Server and ISC-DHCP-Server. Both of these open-source projects are used all over the world and are deployed on many, many servers. While ISC-DHCP-Server is handy and easy to configure, ISC has developed a new DHCP server called Kea that they say will eventually replace ISC-DHCP-Server. ISC claims Kea is better in nearly every way, so I’m going to give it a shot.

While all of ISC’s software is open-source, like many open-source maintainers they generate revenue by selling professional support for the software that they develop. Check out their support page for more info.

Topology

Pretty basic here. Kea will be installed on my Ubuntu 18.04 server and an IPv4 network is configured. We’ll see if we can use DHCP to get an IP address on my Cisco IOSv 15.6 router. I’m doing a packet capture in the middle which is always helpful for troubleshooting and verification.

Installation

Installation was pretty easy, provided you are ok with downloading a script from the internet and running it on your system. This method has some serious security concerns, so if you’re planning to run it on a production system you’ll want to probably take a more cautious approach. However my environment is entirely in GNS3 on a cloned Ubuntu VM so I can destroy the whole thing when I’m done. Please be careful out there, don’t talk to strangers.

It took me some time to find the official documentation, but as usual it was at readthedocs.io.

Running the ISC-Consortium’s bash script went pretty smooth for me, this process installs the package repository that ISC provides:

$ curl -1sLf \
>   'https://dl.cloudsmith.io/public/isc/kea-1-6/cfg/setup/bash.deb.sh' \
>   | sudo bash
Executing the  setup script for the 'isc/kea-1-6' repository ...

  OK: Checking for required executable 'curl' ...
  OK: Checking for required executable 'apt-get' ...
  OK: Detecting your OS distribution and release using system methods ...
 ^^^^: OS detected as: ubuntu 18.04 (bionic)
FAIL: Checking for apt dependency 'apt-transport-https' ...
  OK: Updating apt repository metadata cache ...
  OK: Attempting to install 'apt-transport-https' ...
  OK: Checking for apt dependency 'gnupg' ...
  OK: Importing 'isc/kea-1-6' repository GPG key into apt ...
  OK: Checking if upstream install config is OK ...
  OK: Installing 'isc/kea-1-6' repository via apt ...
  OK: Updating apt repository metadata cache ...
 ^^^^: The repository has been installed successfully - You're ready to rock!

$

Once the repository is successfully installed, you can now install the packages:

$apt install isc-kea-dhcp4-server isc-kea-dhcp6-server isc-kea-admin

Configuration

We’ll start with something really simple. I’ll set the range to be from 10.0.0.100 to 10.0.0.200, the default gateway to 10.0.0.1 (the Ubuntu box) and database set to “memfile” which for Kea means written to a csv file. This is pretty much the exact same functionality as ISC-DHCP-Server.

Ubuntu18.04-1


The default configuration file on my Ubuntu 18.04 box was located at /etc/kea/kea-dhcp4.conf. I deleted all the pre-added stuff and replaced with the following basic configuration:

{
# DHCPv4 configuration starts on the next line
"Dhcp4": {

# First we set up global values
    "valid-lifetime": 4000,
    "renew-timer": 1000,
    "rebind-timer": 2000,

# Next we set up the interfaces to be used by the server.
    "interfaces-config": {
        "interfaces": [ "eth0" ]
    },

# And we specify the type of lease database
    "lease-database": {
        "type": "memfile",
        "persist": true,
        "name": "/var/lib/kea/dhcp4.leases"
    },

# Finally, we list the subnets from which we will be leasing addresses.
    "subnet4": [
        {
            "subnet": "10.0.0.0/24",
            "pools": [
                {
                     "pool": "10.0.0.100 - 10.0.0.200"
                }],
                
            "option-data":[
                {
                     "name": "routers",
                     "data": "10.0.0.1"
                }]
            
            }
        ]
    }
}
# DHCPv4 configuration ends with the next line

And restart Kea with

$systemctl restart isc-kea-dhcp4-server

#and also check status:

$systemctl status isc-kea-dhcp4-server

If everything is good, you should see a green dot from systemd telling you Kea is up and running.


In this step it’s really easy to get hung up because of invalid JSON. Make sure you close all your {‘s and [‘s.

If all is well, Kea should be running and ready to hand out DHCP leases.

CiscoIOSv16.6(2)T-1

Configuration of a DHCP client on an IOS router is dead simple. On the interface just issue “ip address dhcp”. The interface section of your show run will look like this:

interface GigabitEthernet0/0
 ip address dhcp
 duplex auto
 speed auto

Verification

Ubuntu18.04-1

Since we’ve specified in our config that we want to store DHCP leases in /var/lib/kea/dhcp4.leases, my first step is to check to see if there’s a lease in there for the Cisco router.

$cat /var/lib/kea/dhcp4.leases
address,hwaddr,client_id,valid_lifetime,expire,subnet_id,fqdn_fwd,fqdn_rev,hostname,state,user_context
10.0.0.100,0c:b0:c4:ed:c8:00,00:63:69:73:63:6f:2d:30:63:62:30:2e:63:34:65:64:2e:63:38:30:30:2d:47:69:30:2f:30,4000,1583687165,1,1,1,router,0,

It’s a little hard to read, but it’s there.

CiscoIOSv16.6(2)T-1

On the Cisco device you’ll be looking for a log message like this:

*Mar  7 22:39:47.135: %DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0/0 assigned DHCP address 10.0.0.100, mask 255.255.255.0, hostname

You’ll also want to check IP interfaces for an IP address and issue “show ip interface brief”:

Interface                  IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0         10.0.0.100      YES DHCP   up                    up      
GigabitEthernet0/1         unassigned      YES unset  administratively down down    
GigabitEthernet0/2         unassigned      YES unset  administratively down down    
GigabitEthernet0/3         unassigned      YES unset  administratively down down

You’ll also want to check the packet capture for the standard DHCP set of 4 messages – Discover, Offer, Request, Acknowledge. Wireshark will recognize “bootp” as a filter:

Troubleshooting

The first sign of trouble will be that you’ll see nothing but “Discover” messages coming from the Cisco router, and no response from Kea. At that point, you’ll want to make sure Kea is running:

Then check the syslog file. On Ubuntu 18.04, this is in /var/log/syslog. Make sure to grep for “kea” because the file is usually pretty big, I used “cat /var/log/syslog | grep kea”:

In this particular instance, I had written invalid JSON in the Kea configuration file, so Kea failed to start.

I also had an instance where I was trying using a MySQL server to store leases, and my configuration wasn’t correct. Kea started but simply didn’t respond to DHCP requests. If you’re using SQL, make sure communication is happening between Kea and your SQL server.

Optional – MySQL Server for Leases storage

One of the options that Kea supports is storing lease data in a database. For this you can use a MySQL server (or others). ISC states in their documentation that the SQL database option is really for needs specific to large deployments, so it’s optional of course.

To install MySQL:

$sudo apt update
$sudo apt install mysql-server

You can check to make sure it’s running:

$systemctl status mysql

Status should be green:

Then you need to jump into mysql and create a database for Kea to use:

$mysql -u root -p
Enter password:
mysql>CREATE DATABASE james_kea;
Query OK, 1 row affected (0.00 sec)
mysql> CREATE USER 'james'@'localhost' IDENTIFIED BY 'james';
Query OK, 0 rows affected (0.01 sec)
mysql> GRANT ALL ON james_kea.* TO 'james'@'localhost';
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
$

Then initialize the Kea database with the kea-admin tool:

kea-admin db-init mysql -u james -p james -n james_kea

You’re going to need to change the “lease-database” section of /etc/kea/kea-dhcp4.conf to tell it to store leases in MySQL:

    "lease-database": {
    #    "type": "memfile", #I just commented these out.
    #    "persist": true,#I just commented these out.
    #    "name": "/var/lib/kea/dhcp4.leases" #I just commented these out.
        "type":"mysql",
        "name":"james_kea",
        "host":"127.0.0.1",
        "user":"james",
        "password":"james"
        "password":"james"
    },

Restart Kea for it to take effect.

As I mentioned before, if Kea can’t communicate with MySQL, it will start but simply not respond to DHCP requests. If you’re in that situation, make sure to check that your SQL and Kea configurations are correct.

You can take a look at the database via the mysql CLI to see if leases are being stored there:

$mysql -u root -p
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| james_kea          |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)
mysql> use james_kea
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+-------------------------------+
| Tables_in_james_kea           |
+-------------------------------+
| dhcp4_audit                   |
| dhcp4_audit_revision          |
| dhcp4_global_parameter        |
| dhcp4_global_parameter_server |
####EDITED FOR BREVITY####
| lease4                        |
| lease4_stat                   |
| lease6                        |
| lease6_stat                   |
| lease6_types                  |
| lease_hwaddr_source           |
| lease_state                   |
| logs                          |
| modification                  |
| parameter_data_type           |
| schema_version                |
+-------------------------------+
mysql> SELECT * FROM lease4;
+-----------+--------+-----------------------------+----------------+---------------------+-----------+----------+----------+----------+-------+--------------+
| address   | hwaddr | client_id                   | valid_lifetime | expire              | subnet_id | fqdn_fwd | fqdn_rev | hostname | state | user_context |
+-----------+--------+-----------------------------+----------------+---------------------+-----------+----------+----------+----------+-------+--------------+
| 167772260 | 
              ����      |  cisco-0cb0.c4ed.c800-Gi0/0 |           4000 | 2020-03-09 02:16:09 |         1 |        1 |        1 | router   |     0 | NULL         |
+-----------+--------+-----------------------------+----------------+---------------------+-----------+----------+----------+----------+-------+--------------+
1 row in set (0.00 sec)

mysql> 

It shows up better on a terminal with a lot of width, but there is lease data stored for my Cisco router. Success.

Optional – MySQL Server/REST API for Configuration Storage

I was really hoping to get to test this, however I learned this functionality is provided through external libraries, or “hooks” that the ISC team offers through a premium package.

The “Configuration Backend”, as they call it, offers the ability to store configurations for networks, some options, and other stuff in MySQL and not a JSON config file. Changes to these configurations can be made through REST calls to the kea-ctrl-agent which processes and strips away the HTTP headers and forwards the JSON data to Kea. Essentially, you could make configuration changes from a script or program on the fly without having to restart or reconfigure the server. Pretty cool, I can see why they charge for it.

As of this writing in March of 2020, the ISC website says you need a premium support subscription for the premium hooks package.

EIGRP Routing on Linux via Free Range Routing

Background

If you are familiar with networking on Linux, you may know of the Quagga project, which is an old open source project that implemented some routing protocols. You may also have heard that a cool company called Cumulus Linux forked Quagga into an open source project they call “Free Range Routing” (Their github page has the most recent code, the project page has more info, and their documentation is extensive) . They have taken the development of the project into high gear and implemented many new features, new protocols, and generally gone above and beyond the original Quagga project which had seen little to no development activity for years.

While you may be able to find some binary packages out there, building from source is the best way to get the most recent code and the most recent features. However building from source can be time consuming if your not familiar with the project and/or the process of building from source. If you just want to poke around a little bit or do some testing, there is a nice alternative using Ubuntu’s “snap” technology. Snaps are kind of like containers, although they’re different from Docker or LXC. The idea is to package up all the libraries the application needs so it can run on any Linux distribution that can run the snap daemon. Stability, portability, and ease of installation are the goal. You can check out more about snap at the Snapcraft page.

Free Range Routing offers snaps through the snap store and installation is downright easy.

If you’re familiar with Cisco technologies, you may know that their proprietary (and yes, it’s still proprietary) routing protocol EIGRP is only available on Cisco devices. In 2013 Cisco published a draft RFC detailing how to implement EIGRP, opening the gate for anyone interested to implement it. There were a few attempts to add it on to Quagga, however none of them really got past alpha stage. Free Range Routing is the first major attempt to implement it, although it’s still in experimental stage. Installing the Free Range Routing snap will get you the latest EIGRP code, and you can run it on your Linux box.

Installation

There is a readme specific to installing the snap package that comes when you install the snap itself, but it’s at the Github page as well of course.

Ubuntu 18.04

Make sure to update, and install snapd:

sudo apt update
sudo apt install snapd

Then install the frr snap:

sudo snap install frr

Then you’ll need to give it permission to control the network:

snap connect frr:network-control core:network-control

Restart your box to finish.

Topology

Pretty simple idea here. We want PC1 to be able to reach PC2. There are no routes in place, so that’s where EIGRP comes in. The Ubuntu box doesn’t know about 172.16.0.0/24 and the Cisco IOSv Router doesn’t know about 192.168.0.0/24, and we’ll use EIGRP to tell each router about those networks. This blog assumes IPv4 routing has been enabled on the Ubuntu box and IP addresses have been configured on the interfaces shown.

Configuration

Ubuntu 18.04

You’ll find the snap files under “/snap”, so cd to “/snap/bin” where you’ll find all the frr stuff. Most importantly you’ll find frr.vtysh which is the command line interface for frr. Running that will take you into the frr CLI, or you can use an absolute path to it:

sudo /snap/bin/frr.vtysh

You’ll be greeted with the frr CLI which is very Cisco-like and based on the original Quagga CLI. You can configure EIGRP using Cisco-like commands:

#conf t
(config)#
(config-router)#network 192.168.0.0/24
(config-router)#network 172.16.0.0/24
(config-router)#end
#

CiscoIOSv15.6(2)T-1

I’ve abbreviated the output of my show run a bit to make things more readable:

interface GigabitEthernet0/0
 ip address 10.0.0.2 255.255.255.252
!
interface GigabitEthernet0/1
 ip address 172.16.0.1 255.255.255.0
!        
router eigrp 10
 network 10.0.0.0 0.0.0.3
 network 172.16.0.0 0.0.0.255

Verification

Ubuntu 18.04

You can use the same show commands within the frr CLI to see if you’re getting routes:

# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

E   10.0.0.0/30 [90/28160] is directly connected, eth0, 00:43:35
C>* 10.0.0.0/30 is directly connected, eth0, 00:46:38
E>* 172.16.0.0/24 [90/5376] via 10.0.0.2, eth0, 00:42:36
E   192.168.0.0/24 [90/28160] is directly connected, eth1, 00:42:47
C>* 192.168.0.0/24 is directly connected, eth1, 00:45:56
# 

Of course you can always drop out with “exit” to the Bash shell and do “ip route”:

$ip route
10.0.0.0/30 dev eth0 proto kernel scope link src 10.0.0.1 
172.16.0.0/24 via 10.0.0.2 dev eth0 proto 192 metric 20 
192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.1 
$ 

CiscoIOSv15.6(2)T-1

It’s nice when the commands are the same (similar) everywhere.

Router#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.0.0.0/30 is directly connected, GigabitEthernet0/0
L        10.0.0.2/32 is directly connected, GigabitEthernet0/0
      172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C        172.16.0.0/24 is directly connected, GigabitEthernet0/1
L        172.16.0.1/32 is directly connected, GigabitEthernet0/1
D     192.168.0.0/24 [90/28416] via 10.0.0.1, 00:44:51, GigabitEthernet0/0
Router#

Finally, make sure PC1 can ping PC2:

PC1> ping 172.16.0.2
84 bytes from 172.16.0.2 icmp_seq=1 ttl=62 time=1.843 ms
84 bytes from 172.16.0.2 icmp_seq=2 ttl=62 time=1.879 ms
84 bytes from 172.16.0.2 icmp_seq=3 ttl=62 time=1.904 ms
84 bytes from 172.16.0.2 icmp_seq=4 ttl=62 time=1.906 ms

EIGRP on Linux!