Using frr (routing program) on Linux

It took me a day to get frr (Free Range Routing) working on Linux. Some of this was due to missing documentation, and getting it started was a problem until I found the golden path which worked.

What is frr?

frr is an offshoot of quagga, which provides ospf, and rip services etc for IP routing on Linux.

Install frr

I used

sudo apt install frr frr-doc

This creates a userid frr, a group frr and may connect your userid to the group.

Check this with

grep frr /etc/group

This gave me

frrvty:x:146:frr
frr:x:147:

I added myself to the group, so I could edit the configuration files

sudo usermod -a -G frr colin

This does not take effect until next time you logon. In the mean time you can use sudo… to access the files.

It may start up every reboot. To disable this use

sudo systemctl disable frr

and

sudo systemctl enablr frr

to restart at reboot.

You can use

sudo /etc/init.d/frr start

sudo /etc/init.d/frr stop

sudo /etc/init.d/frr restart

Configuration files

You need several configuration files, in /etc/frr. I had to use

sudo nano /etc/frr/…

because gedit did not work in sudo mode.

Make changes; use ctrl-s to save, and ctrl-x to exit.

/etc/frr/daemons

This file says which daemons to start. I was only interested in ripngd, and the parameters to pass to the daemons.

I think the comments about the config apply to the frr.conf and vtysh.conf.

# This file tells the frr package which daemons to start.
#
# Sample configurations for these daemons can be found in
# /usr/share/doc/frr/examples/.
#
# ATTENTION:
#
# When activating a daemon for the first time, a config file, even if it is
# empty, has to be present *and* be owned by the user and group "frr", else
# the daemon will not be started by /etc/init.d/frr. The permissions should
# be u=rw,g=r,o=.
# When using "vtysh" such a config file is also needed. It should be owned by
# group "frrvty" and set to ug=rw,o= though. Check /etc/pam.d/frr, too.
#
# The watchfrr and zebra daemons are always started.
#
bgpd=no
ospfd=no
ospf6d=no
ripd=no
ripngd=yes
isisd=no
pimd=no
ldpd=no
nhrpd=no
eigrpd=no
babeld=no
sharpd=no
pbrd=no
bfdd=no
fabricd=no
vrrpd=no

#
# If this option is set the /etc/init.d/frr script automatically loads
# the config via "vtysh -b" when the servers are started.
# Check /etc/pam.d/frr if you intend to use "vtysh"!
#
vtysh_enable=yes
#zebra_options="  -A 127.0.0.1 -s 90000000 --config_file /etc/frr/frr.conf"
zebra_options="  -A 127.0.0.1 -s 90000000 "
bgpd_options="   -A 127.0.0.1"
ospfd_options="  -A 127.0.0.1"
ospf6d_options=" -A ::1"
ripd_options="   -A 127.0.0.1"
ripngd_options=" -A ::1 "
isisd_options="  -A 127.0.0.1"
pimd_options="   -A 127.0.0.1"
ldpd_options="   -A 127.0.0.1"
nhrpd_options="  -A 127.0.0.1"
eigrpd_options=" -A 127.0.0.1"
babeld_options=" -A 127.0.0.1"
sharpd_options=" -A 127.0.0.1"
pbrd_options="   -A 127.0.0.1"
staticd_options="-A 127.0.0.1"
bfdd_options="   -A 127.0.0.1"
fabricd_options="-A 127.0.0.1"
vrrpd_options="  -A 127.0.0.1"

#
# This is the maximum number of FD's that will be available.
# Upon startup this is read by the control files and ulimit
# is called.  Uncomment and use a reasonable value for your
# setup if you are expecting a large number of peers in
# say BGP.
#MAX_FDS=1024

# The list of daemons to watch is automatically generated by the init script.
#watchfrr_options=""

# for debugging purposes, you can specify a "wrap" command to start instead
# of starting the daemon directly, e.g. to use valgrind on ospfd:
#   ospfd_wrap="/usr/bin/valgrind"
# or you can use "all_wrap" for all daemons, e.g. to use perf record:
#   all_wrap="/usr/bin/perf record --call-graph -"
# the normal daemon command is added to this at the end.

/etc/frr/vtysh.conf

This provides configuration information for the command tool:

service integrated-vtysh-config
hostname laptop
password  zebra
log file /var/frr/vtysh.log debug

• service integrated-vtysh-config this says use one config file (/etc/frr/frr.conf) rather than one per daemon (as used in quagga)
• hostname laptop when using vtysh it puts this value at the start of each line (so you know which system you are working with)
• password zebra I do not know when this is used
• log file /var/frr/vtysh.log debug I do not know when this is used.

You may want to omit the password.

/etc/frr/frr.conf

The option service integrated-vtysh-config above says use one configuration file (the integrated option) /etc/frr/frr.conf . If service integrated-vtysh-config is not specified, you need one config file per daemon.

frr version 7.2.1
frr defaults traditional
hostname Router
log file /var/log/frr/frr.log
log timestamp precision 3
ipv6 forwarding
hostname colinpaice
hostname vtysh3
service integrated-vtysh-config
!
debug ripng events
debug ripng packet
!
enable password zebra
password zebra
!
router ripng
  network enp0s31f6
  network wlp4s0
!
line vty
!

• log file /var/log/frr/frr.log You can write to the syslog daemon or to a file. It defaults to log syslog informational See logging below.
• log timestamp precision 3 Records written to the log have millisecond accuracy (6 gives microseconds). I changed this when trying to get frr to work, to check the config file was being picked up
• debug ripng events this writes information such as time expired to the log.
• debug ripng packet this prints out the data sent and received, for example the addresses.
• enable password zebra
• password zebra
• router ripng this is configuration for the ripng daemon.
  • network enp0s31f6
  • network wlp4s0

Structure of the file

Within the config file you can have

interface enp0s31f6
   ip ospf area 0.0.0.0
   ip ospf hello-interval 30
   description colins ospf first


interface enp0s31f6
 description colins ospf second

if you use vtysh

laptop# show interface enp0s31f6 
Interface enp0s31f6 is up, line protocol is up
  Link ups:       0    last: (never)
  Link downs:     0    last: (never)
  vrf: default
  Description: colins ospf second

In this case the second definition overrides the first definition.

With a ip ospf area 0.1.0.0 in the second definition I got message

Must remove previous area config before changing ospf area 
line 33: Failure to communicate[13] to ospfd, line:  ip ospf area 0.1.0.0

Starting and stopping frr

frr starts even though the configuration has problems, and does not provide any diagnostic information.

To check the configuration file syntax

sudo vtysh -m -f /etc/frr/frr.conf

This displays the file, and reports any errors.

Once frr has started there is a command

sudo vtysh -c “show startup-config”

which is meant to display the contents of the start up configuration file. For me this produced no output.

The following command does display the running configuration.

sudo vtysh -c “show running-config”

Starting frr.

The documentation says

Integrated configuration mode
Integrated configuration mode uses a single configuration file, frr.conf, for all daemons. This replaces the individual files like zebra.conf or bgpd.conf.
frr.conf is located in /etc/frr. All daemons check for the existence of this file at startup, and if it exists will not load their individual configuration files. Instead, vtysh -b must be invoked to process frr.conf and apply its settings to the individual daemons.

It looks like the configuration file is not used until vtysh -b has been issued; vtysh sends the configuration file to the daemons.

I used a script

sudo rm /var/log/frr/frr.log
sudo touch /var/log/frr/frr.log
sudo chown frr:frr /var/log/frr/frr.log

sudo /etc/init.d/frr stop 
sleep 1s
sudo /etc/init.d/frr start 
sudo systemctl start ripngd.service
sleep 1s
sudo /etc/init.d/frr status

sleep 1s
less /var/log/frr/frr.log*
ls -ltr /var/log/frr/

• I could have used sudo /etc/init.d/frr restart instead of stop and start.
• The log file must exist, and have the correct owner:group.

When I ran vtysh -b I got messages

can’t open logfile /var/log/frr/frr.log
line 4: Failure to communicate[13] to zebra, line: log file /var/log/frr/frr.log

Configuration file[/etc/frr/frr.conf] processing failure: 13

which basically means the file does not exist, or has the wrong owner.

When running I had the following threads running

colinpaice@colinpaice:~$ ps -ef |grep frr
root 5107 1 0 09:09 ? 00:00:00 /usr/lib/frr/watchfrr -d zebra ripngd staticd
frr  5124 1 0 09:09 ? 00:00:00 /usr/lib/frr/zebra -d -A 127.0.0.1 -s 90000000
frr  5129 1 0 09:09 ? 00:00:00 /usr/lib/frr/ripngd -d -A ::1
frr  5133 1 0 09:09 ? 00:00:00 /usr/lib/frr/staticd -d -A 127.0.0.1
 

Displaying and configuring frr.

You can use the command

sudo vtysh

or

sudo vtysh -c “show running-config”

To execute commands to frr.

If configured you can use commands

telnet localhost zebra

but vtysh is easier to type.

You can issue

sudo vtysh -c “show ?”

to show the options on the show command.

sudo vtysh -c “show ipv6 ripng”

gave me

Codes: R - RIPng, C - connected, S - Static, O - OSPF, B - BGP
Sub-codes:
      (n) - normal, (s) - static, (d) - default, (r) - redistribute,
      (i) - interface, (a/S) - aggregated/Suppressed

   Network      Next Hop                      Via     Metric Tag Time
C(i) 2a00:23c5:978f:6e01::/64 
                  ::                          self       1    0  

Displaying is not that easy

I had defined an interface with

interface enp0s31f6
   ipv6 ospf6 instance-id 1
   ipv6 nd prefix 2001:db8:5099::/64
   ipv6 ospf6 network point-to-point
   ipv6 ospf6 advertise prefix-list 2001:db8:2::/64
   ipv6 ospf6 advertise prefix-list 2001::/64
   ip ospf area 0.0.0.0
   ip ospf hello-interval 30
   description colins ospf first

interface enp0s31f6
 description colins ospf second

When I had the ospf daemon running, but not the ospf6 daemon, the show running command gave

interface enp0s31f6
 description colins ospf second
 ip ospf area 0.0.0.0
 ip ospf hello-interval 30
 ipv6 nd prefix 2001:db8:5099::/64
!

When both daemons were running the show running command gave

interface enp0s31f6
 description colins ospf second
 ip ospf area 0.0.0.0
 ip ospf hello-interval 30
 ipv6 nd prefix 2001:db8:5099::/64
 ipv6 ospf6 advertise prefix-list 2001::/64
 ipv6 ospf6 instance-id 1
 ipv6 ospf6 network point-to-point

including the ospf6 information.

The show interface enp0s31f6 command gave

Interface enp0s31f6 is up, line protocol is up
  Link ups:       0    last: (never)
  Link downs:     0    last: (never)
  vrf: default
  Description: colins ospf second
  index 2 metric 0 mtu 1500 speed 1000 
  flags: <UP,BROADCAST,RUNNING,MULTICAST>
  Type: Ethernet
  HWaddr: 8c:16:45:36:f4:8a
  inet 10.1.0.2/24
  inet6 2001:db8::1/128
  inet6 fe80::78e8:9e55:9f3f:768/64
  Interface Type Other

This has some information from my configuration (description) and information from querying the system ( HWaddress, ip addresses).

Logging

If you are logging to syslogd, either by design or default, if you remove the log file, and restart frr you may get messages like

Jan 03 08:51:31 colin-ThinkCentre-M920s systemd[1]: Started FRRouting.
can't open logfile /var/log/frr/frr.log
line 7: Failure to communicate[13] to zebra, line: log file /var/log/frr/frr.log debug 

You need to restart the syslogd daemon, for example

systemctl restart rsyslog.service

If you are logging to syslogd, there is an frr file /etc/rsyslog.d/45-frr.conf which defines the log file as

$outchannel frr_log,/var/log/frr/frr.log

The log file filling up

After day’s usage I noticed the files in the log directory:

ls -ltr  /var/log/frr/
total 1720
-rw-r--r-- 1 frr frr   51171 Jan  4 18:40 frr.log.1.gz
-rw-r--r-- 1 frr frr 1701760 Jan  6 16:37 frr.log

it looks like it does log maintenance, and compresses old logs.