Month: January 2023

Why has my ethernet connection stopped connecting?

This morning my Ethernet connection between my two Linux systems stopped working. I could see IPV6 stuff flowing over the network, but Linux did not say connected. Also there was no IPV4 address. It took me almost a day to work out what the problem was. Googling and following the advice may have made it worse!

I also include some useful commands for next time it happens.

The high level problem

It looks like the Network Manager has changed.

A week ago, I had files like

/etc/NetworkManager/system-connections/enp0s31f6

containing the definitions for my Ethernet.

Now Network Manager uses

/etc/NetworkManager/system-connections/BTHub6-9999.nmconnection

and these configuration files were missing configuration data. I have a .nmconnection file going back to November, so something has changed.

Further study shows that the

nmcli connection migrate

converts from old format to .nmconnection files, so perhaps this was done under the covers.

Network manager files

Files in /etc/NetworkManager/system-connections/ must be owned by root and be readably only by root – otherwise NetworkManager will ignore it.

Some of my files had the wrong permissions, and so were ignored.

I used

sudo chmod -R 600 /etc/NetworkManager/system-connections/

and restarted NetworkManager

sudo systemctl restart NetworkManager

and missing files reappeared in Network Manager.

For more information about the files see man nm-settings-keyfile.

The detailed problem

Using Wireshark I could see IPV6 traffic flowing over the connection, so the cable was OK, and some of the definitions were OK.

The ip addr command showed there was an IPV6 address for the connection, but no IPV4 address.

I could not find a log for Network Manager with its error messages, see log below for the messages on syslog.

Looking online, there were suggestions that you delete your existing definition and recreate it, also use nm-connection-editor. This may have been a bad move; it is always better to rename than to delete.

Comparing the definitions currently in use /etc/NetworkManager/system-connections/ with a backup version, I could see that the .nmconnection files were in use.

I used Network Manager to change my Ethernet definitions. Under the IPv4 tab

  • IPv4 method: change from Automatic(DHCP) to Manual
  • Address: Added 10.1.0.2 Netmask 255.255.255.0
  • Route: Added 10.1.0.3 Netmask 255.255.255.255.0 Gateway 10.1.0.2

The route statement says to get to 10.1.0.3 go via 10.1.0.2 .

Once I restarted the connection it became active, and the ip -4 addr command showed it had an IPv4 address.

For the other end of the connection I did the matching changes and the end to end connection burst into life!

For my Ethernet connection my file was 

[connection]
id=Wired connection 1
uuid=ecc4df76-4733-45f5-9b67-9fba9ef2d3bf
type=ethernet
interface-name=enp0s31f6
permissions=
timestamp=1673353909

[ethernet]
mac-address-blacklist=

[ipv4]
address1=10.1.0.2/24

dns-priority=100
dns-search=
method=manual
route1=10.1.0.3/24,10.1.0.2

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto

[proxy]

This defines the IP address 10.1.0.2, and a route to 10.1.0.3 via 10.1.0.2 .

Useful commands

Display the devices

nmcli d

nmcli device

gives

DEVICE             TYPE      STATE         CONNECTION         
wlp4s0             wifi      connected     BTHub6-78RQ        
enp0s31f6          ethernet  connected     Wired connection 1 
virbr0             bridge    connected     virbr0  d

so for my en0s31f6 device, the connection file is ‘Wired connection 1’

Display the connection

nmcli c

nmcli connection

NAME                UUID                                  TYPE       DEVICE    
BTHub6-78RQ         fc74c8e0-6f96-4e8b-a8ba-6389abbe3396  wifi       wlp4s0    
Wired connection 1  ecc4df76-4733-45f5-9b67-9fba9ef2d3bf  ethernet   enp0s31f6 
virbr0              386a5a3a-023b-41d9-9138-04202d8dfda6  bridge     virbr0

Display more information

nmcli -f all c |less

Display only some fields

nmcli -f name,device,FILENAME c |less

gives

NAME                DEVICE     FILENAME                                                                    
BTHub6-78RQ         wlp4s0     /etc/NetworkManager/system-connections/BTHub6-78RQ.nmconnection.old2        
Wired connection 1  enp0s31f6  /etc/NetworkManager/system-connections/Wired connection 1.nmconnection.old2 
virbr0              virbr0     /run/NetworkManager/system-connections/virbr0.nmconnection

Displaying trace

When the Ethernet connection work, /var/log/syslog had entries

 NetworkManager[11240]: <info>  [....0612] device (enp0s31f6): Activation: starting connection 'enp0s31f6' (c066ca29-2253-41ef-8e69-2251fb15f7b8)
 NetworkManager[11240]: <info>  [....0617] audit: op="connection-activate" uuid="c066ca29-2253-41ef-8e69-2251fb15f7b8" name="enp0s31f6" pid=2585 uid=1000 result="success"
 NetworkManager[11240]: <info>  [....0636] device (enp0s31f6): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
 NetworkManager[11240]: <info>  [....0678] device (enp0s31f6): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
 NetworkManager[11240]: <info>  [....0718] device (enp0s31f6): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
 avahi-daemon[1123]: Joining mDNS multicast group on interface enp0s31f6.IPv6 with address fe80::9b07:33a1:aa30:e272.
 avahi-daemon[1123]: New relevant interface enp0s31f6.IPv6 for mDNS.
 avahi-daemon[1123]: Registering new address record for fe80::9b07:33a1:aa30:e272 on enp0s31f6.*.
 avahi-daemon[1123]: Joining mDNS multicast group on interface enp0s31f6.IPv4 with address 10.1.0.2.
 avahi-daemon[1123]: New relevant interface enp0s31f6.IPv4 for mDNS.
 avahi-daemon[1123]: Registering new address record for 10.1.0.2 on enp0s31f6.IPv4.
 NetworkManager[11240]: <info>  [....0830] device (enp0s31f6): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
 NetworkManager[11240]: <info>  [....1005] device (enp0s31f6): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
 NetworkManager[11240]: <info>  [....1008] device (enp0s31f6): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
 NetworkManager[11240]: <info>  [....1021] device (enp0s31f6): Activation: successful, device activated.

When the connection was defined as DHCP the trace was 

NetworkManager: <info>  [...] device (enp0s31f6): state change: ip-config -> deactivating (reason 'user-requested', sys-iface-state: 'managed')
NetworkManager: <info>  [...] audit: op="device-disconnect" interface="enp0s31f6" ifindex=2 pid=2585 uid=1000 result="success"
NetworkManager: <info>  [...] device (enp0s31f6): state change: deactivating -> disconnected (reason 'user-requested', sys-iface-state: 'managed')
avahi-daemon: Withdrawing address record for fe80::78e8:9e55:9f3f:768 on enp0s31f6.
avahi-daemon: Leaving mDNS multicast group on interface enp0s31f6.IPv6 with address fe80::78e8:9e55:9f3f:768.
avahi-daemon: Interface enp0s31f6.IPv6 no longer relevant for mDNS.
NetworkManager: <info>  [...] dhcp4 (enp0s31f6): canceled DHCP transaction
NetworkManager: <info>  [...] dhcp4 (enp0s31f6): state changed unknown -> done
NetworkManager: <info>  [...] device (enp0s31f6): Activation: starting connection 'Wired connection 1' (ecc4df76-4733-45f5-9b67-9fba9ef2d3bf)
NetworkManager: <info>  [...] device (enp0s31f6): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
NetworkManager: <info>  [...] device (enp0s31f6): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
NetworkManager: <info>  [...] device (enp0s31f6): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
NetworkManager: <info>  [...] dhcp4 (enp0s31f6): activation: beginning transaction (timeout in 45 seconds)
avahi-daemon: Joining mDNS multicast group on interface enp0s31f6.IPv6 with address fe80::78e8:9e55:9f3f:768.
avahi-daemon: New relevant interface enp0s31f6.IPv6 for mDNS.
avahi-daemon: Registering new address record for fe80::78e8:9e55:9f3f:768 on enp0s31f6.*.

This has entries about DHCP.

My machine did not have a DHCP server installed – so any request for DHCP will fail to get an address.

Backing up on Linux is easy – how do I restore?

I’ve been using deja-dup front end to duplicity to backup my Ubuntu Linux. This has worked fine. The only problems I had were trying to restore files I had not backed up (/etc….) – a Defective End User problem.

Restoring one of these files was a little challenge, with bumps in the road… here’s how I did it. I’ve listed the errors I received while using duplicity.

A high level thought – should you change the repository name every year – such as including the year?

Where am I backing up to (where do I restore from)?

Use the deja-dup application (use the Linux show applications and find backup).

Under Storage location will be location and folder.

If this is an external drive it may be mounted under /mnt/… or /media/.

High level view

duplicity collection-status file:///media/colinpaice/UbuntuBackup/home/Backup2023

gave

Last full backup date: Sun Jan  1 17:02:48 2023
Collection Status
-----------------
Connecting with backend: BackendWrapper
Archive directory: /home/colinpaice/.cache/duplicity/52b5c01605d52d0976f1434440a7c11a

Found 0 secondary backup chains.

Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Sun Jan  1 17:02:48 2023
Chain end time: Tue Jan 10 08:19:05 2023
Number of contained backup sets: 10
Total number of contained volumes: 250
 Type of backup set:                            Time:   Number of volumes:
                Full         Sun Jan  1 17:02:48 2023               238
         Incremental         Mon Jan  2 08:25:38 2023                 1
         Incremental         Tue Jan  3 08:50:23 2023                 1
         Incremental         Wed Jan  4 08:53:01 2023                 1
         Incremental         Thu Jan  5 08:22:41 2023                 1
         Incremental         Fri Jan  6 08:36:31 2023                 4
         Incremental         Sat Jan  7 10:23:00 2023                 1
         Incremental         Sun Jan  8 12:16:58 2023                 1
         Incremental         Mon Jan  9 09:02:31 2023                 1
         Incremental         Tue Jan 10 08:19:05 2023                 1
-------------------------
No orphaned or incomplete backup sets found.

List files in the backup

duplicity list-current-files file:///media/colinpaice/UbuntuBackup/home/Backup2023/

I saved the list of files using

duplicity list-current-files file:///media/colinpaice/UbuntuBackup/home/Backup2023 >dup.lst
Synchronising remote metadata to local cache…
GnuPG passphrase for decryption:

This took over a minute (it has to check thousands of files names). I think it scans every “signature” file to build a list. The more signature files – the longer it takes.

Inside dup.lst it had entries like

Tue Aug 14 11:41:33 2018 etc/NetworkManager/system-connections/LG-D855_2292

This says the backed up file was from the given date in 2018.

List files available on a given date

I used

duplicity list-current-files –time 5D file:///media/colinpaice/UbuntuBackup/home/Backup2023 > aa.lst

to get a list of files available 5 days ago.

Restore the file

duplicity restore -t 3D –file-to-restore etc/NetworkManager/system-connections/LG-D855_2292 file:///media/colinpaice/UbuntuBackup/home/Backup2023 /home/colinpaice/secret/abc

where

  • -t ( or –time) specify a time period
  • 3D for three days ago. Other options s, m, h, D, W, M, or Y. 1M3d is 33 days ago.
  • etc/Network…. this is the file I wanted to restore. Note no leading / .
  • file:///media…/home/Backup2023 from this backup location.
  • /home/colinpaice/secret/abc and put the file here. The file/directory must not exist.

Restore a directory

Specify the folder (ending in /) and specify a target directory (/home/colinpaice/secret/abcd). It will create the directory if needed.

If you specify a name like etc/…/* it will report it was not found in archive.

duplicity restore -t 3D –file-to-restore etc/NetworkManager/system-connections/ file:///media/colinpaice/UbuntuBackup/home/Backup2023 /home/colinpaice/secret/abcd

Error messages

gpg: WARNING: unsafe permissions on homedir

This says the contents of your secret directory are publicly available. Change the permissions.

chmod 600 /home/colinpaice/.gnupg/*
chmod 700 /home/colinpaice/.gnupg/

Local and Remote metadata are synchronized, no sync needed.

This is OK. It means the metadata is the same in the local cache as in the remote store.

duplicity.dup_collections.CollectionsError: No signature chains found

You have the wrong backup repository name,

  • it needs ///
  • ls should give a filename like /media/colinpaice/UbuntuBackup/…./duplicity-new-signatures.20230101T170248Z.to.20230102T082538Z.sigtar.gpg

Error ‘[Errno 1] Operation not permitted: b’/home/colinpaice/secret/abc” processing .

After the restore, duplicity was not able to use chown on the file to restore it to the original owner.

GPGError: GPG Failed, see log below:

===== Begin GnuPG log =====
gpg: AES256 encrypted data
gpg: gcry_kdf_derive failed: Invalid data
gpg: encrypted with 1 passphrase
gpg: decryption failed: No secret key
===== End GnuPG log =====

I got this asking for a date before backups were taken. See

duplicity collection-status file:///media/colinpaice/UbuntuBackup/home/Backup2023

I was looking for a backup in 2002 – which was not in the 2003 backup.

Configuring frr on Linux

The FRR package is “a fully featured, high performance, free software IP routing suite.
It implements all standard routing protocols such as BGP, RIP, OSPF, IS-IS and more (see Feature Matrix), as well as many of their extensions”. The package was developed from the zebra and quagga packages.

This looks an excellent package which works well. There is a lot of good documentation, but the documentation is not entirely accurate, and some of the examples do not work.

Looking at the code helped, but I found that that using the administration tool, almost does away with the need for documentation to configure frr – but the path is not obvious.

I’ll give the steps I took to configure router ospf6 in frr.

I’m assuming you have frr active, and

sudo frr -m -f /etc/frr/frr.conf

runs with no errors. This checks the syntax of the configuration file (/etc/frr/frr.con).

Copy this configuration file, because when frr configuration writes the configuration file, the file is replaced, and may be missing stuff.

Invoke the configuration tool.

sudo vtysh

This gives me

Hello, this is FRRouting (version 7.2.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

laptop#

where laptop was the hostname name I specified in the /etc/frr/vtysh.conf file. This is a useful facility when you are using multiple systems.

You can use the up arrow and down arrow to scroll through the command history.

You can use ? to display available commands and options.

Configuring the ospf6 router

#configure terminal
#router ospf6

This gives

laptop(config-ospf6)#

so you know you are in the configuration of the ospf6 router.

  • A question mark ? – gives the available commands (and options).
  • There is command-complete so you only need to type the first few characters followed by tab for it to complete the command. For example if “¬” is the tab key, list p¬ gives list permutations. Interface ¬ gives the name of the interfaces so int¬ e¬ completes to interface enp0s31f6; int¬ e¬ ¬ gives interface enp0s31f6 area.
  • The list command gives a short summary of the commands and options, for example 
interface IFNAME area A.B.C.D
...
ospf6 router-id A.B.C.D

Capital letters indicate where you specify values.

If you do not know which command option to use, you can use ?. 

laptop(config-ospf6)# ?
  area                   OSPF6 area parameters
  auto-cost              Calculate OSPF interface cost according to bandwidth
  distance               Administrative distance
  end                    End current mode and change to enable mode
  exit                   Exit current mode and down to previous mode
  find                   Find CLI command matching a regular expression
  interface              Enable routing on an IPv6 interface
  list                   Print command list
  log-adjacency-changes  Log changes in adjacency state
  no                     Negate a command or set its defaults
  ospf6                  Open Shortest Path First (OSPF) for IPv6
  output                 Direct vtysh output to file
  quit                   Exit current mode and down to previous mode
  redistribute           Redistribute
  stub-router            Make router a stub router
  timers                 Adjust routing timers

laptop(config-ospf6)#ospf6 ?

gave

laptop(config-ospf6)# router-id Configure OSPF6 Router-ID

this means the command is ospf6 router…. and the description of the name is Configure OSPF6 Router-ID.

laptop(config-ospf6)#ospf6 router ?

gave

laptop(config-ospf6)# A.B.C.D specify by IPv4 address notation(e.g. 0.0.0.0)

so the command is ospf6 router A.B.C.D

ospf6 router-id 2.2.2.2

this worked with no error messages.

To get out of config-ospf6# use

end

this gets you out of config-ospf6 and back to 

laptop#

You can display the current status

laptop# write terminal

to save it to file

write file

gave

laptop# write file
Note: this version of vtysh never writes vtysh.conf
Building Configuration...
Integrated configuration saved to /etc/frr/frr.conf
[OK]
laptop#

Note: I had the ospf6 agent running, and had stopped the rip agent. The rip configuration was not written to the file! You can use your backed up copy and re-enter it.

To exit vtysh 

laptop# quit

Displaying current status

You can use the show command to display information from frr. It works the same way. show ? gives you the command syntax. It has command-completion.

One-liner commands

You can use sudo vtysh -c “show ipv6…” to display one command (and then pipe it into grep for example).