How to reduce the chance of screwing up in a Linux window.

I had multiple Linux terminal windows open, doing SSH to different machines. I typed shutdown in the wrong window – and the wrong server shutdown!

I had configured different profiles so I could have a white background, a green background and a yellow background for my different systems, but I had got careless and not used them.

I found a neat way of colouring the windows automatically.

xdotool is a command-line X11 automation tool, which allows you to programmatically press keys. You can use this to set the profile of a terminal window.

To create a profile

From the hamburger options,

preferences,
profiles +
give the profile a name, create
select the named profile, it will display customising options
colours
untick Use colours from system theme
Click Text or Background
Pick a colour, Select
Close the window

Select a profile

Manually

From the hamburger option
- Select Profile
- Pick a profile
or Shift+f10
- r (for Profile)
- 3 for the third option in the list

Programmatically

xdotool key shift+F10 r 3

This does Shift+10, then select r for Profile, then picks the 3rd option

The clever bit

You can make an alias such as

alias somehost="xdotool key shift+F10 r 3; ssh user@somehost; xdotool key shift+F10 r 2"

or a somehost.sh script

#!/bin/sh
xdotool key shift+F10 r 3
ssh me@10.1.0.5
xdotool key shift+F10 r 2

If you enter the somehost command – it will select the 3rd profile, do the ssh. On exit from ssh it resets it back to the 2nd profile.

Restore files from Linux using Duplicity

Duplicity is a program which manages backup and restore of files on your Linux machine.

What is duplicity?

Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server.

Backing up files

You run the backup application, and can specify the directories to backup. You can specify which directories to ignore.

The backups can be stored

Google drive
Networks server
Locally attached drive
Local Folder

That’s the easy bit.

What files are backed up?

I have my backups going to an external USB drive /media/colinpaice/UbuntuBackup1/home/Backup2024 on my Linux machine. The files have names like

duplicity-full.20240308T084825Z.vol295.difftar.gpg

The command

duplicity list-current-files file:///media/colinpaice/UbuntuBackup1/home/Backup2024 >files2024

Restore a file

duplicity restore -t 3D –file-to-restore ~/ssl/ssl2/rsaca256.csr file:///media/colinpaice/UbuntuBackup/home/Backup2021 ~/ssl/ssl2/rsa256.csr

This restores a file

-t 3D from 3 days ago
–file-to-restore ~/ssl/ssl2/rsaca256.csr
file:///media/colinpaice/UbuntuBackup/home/Backup2021 from this device
~/ssl/ssl2/rsa256.csr to this file

What next

Duplicity can do much more than this. Ive just provided information on the most basic stuff that I have used. See the products web page or man duplicity

Configuring frr on Linux

frr is a network router for Linux systems.

It works well, and has a lot of good documentation about all of the commands and options, but it does doesn’t have a “getting started” section. It is a bit like getting the wiring diagram for a car, when all you to do is get in and drive the car.

Below are some of the ~~things I stumbled across~~ lessons I learned.

Some later definitions override earlier definitions.

With

router ospf6
  ospf6 router-id 6.6.6.6  
  ospf6 router-id 6.6.6.7

The definition used is 6.6.6.7 because it overrides the earlier 6.6.6.6

interface AB
 description colins AB first
  ip ospf area 0.0.0.0
  ip ospf area 20.0.0.0

In this case the area 0.0.0.0 is used – so the order does not look consistent.

interface eno1 
   ipv6 nd prefix 2001:db8:6099::/64
   ipv6 nd prefix 2008::/64

 
interface eno1
   description server2 ospf6
   ipv6 ospf6 instance-id 2

The description is description server2 ospf6 because it was the only one specified.

Both of the nd prefix values are used.

What is used where?

If you want to configure a daemon, you needs to specify parameters in different places.

For example, the ospf6 daemon

router ospf6
  ospf6 router-id 6.6.6.6
...  

interface enp0s31f6
  ipv6 ospf6 area 0.0.0.0

The router ospf6 configuration is for the daemon thread.

The information on the interface…. statement is for each interface. This means all of the configuration information for an interface is under the interface…. section.

To display the ospf6 configuration use

show running-config ospf6

It extracts the ospf6 interesting information from the configuration

Current configuration:
!
frr version 8.1
...
hostname Server
!
debug ospf6 zebra
debug ospf6 interface
!
interface eno1
 description server2 ospzz
 ipv6 ospf6 area 0.0.0.0
 ipv6 ospf6 instance-id 2
 ipv6 ospf6 network point-to-point
exit
!

router ospf6
 ospf6 router-id 6.6.1.1
 redistribute static
 redistribute ripng
exit
!
end

and ignored other information like

debug ospf zebra
debug ospf interface 

interface eno1 
   ipv6 nd prefix 2001:db8:6099::/64
   ipv6 nd prefix 2008::/64

The debug ospf… is ignored because this relates to ospf not to ospf6.

The ipv6 nd is ignored because this is not ospf6 related.

Configuring using vtysh

You can use vtysh to configure your /etc/frr/frr.cntl for example

vtysh
configure
interface vl100
ipv6 address 3000::1/64
do write
quit
quite
quit

it is a good idea to make a copy of the /etc/frr/frr.conf before you do this.

Show doesn’t always show

You can use the command

show running-config

and that shows you most of what is configured and running.

Note that default values may not be displayed.

When my definitions had

interface eno1 
   ip ospf hello-interval 11

the show running-configuration gave

interface eno1
 description server ospzz
 ip address 10.1.0.3 peer 10.1.0.2/24
 ip ospf area 0.0.0.0
 ip ospf dead-interval 40
 ip ospf hello-interval 11

When I had ip ospf hello-interval 10 the output did not include ip ospf hello-interval 10 because it was the default value.

You can use sudo vtysh

show running config
write file

to rewrite the /etc/frr/frr.conf control file with the defaults removed, and the frr version line updated.

If you use the command

show ip ospf interface

it gives you the values for each interface such as

Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit 5

OSPF defaults

these were in the file lib/libospf.h in the frr source.

 OSPF_MIN_LS_INTERVAL                  5000     /* msec */
 OSPF_MIN_LS_ARRIVAL                   1000     /* in milliseconds */
 OSPF_LSA_INITIAL_AGE                     0      /* useful for debug */
 OSPF_LSA_MAXAGE                       3600
 OSPF_CHECK_AGE                         300
 OSPF_LSA_MAXAGE_DIFF                   900
 OSPF_LS_INFINITY                  0xffffff
 OSPF_DEFAULT_DESTINATION        0x00000000      /* 0.0.0.0 */
 OSPF_INITIAL_SEQUENCE_NUMBER    0x80000001U
 OSPF_MAX_SEQUENCE_NUMBER        0x7fffffffU


/* OSPF interface default values. */
 OSPF_OUTPUT_COST_DEFAULT           10
 OSPF_OUTPUT_COST_INFINITE          UINT16_MAX
 OSPF_ROUTER_DEAD_INTERVAL_DEFAULT  40

 OSPF_ROUTER_DEAD_INTERVAL_MINIMAL   1
 OSPF_HELLO_INTERVAL_DEFAULT        10
 OSPF_ROUTER_PRIORITY_DEFAULT        1
 OSPF_RETRANSMIT_INTERVAL_DEFAULT    5
 OSPF_TRANSMIT_DELAY_DEFAULT         1
 OSPF_DEFAULT_BANDWIDTH           10000  /* Mbps */

 OSPF_DEFAULT_REF_BANDWIDTH      100000  /* Mbps */

 OSPF_POLL_INTERVAL_DEFAULT         60
 OSPF_NEIGHBOR_PRIORITY_DEFAULT      0

 OSPF_MTU_IGNORE_DEFAULT             0
 OSPF_FAST_HELLO_DEFAULT             0

 OSPF_AREA_BACKBONE              0x00000000      /* 0.0.0.0 */
 OSPF_AREA_RANGE_COST_UNSPEC     -1U

 OSPF_AREA_DEFAULT       0
 OSPF_AREA_STUB          1
 OSPF_AREA_NSSA          2
 OSPF_AREA_TYPE_MAX      3

/* SPF Throttling timer values. */
 OSPF_SPF_DELAY_DEFAULT              0
 OSPF_SPF_HOLDTIME_DEFAULT           50
 OSPF_SPF_MAX_HOLDTIME_DEFAULT       5000

 OSPF_LSA_MAXAGE_CHECK_INTERVAL          30
 OSPF_LSA_MAXAGE_REMOVE_DELAY_DEFAULT    60

Setting up for PING can be difficult!

Setting up ping within one machine is trivial
Setting up ping between two machines is relatively easy
Setting up ping with 3 machines can be hard to get right and to get working.

Most documentation describes how to set up PING between two machines, and does not mention anything more complex.

Ancient history (well 1980’s)

To understand modern TCPIP networks, it helps to know about the origins of Ethernet.

Originally with Ethernet there was a bus; a very long cable. You plugged your computer into this bus. Each computer on the bus had a unique 48 bit MAC address. You could send a request to another computer on this bus using their Ethernet address. You could send a request to ALL computers on this bus. These days this might be, send to all computers:”does anyone have this IP address…?”

An Ethernet bus is connected to an Ethernet router which can route packets to other routers.

Ethernet has evolved and instead of a long bus shared by all users, you have a switch device, and you plug the Ethernet cable from your computer to the switch. Conceptually it is the same, but with the switch you get much better performance and usability. You still have routers to get between different Ethernet environments. The request send to all computers:”does anyone have this IP address…?” goes to the switch, and the switch sends it to all plugged in computers.

When you start TCPIP, it configures the Ethernet hardware adapter with the addresses is should listen for.

Other terminology and concepts

A router is a networking device which forwards packets between different networks. Packets of data get sent from the originator through zero or more routers to the final destination.
A gateway has several meanings
- It can connect different network types, for example act as a protocol translator, it can have a built-in firewall
- It can route IP traffic between different IP subnets
- It can loosely mean a router

Usually a router or gateway is a dedicated device or hardware.

You can have a computer act as a router or gateway; it takes data from one interface and routes it to a different interface. The computer can pass the data through a fire wall, or transform it, for example converting internal IP addresses to external IP addresses (NAT translation).

At a concept level, a computer’s network adapter is configured only to listen for packets which match the IP addresses of the interface, and ignore the rest.

If you want a computer to act as a router or gateway, you need to configure the network adapter to listen to all traffic, and to process the traffic. This is important when routing traffic from computer A through computer B to computer C.

A simple one hop ping, first time

I have set up my laptop to talk to a server over Ethernet.

I have configured my laptop using

sudo ip -6 addr add 2001:7::/64 dev enp0s31f6

This says for any IP V6 address starting with 2001:0007:0000:0000 then try sending it down the Ethernet cable with interface name enp0s31f6.

Ping a non existing address

If I try to ping an address say 2001:7::99 a packet is sent to all computers on the Ethernet bus

from myipAddress to everyone on the Ethernet bus, does anyone have IP address 2001:7::99?

There is no reply because no computer on the bus, has the address.

Ping the adjacent box

On the server, the IP address of the Ethernet cable is 2001:7::2.

If I ping this address, there are the following flows

From myipAddress (and myMAC) to everyone on the Ethernet bus, does anyone have IP address 2001:7::2?
From 2001:7::2 to myipAddress (MAC), yes I have that IP address. My Ethernet MAC address is …

My laptop puts the remote IP address and its MAC address into its neighbour cache, then issues the ping request.

The ping request looks in the neighbour cache find the IP address and MAC address and issues.

From myipAddress to 2001:7::2 at macAddress .. ping request.

The second and later times I issue a ping

The ping request looks in the neighbour cache find the IP address and MAC address and issues the ping

From myipAddress to 2001:7::2 at macAddress .. ping request.

So while the neighbour cache still has the IP of the target, and it’s MAC address, the ping can be routed to the next hop.

Setting up a multi hop ping

I have my laptop connected via a physical Ethernet cable to my server. The server is connected to z/OS via a virtual Ethernet connection. The IP address of the z/OS end of the virtual Ethernet cable is 2001:7::4.

The ping 2001:7::4 request on my laptop, does not work (as we saw above), because TCP asked everyone on the Ethernet bus if it has address 2001:7::4 – and no machine replies.

You need to define the link to the server machine as a gateway or router-like device which can handle IP addresses which are not on the Ethernet bus. You define it like

sudo ip -6 route add 2001:7::/64 via 2001:7::2

This says for any traffic with IP address 2001:0007:0000:0000:* send it via address 2001:7::2.

This requires 2001:7::2 to be known about, and so needs the following to be configured first

sudo ip -6 route add 2001:7::2 dev enp0s31f6

for TCPIP to know where to send the traffic to.

The route add 2001:7::2 dev enp0s31f6 command sends a broadcast on the Ethernet bus asking – does anyone have 2001:7::2. My server replies saying yes I have it. This is the same as for the ping above.

In summary

To send traffic on the same Ethernet bus you use

sudo ip -6 route add 2001:7::2 dev enp0s31f6

To route it via a router, switch, or computer acting as a router or switch you need.

sudo ip -6 route add 2001:7::/64 via 2001:7::2 or
sudo ip -6 route add 2001:7::/64 via 2001:7::2 <dev enp0s31f6>

The computer acting as a router or switch may need additional configuration. For example to allow it to route traffic from one Ethernet bus to another Ethernet bus you need to enable packet forwarding. On Linux to enable forwarding for all interfaces use

sysctl -w net.ipv6.conf.all.forwarding=1

On z/OS you use a static route such as

ROUTE 2001:7::/64 2001:7::3 JFPORTCP6

Where 2001:7::/64 is the range of addresses 2001:7::3 is (one of) the addresses of the interface at the remote end of the cabl, and JFPORTCP6 is the interface name. This is similar to the Linux route statement above.

You might need to set up the firewall. On my server I needed

sudo ufw route allow in on eno1 out on tap2

What IP Address does the sender have?

This is where is starts to get more complex.

Every network connection has at least one IP address.

With IP V6

Each interface gets an “internal” IP address such as fe80::9b07:33a1:aa30:e272
You can allocate an external address using the Linux command sudo ip -6 addr add 2001:7::1 dev enp0s31f6
If the interface has one external IP address configured, then this will be used.
If the interface has more than one external address configured then the first in the list may be used (not always).
If the interface does not have an external IP address, then TCPIP will find one from another interface, or allocate one for the duration of the request, such as 2a00:23c5:978f:9999:210a:1e9b:94a4:c8e. This address comes from my wireless connection

When my laptop is started, only the wireless connection has an IPV6 address. A ping request to 2001:7::4 had the origin IP address of 2a00:23c5:978f:9999:210a:1e9b:94a4:c8e which is the first address in the list for the wireless connection.

You can tell ping which address to use, for example

ping -I 2a00:23c5:978f:9999:cff1:dc13:4fc6:f21b 2007:1::4 (this fails because the server does now know to send the response to the requester)

I defined a new address for the interface using

sudo ip -6 addr rep 2002:7::1 dev enp0s31f6

I could issue ping -I 2002:7::1 2001:7::4, but this failed to get a response, because the back-end and intermediate nodes, did not know how to get the response back to 2002:7::1

Does this sender address matter?

Yes, because the remote end needs to have a definition to be able to send the response back.

I had my laptop connected to a Linux server over Ethernet, which in turn was connected to z/OS over a virtual Ethernet.

On z/OS I could see the ping request arrive, but it could not send the response back because it did not know how to get to 2a00:23c5:978f:9999….

I configured the laptop end of the Ethernet to give it an IP address 2001:7::1

sudo ip -6 addr add 2001:7::1/64 dev enp0s31f6

I configured the server to laptop to have an IP address of 2001:7::2

sudo ip -6 route add 2001:7::1/128 dev eno1

I configured the server to z/OS with an IP address and a route

sudo ip -6 addr add 2001:7::3/128 dev tap2
sudo ip -6 route add 2001:7::4/128 dev tap2

Now when I did the ping, the originator was 2001:7::1.

I configured the z/OS interface to send stuff back

INTERFACE JFPORTCP6 
    DEFINE IPAQENET6 
    CHPIDTYPE OSD 
    PORTNAME PORTC 
                                   
INTERFACE JFPORTCP6 
    ADDADDR 2001:DB8:8::9 
INTERFACE JFPORTCP6 
    ADDADDR 2001:DB8::9 
INTERFACE JFPORTCP6 
    ADDADDR 2001:7::4 
START JFPORTCP6

and the routing

BEGINRoutes 
;     Destination      Gateway                 LinkName  Size 
ROUTE 2001:7::/64      2001:7::3              JFPORTCP6   MTU 5000 
...
ENDRoutes

This says that all traffic with destination address 2001:0007:0000:000…. send to interface JFPORTCP6. This interface is connected to a gateway with the address of the remote end of the Ethernet (so on the server) of 2001:7::3.

The server machine needs to act as a router between the different Ethernet buses. You can display and configure this using

sysctl -a |grep forwarding
sudo sysctl -w net.ipv6.conf.all.forwarding=1

Packet forwarding on z/OS

By default the z/OS interface only listens for packets with one of the IP addresses of the interface. For z/OS to be able to be a router; accept all packets, and route them to other interfaces you need:

IPCONFIG DATAGRAMFWD in the TCP/IP Profile
PRIROUTER on the Interface definition . This configures the Ethernet adapter (hardware) so If a datagram is received at this device for an unknown IP address, the datagram is routed to this TCP/IP instance.

But normally if you are running z/OS you use a cheaper, physical router, rather than use the z/OS to do your routing. It might only be people like me who run z/OS on their laptop who want to try routing through z/OS.

MFA: setting up Linux as an authenticator to generate a TOTP password

Overview of the process

From Linux, send the userid and password request in a URL from Linux to https://….:6793/AZFTOTP1/genericToken.
This sends back a response with
- “status”:0,
- “rc”:0,
- “rsn”:0,
- “preflightPathBase”:”/AZFTOTP1/preflight”,
- “inviteCode”:”HXKNSSWMMDIZ2DVB1R2VNTGA2MOUJDKE”,
- “otpauthURL”:”otpauth://totp/T012@MYREALM
- ?secret=MQB4T…2Y
- &issuer=MYREALM
- &algorithm=SHA1
- &digits=6
- &period=30″}
The data is
- The “preflightPathBase” is the URL to send back the code to
- The “inviteCode” identifies the specific request to MFA
- The secret is the value used to generate the OTP – it should be kept securely
- Algorithm specifies the algorithm to be used, for example SHA1, or SHA256
- The number of digits to be generated in the numeric code
- The period when it should generate a new code, typically 30 seconds.
The oathtool is invoked passing the secret and the algorithm etc. and it generates a code
The code is sent back to the requester to complete the registration process

The bash script

url="https://10.1.1.2:6793"
http=$url"/AZFTOTP1/genericToken"

data='{"userid":"T012","password":"11111111"}'

x=$(curl --config curlyubi.parms --tlsv1.2  --data $data $http  )
# echo "response" $x

rc=$(echo $x | cut -d',' -f 1 )
rsn=$(echo $x | cut -d',' -f 2 )
echo "rc:"$rc
# check it is a valid response
if [ "$rc" = "{\"status\":0" ]; then
   echo "valid....."
else
  echo "RC:"$rc "RSN:"$rsn $x
  return 
fi
# extract the data from the input data - split at ',' and return the nth field
# remove surrounding quotes etc
preFlight=$(echo $x | cut -d',' -f 4 |awk -F'":"|"' '{print $3}')
echo "preflight:"$preFlight 

invite=$(echo $x | cut -d',' -f 5 |awk -F'":"|"' '{print $3}')
#a5=$(echo $x | cut -d',' -f 5 )
echo "invite code:"$invite

secret=$(echo $x | cut -d',' -f 6 | awk -F'secret=|&issuer' '{print $2}')
#echo "secret is "$secret

alg=$(echo $x | cut -d',' -f 6 | awk -F'algorithm=|&digits' '{print $2}')
#echo "algorithm" $alg

digits=$(echo $x | cut -d',' -f 6 | awk -F'digits=|&period' '{print $2}')
#echo "digits:"$digits

period=$(echo $x | cut -d',' -f 6 | awk -F'period=|"}' '{print $2}')

# echo "period:" $period
# generate the code
tss="--time-step-size="$period
code=$(oathtool --totp=$alg  --digits=$digits $tss --window=0 --base32 $secret)

# and send it back
ht=$url""$preFlight"/"$invite"/"$code
y=$(curl $trace --config curlyubi.parms --tlsv1.2 -X GET $ht  )
echo "Final response" $y

Where the bash code does

rsn=$(echo $x | cut -d’,’ -f 2 ) says take the data in variable x, split it at “,” (cut -d’,’) and take the second field (-f 2)
awk -F’digits=|&period’ ‘{print $2}’ says split the data into … “digits=” …. “&period” and take the second piece of data, the ….

When this ran successfully it generated

{“msg”:”yes”,”status”:0}

When I reran it I got a message from the first curl request

response {“rc”:8,”rsn”:23}

if you register using the web browser, the message is

TOTP Enrollment is not available for your User ID. Your account is already provisioned.

In the ./htdocs/js/totpUtils.js is the mapping from rsn code to message.

Generating a code when needed

Whenever you execute

code=$(oathtool –totp=$alg –digits=$digits $tss –window=0 –base32 $secret)

a new code is generated. This means you need to retain the various parameters ( secret, digits etc) on Linux. Having the parameters in a file may be insecure, so you may want to consider encrypting the file with the parameters, and use, for example, a dongle with the private key on it. This means to be able to access the parameters you need the dongle.

What is supported?

The oathtool help says the supported algorithms are “SHA1”, “SHA256”, or “SHA512” The MFA product also supports SHA384 – but cannot be used with authtool.

Which terminal am I in?

I use Ubuntu as my main work station, and found I was getting confused with so many terminals windows open. I found the following helped me manage it.

From a terminal Ctrl-Alt-T gives you a new tab with a new terminal.

I can quickly move between them using Ctrl-PgDn and Ctrl-Pg-Up – just like in a browser.

I can rearrange them using Ctrl-Alt-PgDN and Ctrl-Alt-PgUp, so my commonly used tabs are adjacent.

I can colour the terminal sessions. Select a terminal, right click, preferences. You can now create more profiles, such as “Blue” or “QMPROD” and select the colour of the font and background. I can then, from my terminal, right click, profiles, and select “Blue”. As long as I remember what I use each colour for – it is a great help.