Understanding traceroute (or tracerte)

I was trying to use traceroute to find the route between two nodes and I did not understand the output. Like many things, once you understand it, is obvious.

This is another of the little topics which I thought I understood, and found I did not.

For example

traceroute ibm.com

produces

traceroute to ibm.com (23.39.199.16), 30 hops max, 60 byte packets
 1  bthub.home (192.168.1.254)  4.139 ms  7.528 ms  10.629 ms
 2  * * *
 3  * * *

What does the “*” mean – and why are there 3 “*”?

On Linux there is the command

ip -6 route get to 2001:db8:1::9

which display information like

2001:db8:1::9 from :: via fe80::a2f0:9936:ddfd:95fa dev enp0s31f6 proto ra src 2001:db8::e02a:bfec:a02e:f17 metric 100 pref medium

Traceroute logic

At a conceptual level, the logic of traceroute is:

At the origin

  • Send a UDP packet over a link towards the remote node, with hop limit = 1.
  • Set a timer
  • Wait for the reply (with time out)

At an intermediate node

  • Set hop limit = hop limit -1
  • If hop limit = 0
    • then send a UDP packet back to the originator, giving the IP address of the intermediate node, and information like “Destination unreachable (Port unreachable)”, or “request timed out”.
    • else send the packet over a link towards the remote destination.

Back at the origin

  • Wait for the response. When the response arrives, stop the timer and calculate the duration.
  • Lookup the IP address of the intermediate node, to find the node name.
  • Display original hop count, the name of the intermediate node, IP address of the intermediate node, and duration of the request.

For example

1 colin.Linux.Server (2001:db8::2) 0.988 ms

This gives you information on the first hop in the chain.

Go to the next level.

You can take this further.

  • Repeat the operation multiple times. This allows you to get multiple response times, so you can see the range of responses times, and get an idea of the variation (or consistency) of the response time.
  • Repeat it with hop count = 2,3,4,5… . When the hop count is 1, you get information about the first hop, when the hop count is 2, you get information about the next hop etc.

For example for Linux

traceroute to 2001:db8:1::9 (2001:db8:1::9), 30 hops max, 80 byte packets
1 colin.Linux.Server (2001:db8::2)0.267 ms 0.207 ms 0.140 ms
2 Colin.zOS (2001:db8:1::9) 3.794 ms 6.215 ms 6.920 ms

It gets more interesting.

If you send multiple request, a node may decide to route the request down a different link, so you may get multiple IP addresses for each hop.

What if there is a problem?

Unknown address

Traceroute will report as much as it can. For example 2001:db8:1::10 does not exist.

traceroute to 2001:db8:1::10 (2001:db8:1::10), 30 hops max, 80 byte packets
1 colinpaice (2001:db8::7) 3053.091 ms !H 3052.807 ms !H

This reports as far as it got (colinpaice 2001:db8::7); and !H. On Linux you can have additional information (!H)

  • !H host unreachable
  • !N network unreachable
  • !P protocol unreachable
  • !S source route failed
  • !F fragmentation needed
  • !X communication administratively prohibited
  • !V host precedence violation
  • !C precedence cutoff in effect
  • !<num> ICMP unreachable code <num>

Lost or dropped packets

A intermediate node may not be able to send the response back, for example, a firewall may block (and drop) any UPD packets. The originator times-out waiting for the reply. In this case it reports “*” as the IP address, and cannot provide the duration of the requests. This can occur if the router does not support traceroute, there is no link back to the originator, or there is a firewall which drops packets (going out, or coming back).

More advanced requests

Specify a different home

By default traceroute uses the IP address of the connection it will use to send the packet.

For example I have a system with two interfaces

  • tap1,my end of the connection is 2001:db8:1::3 with the remote end having 2001:db8:1::9 (my z/OS)
  • eno1, my end of the connection is 2001:db8::2 with the remote end having 2001:db8::7 (my laptop)

If I use traceroute to my laptop, I can issue

traceroute 2001:db8::7

by default traceroute uses 2001:db8::2 as its starting point (the IP adddess of the direct connection). I can see this in the wireshark trace.

I can use traceroute to my laptop , and say start from the IP address of the connection to z/OS

traceroute6 -s 2001:db8:1::3 2001:db8::7

-s says use a different starting address 2001:db8:1::3 – corresponding to the link to z/OS as its starting point.

When I used this command, my request was blocked, as my firewall was configured to accept traffic from 2001:db8:2, and not from 2001:db8:1::3.

Make traceroute fail quicker

The traceroute defaults are to try a maximum of 30 hops and wait 5 seconds so you could wait for over 2 minutes if there was a problem). If you know your network is small (at most 3 hops) and responds in under a second, you can use

traceroute -6 -w 2 -q 5 -m 3 2001:db8::7

  • -6 for IP V6 (or just use traceroute6)
  • -w 2 wait for up to 2 seconds
  • -q 1 send out 1 UPD request on each hop
  • -m 3 a maximum of 3 hops.

On z/OS use the tracerte command

tso tracerte 2001:db8::7 (try 1 wait 1 max 2