Re: Why ping latency is smaller with shorter send interval?

[Eric Dumazet]

On 10/2/20 10:51 AM, 叶小龙 wrote:
> Hi, net experts,
> 
> Hope this is the right place to ask the question :)
> 
> Recently I've tried to measure the network latency between two
> machines by using ping, one interesting observation I found is that
> ping latency will be smaller if I use a shorter interval with -i
> option. For example,
> 
> when I use default ping (interval is 1s), then the ping result is as
> below with avg latency 0.062ms
> 
> # ping 9.9.9.2 -c 10
> PING 9.9.9.2 (9.9.9.2) 56(84) bytes of data.
> 64 bytes from 9.9.9.2: icmp_seq=1 ttl=64 time=0.059 ms
> 64 bytes from 9.9.9.2: icmp_seq=2 ttl=64 time=0.079 ms
> 64 bytes from 9.9.9.2: icmp_seq=3 ttl=64 time=0.060 ms
> 64 bytes from 9.9.9.2: icmp_seq=4 ttl=64 time=0.072 ms
> 64 bytes from 9.9.9.2: icmp_seq=5 ttl=64 time=0.048 ms
> 64 bytes from 9.9.9.2: icmp_seq=6 ttl=64 time=0.069 ms
> 64 bytes from 9.9.9.2: icmp_seq=7 ttl=64 time=0.067 ms
> 64 bytes from 9.9.9.2: icmp_seq=8 ttl=64 time=0.055 ms
> 64 bytes from 9.9.9.2: icmp_seq=9 ttl=64 time=0.058 ms
> 64 bytes from 9.9.9.2: icmp_seq=10 ttl=64 time=0.055 ms
> 
> --- 9.9.9.2 ping statistics ---
> 10 packets transmitted, 10 received, 0% packet loss, time 9001ms
> rtt min/avg/max/mdev = 0.048/0.062/0.079/0.010 ms
> 
> Then I use "-i 0.001", the lateny (0.038) is way better than defaut ping
> 
> # ping 9.9.9.2 -i 0.001 -c 10
> PING 9.9.9.2 (9.9.9.2) 56(84) bytes of data.
> 64 bytes from 9.9.9.2: icmp_seq=1 ttl=64 time=0.069 ms
> 64 bytes from 9.9.9.2: icmp_seq=2 ttl=64 time=0.039 ms
> 64 bytes from 9.9.9.2: icmp_seq=3 ttl=64 time=0.034 ms
> 64 bytes from 9.9.9.2: icmp_seq=4 ttl=64 time=0.033 ms
> 64 bytes from 9.9.9.2: icmp_seq=5 ttl=64 time=0.033 ms
> 64 bytes from 9.9.9.2: icmp_seq=6 ttl=64 time=0.033 ms
> 64 bytes from 9.9.9.2: icmp_seq=7 ttl=64 time=0.034 ms
> 64 bytes from 9.9.9.2: icmp_seq=8 ttl=64 time=0.036 ms
> 64 bytes from 9.9.9.2: icmp_seq=9 ttl=64 time=0.037 ms
> 64 bytes from 9.9.9.2: icmp_seq=10 ttl=64 time=0.038 ms
> 
> --- 9.9.9.2 ping statistics ---
> 10 packets transmitted, 10 received, 0% packet loss, time 9ms
> rtt min/avg/max/mdev = 0.033/0.038/0.069/0.012 ms
> 
> 
> ping loopback shows the similar result.
> 
> Default ping avg latency is 0.049ms
> 
> # ping 127.0.0.1 -c 10
> PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
> 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.032 ms
> 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.049 ms
> 64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.054 ms
> 64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.058 ms
> 64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.049 ms
> 64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.042 ms
> 64 bytes from 127.0.0.1: icmp_seq=7 ttl=64 time=0.052 ms
> 64 bytes from 127.0.0.1: icmp_seq=8 ttl=64 time=0.052 ms
> 64 bytes from 127.0.0.1: icmp_seq=9 ttl=64 time=0.053 ms
> 64 bytes from 127.0.0.1: icmp_seq=10 ttl=64 time=0.055 ms
> 
> --- 127.0.0.1 ping statistics ---
> 10 packets transmitted, 10 received, 0% packet loss, time 9001ms
> rtt min/avg/max/mdev = 0.032/0.049/0.058/0.010 ms
> 
> ping with "-i 0.001" shows 0.014ms avg latency.
> 
> # ping 127.0.0.1 -i 0.001 -c 10
> PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
> 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.040 ms
> 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.014 ms
> 64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.012 ms
> 64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.011 ms
> 64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.011 ms
> 64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.011 ms
> 64 bytes from 127.0.0.1: icmp_seq=7 ttl=64 time=0.011 ms
> 64 bytes from 127.0.0.1: icmp_seq=8 ttl=64 time=0.010 ms
> 64 bytes from 127.0.0.1: icmp_seq=9 ttl=64 time=0.010 ms
> 64 bytes from 127.0.0.1: icmp_seq=10 ttl=64 time=0.011 ms
> 
> --- 127.0.0.1 ping statistics ---
> 10 packets transmitted, 10 received, 0% packet loss, time 9ms
> rtt min/avg/max/mdev = 0.010/0.014/0.040/0.008 ms
> 
> I'm using centos 7.2 with kernel 3.10.
> 
> I am very confused about the result. As I understand it, it doesn't
> matter how frequently I send packets, each packet's latency should be
> the same. So How can I understand it from network stack point of view?
> 
> Any thoughts or suggestions would be highly appreciated.
> 
> 
> Thanks,
> Xiaolong
> 

Many factors in play here.

1) if you keep cpus busy enough, they tend to keep in their caches
the data needed to serve your requests. In your case, time taken to
process an ICMP packet can be very different depending on how hot
cpu caches are.

2) Idle cpus can be put in a power conserving state.
  It takes time to exit from these states, as you noticed.
  These delays can typically be around 50 usec, or more.

Search for cpu C-states , and powertop program.