wiki:TipAndDoc/VM/network/iproute

Version 3 (modified by mitty, 15 years ago) (diff)

--

iproute2

  • ../ のネットワークを使用

centos-inner

  • パケットの往路と復路が異なってしまう問題への対応

設定前

  • centos-inner ~]$ route
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
    192.168.50.0    *               255.255.255.0   U     0      0        0 eth0
    10.2.0.0        *               255.255.0.0     U     0      0        0 eth2
    10.0.0.0        10.1.0.254      255.255.0.0     UG    0      0        0 eth1
    10.1.0.0        *               255.255.0.0     U     0      0        0 eth1
    169.254.0.0     *               255.255.0.0     U     0      0        0 eth2
    default         192.168.50.1    0.0.0.0         UG    0      0        0 eth0
    
    • 10.0.0.0/16のルートについては設定済み
  • centos-inner ~]$ ip route
    192.168.50.0/24 dev eth0  proto kernel  scope link  src 192.168.50.110
    10.2.0.0/16 dev eth2  proto kernel  scope link  src 10.2.0.110
    10.0.0.0/16 via 10.1.0.254 dev eth1
    10.1.0.0/16 dev eth1  proto kernel  scope link  src 10.1.0.110
    169.254.0.0/16 dev eth2  scope link
    default via 192.168.50.1 dev eth0
    
    • centos-inner ~]$ ip route show dev eth0
      192.168.50.0/24  proto kernel  scope link  src 192.168.50.110
      default via 192.168.50.1
      
    • centos-inner ~]$ ip route show dev eth1
      10.0.0.0/16 via 10.1.0.254
      10.1.0.0/16  proto kernel  scope link  src 10.1.0.110
      
    • centos-inner ~]$ ip route show dev eth2
      10.2.0.0/16  proto kernel  scope link  src 10.2.0.110
      169.254.0.0/16  scope link
      
  • centos-inner ~]$ ip rule
    0:      from all lookup 255
    32766:  from all lookup main
    32767:  from all lookup default
    
  • centos-outer <=(router)=> centos-inner
  • centos-outer ~]$ ping 10.2.0.110 -c 1
    1. centos-outer:eth1
    2. ubuntu-router:eth0
    3. ubuntu-router:eth2
      17:18:03.739868 IP 10.0.0.20 > 10.2.0.110: ICMP echo request, id 31500, seq 1, length 64
      
    4. centos-inner:eth2
      17:18:03.794897 IP 10.0.0.20 > 10.2.0.110: ICMP echo request, id 31500, seq 1, length 64
      
    5. centos-inner:eth1
      17:18:03.795150 IP 10.2.0.110 > 10.0.0.20: ICMP echo reply, id 31500, seq 1, length 64
      
    6. ubuntu-router:eth1
      17:18:03.741063 IP 10.2.0.110 > 10.0.0.20: ICMP echo reply, id 31500, seq 1, length 64
      
    7. ubuntu-router:eth0
    8. centos-outer:eth1

設定

  • routing table 設定
    1. centos-inner ~]$ sudo ip route add default via 10.1.0.254 table 1
    2. centos-inner ~]$ sudo ip route add default via 10.2.0.254 table 2
  • routing policy 設定
    1. centos-inner ~]$ sudo ip rule add from 10.1.0.110 table 1 prio 1
    2. centos-inner ~]$ sudo ip rule add from 10.2.0.110 table 2 prio 2
  • routing policy に関しては、centos-innerはeth1とeth2でネットワークが分かれているので以下のような設定でも良い
    1. centos-inner ~]$ sudo ip rule add from 10.1.0.0/16 table 1 prio 1
    2. centos-inner ~]$ sudo ip rule add from 10.2.0.0/16 table 2 prio 2
    • centos-inner ~]$ ip rule
      0:      from all lookup 255
      1:      from 10.1.0.0/16 lookup 1
      2:      from 10.2.0.0/16 lookup 2
      32766:  from all lookup main
      32767:  from all lookup default
      

反映

  • ip route list table main
    192.168.50.0/24 dev eth0  proto kernel  scope link  src 192.168.50.110
    10.2.0.0/16 dev eth2  proto kernel  scope link  src 10.2.0.110
    10.0.0.0/16 via 10.1.0.254 dev eth1
    10.1.0.0/16 dev eth1  proto kernel  scope link  src 10.1.0.110
    169.254.0.0/16 dev eth2  scope link
    default via 192.168.50.1 dev eth0
    
    • main(default)のtableには変化無し
  • ip route list table 1
    default via 10.1.0.254 dev eth1
    
  • ip route list table 2
    default via 10.2.0.254 dev eth2
    
  • ip rule
    0:      from all lookup 255
    1:      from 10.1.0.110 lookup 1
    2:      from 10.2.0.110 lookup 2
    32766:  from all lookup main
    32767:  from all lookup default
    

結果

  • eth1(10.1.0.110)に入ってきたパケットはeth1から、eth2(10.2.0.110)に入ってきたパケットはeth2から送出されるようになる。
  • centos-outer ~]$ ping 10.2.0.110 -c 1
    1. centos-outer:eth1
    2. ubuntu-router:eth0
    3. ubuntu-router:eth2
      17:09:25.667375 IP 10.0.0.20 > 10.2.0.110: ICMP echo request, id 22028, seq 1, length 64
      17:09:25.668310 IP 10.2.0.110 > 10.0.0.20: ICMP echo reply, id 22028, seq 1, length 64
      
    4. centos-inner:eth2
      17:09:25.682422 IP 10.0.0.20 > 10.2.0.110: ICMP echo request, id 22028, seq 1, length 64
      17:09:25.682547 IP 10.2.0.110 > 10.0.0.20: ICMP echo reply, id 22028, seq 1, length 64
      
  • centos-outer ~]$ ping 10.1.0.110 -c 1
    1. centos-outer:eth1
    2. ubuntu-router:eth0
    3. ubuntu-router:eth1
      17:12:29.065247 IP 10.0.0.20 > 10.1.0.110: ICMP echo request, id 25356, seq 1, length 64
      17:12:29.075733 IP 10.1.0.110 > 10.0.0.20: ICMP echo reply, id 25356, seq 1, length 64
      
    4. centos-inner:eth1
      17:12:29.111101 IP 10.0.0.20 > 10.1.0.110: ICMP echo request, id 25356, seq 1, length 64
      17:12:29.116137 IP 10.1.0.110 > 10.0.0.20: ICMP echo reply, id 25356, seq 1, length 64
      

補足

  • ルールを削除する場合は、table あるいは priority を指定して削除する
    0:      from all lookup 255
    1:      from 10.1.0.110 lookup 1
    2:      from 10.2.0.110 lookup 2
    32766:  from all lookup main
    32767:  from all lookup default
    
    • sudo ip rule del table 1
    • sudo ip rule del prio 1

Really, for historical reasons ip rule add does not require a priority value and allows them to be non-unique. If the user does not supplied a priority, it is selected by the kernel. If the user creates a rule with a priority value that already exists, the kernel does not reject the request. It adds the new rule before all old rules of the same priority. It is mistake in design, no more. And it will be fixed one day, so do not rely on this feature. Use explicit priorities.

  • prio(priorityの省略形)の代わりに、preferも使える模様
    • sudo ip rule add from 10.1.0.110 table 1 prefer 1
    • ip rule
      0:      from all lookup 255
      1:      from 10.1.0.110 lookup 1
      32766:  from all lookup main
      32767:  from all lookup default
      
  • routing table はキャッシュされるので、すぐに反映されない場合がある
    • cacheの確認 => ip route show cache
    • cacheの破棄 => sudo ip route flush cache

Warning: Changes to the RPDB made with these commands do not become active immediately. It is assumed that after a script finishes a batch of updates, it flushes the routing cache with ip route flush cache.

ubuntu-inner-AB

  • routing table 設定
    1. sudo ip route add default via 10.1.0.254 table 1
    2. sudo ip route add default via 10.2.0.254 table 2
  • routing policy 設定
    1. sudo ip rule add from 10.1.0.0/16 table 1 prio 1
    2. sudo ip rule add from 10.2.0.0/16 table 2 prio 2
    • 上記#centos-innerと同様に、NICのIP or networkのどちらでも良い

ubuntu-inner-AA

  • routing table 設定
    1. sudo ip route add default via 10.1.0.254 dev eth0 table 10
    2. sudo ip route add default via 10.1.0.254 dev eth1 table 20
  • routing policy 設定
    1. sudo ip rule add from 10.1.0.10 table 10 prio 1
    2. sudo ip rule add from 10.1.0.20 table 20 prio 2
  • requestに関しては、eth1(10.1.0.20)ではなくeth0(10.1.0.10)側に届いてしまう
    • VMなのが問題なのか、route設定がおかしいのか謎
  • ubuntu-outer:~$ ping 10.1.0.20 -c 1
    PING 10.1.0.20 (10.1.0.20) 56(84) bytes of data.
    64 bytes from 10.1.0.20: icmp_seq=1 ttl=63 time=18.0 ms
    
    --- 10.1.0.20 ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 18.019/18.019/18.019/0.000 ms
    
    1. ubuntu-outer:eth1
      03:21:26.529593 IP 10.0.0.10 > 10.1.0.20: ICMP echo request, id 28200, seq 1, length 64
      03:21:26.547598 IP 10.1.0.20 > 10.0.0.10: ICMP echo reply, id 28200, seq 1, length 64
      
    2. ubuntu-router:eth0
      03:21:26.528120 IP 10.0.0.10 > 10.1.0.20: ICMP echo request, id 28200, seq 1, length 64
      03:21:26.535301 IP 10.1.0.20 > 10.0.0.10: ICMP echo reply, id 28200, seq 1, length 64
      
    3. ubuntu-router:eth1
      03:21:26.529233 IP 10.0.0.10 > 10.1.0.20: ICMP echo request, id 28200, seq 1, length 64
      03:21:26.534766 IP 10.1.0.20 > 10.0.0.10: ICMP echo reply, id 28200, seq 1, length 64
      03:21:26.535841 IP 10.1.0.254 > 10.1.0.10: ICMP redirect 10.1.0.110 to host 10.1.0.110, length 188
      
    4. ubuntu-inner-AA:eth0
      03:21:26.516873 IP 10.0.0.10 > 10.1.0.20: ICMP echo request, id 28200, seq 1, length 64
      03:21:26.523308 IP 10.1.0.254 > 10.1.0.10: ICMP redirect 10.1.0.110 to host 10.1.0.110, length 188
      
    5. ubuntu-inner-AA:eth1
      03:21:26.520911 IP 10.1.0.20 > 10.0.0.10: ICMP echo reply, id 28200, seq 1, length 64
      
  • 数分放置したところrequestもeth1に届くようになった => cacheのせい?
  • ubuntu-outer:~$ ping 10.1.0.20 -c 1
    1. ubuntu-inner-AA:eth1
      03:31:16.124510 IP 10.0.0.10 > 10.1.0.20: ICMP echo request, id 49704, seq 1, length 64
      03:31:16.125227 IP 10.1.0.20 > 10.0.0.10: ICMP echo reply, id 49704, seq 1, length 64
      
  • ubuntu-inner-AA上のrouting policyを削除したところ、前とは逆にeth1にrequest、eth0からreplyという状態になった。
    • ubuntu-inner-AA:~$ sudo ip rule del prefer 1
    • ubuntu-inner-AA:~$ sudo ip rule del prefer 2
    • ubuntu-outer:~$ ping 10.1.0.20 -c 1
      1. ubuntu-inner-AA:eth1
        03:35:13.581262 IP 10.0.0.10 > 10.1.0.20: ICMP echo request, id 58152, seq 1, length 64
        
      2. ubuntu-inner-AA:eth0
        03:35:13.581520 IP 10.1.0.20 > 10.0.0.10: ICMP echo reply, id 58152, seq 1, length 64
        
  • いよいよもって謎。
    • これも数分放置したところ、eth0のみを通る状態に戻ったので、やはりcacheのせいと思われる
  • ubuntu-outer:~$ ping 10.1.0.20 -c 1
    1. ubuntu-inner-AA:eth0
      03:46:28.309953 IP 10.0.0.10 > 10.1.0.20: ICMP echo request, id 16425, seq 1, length 64
      03:46:28.310817 IP 10.1.0.20 > 10.0.0.10: ICMP echo reply, id 16425, seq 1, length 64