wiki:TipAndDoc/Linux/LXC

Version 51 (modified by mitty, 4 years ago) (diff)

--

  • LinuX Container
  • ホスト側のvethデバイス名を指定する
    • lxc-chroot-linux-containers - Re: (Lxc-users) veth name - msg#00058 - Recent Discussion OSDir.com
    • man 5 lxc.conf

      By default lxc choose a name for the network device belonging to the outside of the container, this name is handled by lxc, but if you wish to handle this name yourself, you can tell lxc to set a specific name with the lxc.network.veth.pair option.

    • 複数のコンテナで同じ名前のpairを指定すると、二つ目は起動しない(下記の例ではtest1, test2コンテナでlxc.network.veth.pair = testと指定)
    • $ sudo lxc-start -n test2
      lxc-start: failed to create test-vethNdql6o : File exists
      lxc-start: failed to create netdev
      lxc-start: failed to create the network
      lxc-start: failed to spawn 'test2'
      lxc-start: No such file or directory - failed to remove cgroup '/sys/fs/cgroup/cpu/sysdefault/lxc/test2'
      
  • 指定しない場合、mktemp("vethXXXXXX")で決定される
    • SourceForge - lxc/lxc/blob - src/lxc/conf.c

      static int instanciate_veth(struct lxc_handler *handler, struct lxc_netdev *netdev)

    • ホスト側の物理NIC(lxc.network.link)とリンクするveth1の名前やifindexを外部から知る手段が無い模様
      • $ sudo lxc-start -d -n test -o test.log -l DEBUG
              lxc-start 1356615687.534 DEBUG    lxc_conf - instanciated veth 'vethtgqRWu/vethnwRgNf', index is '99'
        
      • lxc.network.veth.pair = vtest とすると以下の様になる
              lxc-start 1356615810.522 DEBUG    lxc_conf - instanciated veth 'vtest/vethA3QwQ9', index is '102'
        
  • mitty@precise:~$ lxc-create -t ubuntu -h
    usage: lxc-create -n <name> [-f configuration] [-t template] [-h] -- [template_options]
    usage: lxc-create -n <name> [-f configuration] [-t template] [-h] [fsopts] -- [template_options]
       fsopts: -B none
       fsopts: -B lvm [--lvname lvname] [--vgname vgname] [--fstype fstype] [--fssize fssize]
       fsopts: -B btrfs
               flag is not necessary, if possible btrfs support will be used
    
    creates a lxc system object.
    
    Options:
    name         : name of the container
    configuration: lxc configuration
    template     : lxc-template is an accessible template script
    
    The container backing store can be altered using '-B'.  By default it
    is 'none', which is a simple directory tree under /var/lib/lxc/<name>/rootfs
    Otherwise, the following option values may be relevant:
    lvname       : [for -lvm] name of lv in which to create lv,
                    container-name by default
    vgname       : [for -lvm] name of vg in which to create lv, 'lxc' by default
    fstype       : name of filesystem to create, ext4 by default
    fssize       : size of filesystem to create, 1G by default
    
    template-specific help follows: (these options follow '--')
    /usr/lib/lxc/templates/lxc-ubuntu -h|--help [-a|--arch] [-b|--bindhome <user>] [--trim] [-d|--debug]
       [-F | --flush-cache] [-r|--release <release>] [ -S | --auth-key <keyfile>]
    release: the ubuntu release (e.g. precise): defaults to host release on ubuntu, otherwise uses latest LTS
    trim: make a minimal (faster, but not upgrade-safe) container
    bindhome: bind <user>'s home into the container
              The ubuntu user will not be created, and <user> will have
              sudo access.
    arch: the container architecture (e.g. amd64): defaults to host arch
    auth-key: SSH Public key file to inject into container
    

Ubuntu 12.04

  • mitty@precise:~$ sudo aptitude install lxc
    The following NEW packages will be installed:
      bridge-utils{a} cgroup-lite{a} cloud-utils{a} debootstrap{a}
      dnsmasq-base{a} euca2ools{a} libapparmor1{a} libcap2-bin{a} libgmp10{a}
      libnetfilter-conntrack3{a} libpam-cap{a} libyaml-0-2{a} lxc
      python-boto{a} python-crypto{a} python-m2crypto{a} python-paramiko{a}
      python-yaml{a}
    0 packages upgraded, 18 newly installed, 0 to remove and 0 not upgraded.
    Need to get 2,873 kB of archives. After unpacking 16.1 MB will be used.
    

lxcbr0

  • mitty@precise:~$ ifconfig lxcbr0
    lxcbr0    Link encap:Ethernet  HWaddr 12:5e:23:12:4a:0f
              inet addr:10.0.3.1  Bcast:10.0.3.255  Mask:255.255.255.0
              inet6 addr: fe80::105e:23ff:fe12:4a0f/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 B)  TX bytes:554 (554.0 B)
    
    • /etc/default/lxc
      USE_LXC_BRIDGE="true"
      
      
      LXC_BRIDGE="lxcbr0"
      LXC_ADDR="10.0.3.1"
      LXC_NETMASK="255.255.255.0"
      LXC_NETWORK="10.0.3.0/24"
      LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
      LXC_DHCP_MAX="253"
      
    • /etc/lxc/lxc.conf
      lxc.network.type=veth
      lxc.network.link=lxcbr0
      lxc.network.flags=up
      
    • /etc/dnsmasq.d/lxc
      bind-interfaces
      except-interface=lxcbr0
      
  • /etc/init/lxc-net.conf も参考になる

USE_LXC_BRIDGE="false"

  • 手動でLXCコンテナのNAT設定を行う
  • eth0 -> br0にブリッジされていて、KVMで使用
    • LXCとKVMの共存のテスト

ip forwarding

  • mitty@precise:~$ cat /etc/sysctl.d/60-ip_forward.conf
    net.ipv4.ip_forward=1
    net.ipv6.conf.all.forwarding=1
    
  • mitty@precise:~$ cat /etc/network/interfaces
    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).
    
    # The loopback network interface
    auto lo
    iface lo inet loopback
    
    # The primary network interface
    auto eth0
    iface eth0 inet manual
    
    auto br0
    iface br0 inet dhcp
    	bridge_ports eth0 eth0
    	bridge_maxwait 0
    
    auto lxcbr0
    iface lxcbr0 inet static
    	bridge_ports none
    	bridge_maxwait 0
    	address 10.0.0.254
    	netmask 255.255.255.0
    	post-up iptables -A POSTROUTING -s 10.0.0.0/24 -t nat -j MASQUERADE
    	pre-down iptables -D POSTROUTING -s 10.0.0.0/24 -t nat -j MASQUERADE
    

change apt repository mirror and disable auto start lxcbr0

  • デフォルトのミラーが遅いので、ftp.tsukubaに変更
  • lxcの起動スクリプトによるlxcbr0の作成を抑制
  • mitty@precise:~$ cat /etc/default/lxc
    MIRROR="http://ftp.tsukuba.wide.ad.jp/Linux/ubuntu"
    
    USE_LXC_BRIDGE="false"
    

host settings

  • mitty@precise:~$ ifconfig -a
    br0       Link encap:Ethernet  HWaddr 52:54:00:bc:53:bc
              inet addr:192.168.10.172  Bcast:192.168.10.255  Mask:255.255.255.0
              inet6 addr: fe80::5054:ff:febc:53bc/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:1340 errors:0 dropped:0 overruns:0 frame:0
              TX packets:948 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:116870 (116.8 KB)  TX bytes:111171 (111.1 KB)
    
    eth0      Link encap:Ethernet  HWaddr 52:54:00:bc:53:bc
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:1400 errors:0 dropped:0 overruns:0 frame:0
              TX packets:945 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:150975 (150.9 KB)  TX bytes:110725 (110.7 KB)
    
    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:16436  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
    
    lxcbr0    Link encap:Ethernet  HWaddr 4a:5a:12:a4:0a:ac
              inet addr:10.0.0.254  Bcast:10.0.0.255  Mask:255.255.255.0
              inet6 addr: fe80::485a:12ff:fea4:aac/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 B)  TX bytes:408 (408.0 B)
    
  • mitty@precise:~$ ip route
    default via 192.168.10.254 dev br0  metric 100
    10.0.0.0/24 dev lxcbr0  proto kernel  scope link  src 10.0.0.254
    192.168.10.0/24 dev br0  proto kernel  scope link  src 192.168.10.172
    
  • mitty@precise:~$ brctl show
    bridge name     bridge id               STP enabled     interfaces
    br0             8000.525400bc53bc       no              eth0
    lxcbr0          8000.000000000000       no
    

iptables on host

  • mitty@precise:~$ sudo iptables -L -t nat -vx
    Chain PREROUTING (policy ACCEPT 30 packets, 10827 bytes)
        pkts      bytes target     prot opt in     out     source               destination
    
    Chain INPUT (policy ACCEPT 4 packets, 323 bytes)
        pkts      bytes target     prot opt in     out     source               destination
    
    Chain OUTPUT (policy ACCEPT 83 packets, 5999 bytes)
        pkts      bytes target     prot opt in     out     source               destination
    
    Chain POSTROUTING (policy ACCEPT 83 packets, 5999 bytes)
        pkts      bytes target     prot opt in     out     source               destination
           0        0 MASQUERADE  all  --  any    any     10.0.0.0/24          anywhere
    
  • MASQUERADEが正しく設定されている

make LXC container with ubuntu template

  • mitty@precise:~$ sudo lxc-create -t ubuntu -n lxc-test
    No config file specified, using the default config
    debootstrap is /usr/sbin/debootstrap
    Checking cache download in /var/cache/lxc/precise/rootfs-amd64 ...
    installing packages: vim,ssh
    Downloading ubuntu precise minimal ...
    I: Retrieving Release
    I: Retrieving Release.gpg
    I: Checking Release signature
    
    ....
    
    I: Checking component main on http://ftp.tsukuba.wide.ad.jp/Linux/ubuntu...
    
    ....
    
    Processing triggers for initramfs-tools ...
    Download complete
    Copy /var/cache/lxc/precise/rootfs-amd64 to /var/lib/lxc/lxc-test/rootfs ...
    Copying rootfs to /var/lib/lxc/lxc-test/rootfs ...
    
    ##
    # The default user is 'ubuntu' with password 'ubuntu'!
    # Use the 'sudo' command to run tasks as root in the container.
    ##
    
    'ubuntu' template installed
    'lxc-test' created
    

set container IP with LXC/config

  • LXCコンテナのconfigファイルからIPアドレスを指定する
  • 結論としては、デフォルトゲートウェイなどを設定できないので、不便
  • mitty@precise:~$ sudo vim /var/lib/lxc/lxc-test/config
    lxc.network.ipv4 = 10.0.0.10/24
    
  • mitty@precise:~$ sudo lxc-start -n lxc-test -d
  • mitty@precise:~$ ping 10.0.0.10 -c 1
    PING 10.0.0.10 (10.0.0.10) 56(84) bytes of data.
    64 bytes from 10.0.0.10: icmp_req=1 ttl=64 time=0.060 ms
    
    --- 10.0.0.10 ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 0.060/0.060/0.060/0.000 ms
    
  • mitty@precise:~$ ssh 10.0.0.10 -l ubuntu
    • ubuntu@lxc-test:~$ ifconfig
      eth0      Link encap:Ethernet  HWaddr 00:16:3e:ba:3e:ef
                inet addr:10.0.0.10  Bcast:10.0.0.255  Mask:255.255.255.0
      
    • ubuntu@lxc-test:~$ ip route
      10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.10
      
    • ubuntu@lxc-test:~$ ping 8.8.8.8
      connect: Network is unreachable
      

set container IP with interfaces

  • ホストOS上から、あらかじめコンテナのinterfacesファイルを設定して、起動する
  • 間違いが無く、かつ楽そう
  • mitty@precise:~$ sudo vim /var/lib/lxc/lxc-test/rootfs/etc/network/interfaces
    auto eth0
    iface eth0 inet static
    	address 10.0.0.1
    	netmask 255.255.255.0
    	gateway 10.0.0.254
    
  • mitty@precise:~$ ssh 10.0.0.1 -l ubuntu
    • ubuntu@lxc-test:~$ ifconfig
      eth0      Link encap:Ethernet  HWaddr 00:16:3e:ba:3e:ef
                inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
      
    • ubuntu@lxc-test:~$ ip route
      default via 10.0.0.254 dev eth0  metric 100
      10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.1
      
    • ubuntu@lxc-test:~$ ping 8.8.8.8 -c 1
      PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
      64 bytes from 8.8.8.8: icmp_req=1 ttl=52 time=7.66 ms
      
      --- 8.8.8.8 ping statistics ---
      1 packets transmitted, 1 received, 0% packet loss, time 0ms
      rtt min/avg/max/mdev = 7.667/7.667/7.667/0.000 ms
      

two NICs in one container

example

  • 通常のbridge(veth)ではなくmacvlanであるが、vethでも同じはず
  • /var/lib/lxc/test/config
    lxc.network.type=macvlan
    lxc.network.macvlan.mode=bridge
    lxc.network.link=em1
    lxc.network.flags=up
    lxc.network.hwaddr = 00:16:3e:85:2e:da
    lxc.network.type=macvlan
    lxc.network.macvlan.mode=bridge
    lxc.network.link=em1
    lxc.network.flags=up
    lxc.network.hwaddr = 00:16:3e:85:2e:db
    
  • mitty@test:~$ ifconfig -a | egrep 'addr|Link'
    eth0      Link encap:Ethernet  HWaddr 00:16:3e:85:2e:da
              inet addr:192.168.83.207  Bcast:192.168.83.255  Mask:255.255.255.0
              inet6 addr: fe80::216:3eff:fe85:2eda/64 Scope:Link
    eth1      Link encap:Ethernet  HWaddr 00:16:3e:85:2e:db
              inet addr:192.168.83.212  Bcast:192.168.83.255  Mask:255.255.255.0
              inet6 addr: fe80::216:3eff:fe85:2edb/64 Scope:Link
    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
    
  • mitty@test:~$ ip route
    default via 192.168.83.243 dev eth0
    192.168.83.0/24 dev eth0  proto kernel  scope link  src 192.168.83.207
    192.168.83.0/24 dev eth1  proto kernel  scope link  src 192.168.83.212
    

mount bind

  • /var/lib/lxc/>container name>/fstab を用いて、ホストの特定ディレクトリ以下をゲストにbind出来る
  • fstab
    /media mnt none bind 0 0
    
    • host:/media -> guest:/mnt とマウントされる
    • mntに「/」が無いことに注意
    • mitty@lxc:~$ mount
      /dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX on /mnt type ext4 (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
      
    • mitty@lxc:~$ ls -l /mnt/
      total 0
      -rw-r--r-- 1 root root 0 Oct 23 22:01 host-media
      
  1. mitty@lxc:~$ sudo rm /mnt/host-media
    removed `/mnt/host-media'
    
  2. mitty@host:~$ ls -l /media/host-media
    ls: cannot access /media/host-media: No such file or directory
    

read-only bind

  • fstab
    /media mnt none bind,ro 0 0
    
  • mountコマンドの出力結果は「(rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)」で変わらないが、read-only化される
  • mitty@lxc:~$ sudo rm /mnt/host-media
    rm: cannot remove `/mnt/host-media': Read-only file system
    

X11 with VNC

  • 12.04上のLXCでは、configのcgroupの設定やmknodなどは特に必要ない模様
  • vnc自体については => network/vnc
  • 基本的にaptitude install -Rでインストールしている
  • エラーメッセージについては ~/.vnc/ 以下に保存されるログファイルから

common packages

  • vnc4server
  • xfonts-base
    could not open default font 'fixed'
    

GNOME

  • ~/.vnc/xstartup
    xsetroot -solid grey
    exec gnome-session &
    

Xfce

  • xfce4
  • ~/.vnc/xstartup
    xsetroot -solid grey
    exec startxfce4 &
    
  • もしくは
    xsetroot -solid grey
    exec xfce4-session &
    
  • 後者だとTrashディレクトリなどがFile Managerに表示されない
    • ファイルアイコンが表示されない

LXDE

  • lxde
  • ~/.vnc/xstartup
    xsetroot -solid grey
    exec lxsession -s LXDE &
    
    • -s LXDEは無くても良い模様

munin plugin

lxc_proc

  • https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt

    Each cgroup is represented by a directory in the cgroup file system containing the following files describing that cgroup:

    • tasks: list of tasks (by pid) attached to that cgroup. This list is not guaranteed to be sorted. Writing a thread id into this file moves the thread into this cgroup.
    • cgroup.procs: list of tgids in the cgroup. This list is not guaranteed to be sorted or free of duplicate tgids, and userspace should sort/uniquify the list if this property is required. Writing a thread group id into this file moves all threads in that group into this cgroup.
    • notify_on_release flag: run the release agent on exit?
    • release_agent: the path to use for release notifications (this file exists in the top cgroup only)
  • まとめ
    • lxc-cgroupコマンドを使わない場合、以下のsysfsにアクセスすると良い模様
    • Debian 6.0: /sys/fs/cgroup/<container>/tasks
    • Ubuntu 12.04 with fstab: /sys/fs/cgroup/lxc/<container>/tasks
    • Ubuntu 12.04 with cgroup-lite: /sys/fs/cgroup/cpuacct/lxc/<container>/tasks
    • Ubuntu 12.04 with cgroup-bin: /sys/fs/cgroup/cpuacct/sysdefault/lxc/<container>/tasks

Debian 6.0

  • http://wiki.debian.org/LXC

    Add this line to /etc/fstab

    cgroup  /sys/fs/cgroup  cgroup  defaults  0   0
    

    Try to mount it (a reboot solves an eventual "resource busy problem" in any case)

    mount /sys/fs/cgroup
    
  • $ sudo lxc-start -d -n lxc-test
  • $ sudo lxc-cgroup -n lxc-test tasks
    1432
    1569
    1571
    1598
    
  • $ ls -1F /sys/fs/cgroup/
    cgroup.procs
    cpuacct.stat
    cpuacct.usage
    cpuacct.usage_percpu
    cpuset.cpu_exclusive
    cpuset.cpus
    cpuset.mem_exclusive
    cpuset.mem_hardwall
    cpuset.memory_migrate
    cpuset.memory_pressure
    cpuset.memory_pressure_enabled
    cpuset.memory_spread_page
    cpuset.memory_spread_slab
    cpuset.mems
    cpuset.sched_load_balance
    cpuset.sched_relax_domain_level
    cpu.shares
    devices.allow
    devices.deny
    devices.list
    lxc-test/
    net_cls.classid
    notify_on_release
    release_agent
    tasks
    
    • /sys/fs/cgroup/lxc-test/ も同じ構造
  • $ cat /sys/fs/cgroup/lxc-test/tasks
    1432
    1569
    1571
    1598
    

Ubuntu 12.04

  • Ubuntu 12.04 with aptitude install -R lxc
    • cgroup関係のパッケージは-Rをつけるとインストールされない
  • $ ls -1F /sys/fs/cgroup/
    • N/A
  • $ sudo lxc-start -n lxc-test
    lxc-start: failed to spawn 'lxc-test'
    

/etc/fstab

  • Ubuntu 12.04 with /etc/fstab:cgroup /sys/fs/cgroup cgroup defaults 0 0
    • 起動時に自動でマウントされないため、mount -aする必要がある
  • $ sudo mount -a
  • $ sudo lxc-start -d -n lxc-test
  • $ sudo lxc-cgroup -n lxc-test tasks
    lxc-cgroup: cgroup is not mounted
    lxc-cgroup: failed to retrieve value of 'tasks' for 'lxc-test'
    
  • $ ls -1F /sys/fs/cgroup/
    blkio.io_merged
    blkio.io_queued
    blkio.io_service_bytes
    blkio.io_serviced
    blkio.io_service_time
    blkio.io_wait_time
    blkio.reset_stats
    blkio.sectors
    blkio.throttle.io_service_bytes
    blkio.throttle.io_serviced
    blkio.throttle.read_bps_device
    blkio.throttle.read_iops_device
    blkio.throttle.write_bps_device
    blkio.throttle.write_iops_device
    mitty@lein:~$ ls -1F /sys/fs/cgroup/
    blkio.io_merged
    blkio.io_queued
    blkio.io_service_bytes
    blkio.io_serviced
    blkio.io_service_time
    blkio.io_wait_time
    blkio.reset_stats
    blkio.sectors
    blkio.throttle.io_service_bytes
    blkio.throttle.io_serviced
    blkio.throttle.read_bps_device
    blkio.throttle.read_iops_device
    blkio.throttle.write_bps_device
    blkio.throttle.write_iops_device
    blkio.time
    blkio.weight
    blkio.weight_device
    cgroup.clone_children
    cgroup.event_control
    cgroup.procs
    cpuacct.stat
    cpuacct.usage
    cpuacct.usage_percpu
    cpu.cfs_period_us
    cpu.cfs_quota_us
    cpu.rt_period_us
    cpu.rt_runtime_us
    cpuset.cpu_exclusive
    cpuset.cpus
    cpuset.mem_exclusive
    cpuset.mem_hardwall
    cpuset.memory_migrate
    cpuset.memory_pressure
    cpuset.memory_pressure_enabled
    cpuset.memory_spread_page
    cpuset.memory_spread_slab
    cpuset.mems
    cpuset.sched_load_balance
    cpuset.sched_relax_domain_level
    cpu.shares
    cpu.stat
    devices.allow
    devices.deny
    devices.list
    lxc/
    memory.failcnt
    memory.force_empty
    memory.limit_in_bytes
    memory.max_usage_in_bytes
    memory.memsw.failcnt
    memory.memsw.limit_in_bytes
    memory.memsw.max_usage_in_bytes
    memory.memsw.usage_in_bytes
    memory.move_charge_at_immigrate
    memory.numa_stat
    memory.oom_control
    memory.soft_limit_in_bytes
    memory.stat
    memory.swappiness
    memory.usage_in_bytes
    memory.use_hierarchy
    notify_on_release
    release_agent
    tasks
    
    • /sys/fs/cgroup/lxc/, /sys/fs/cgroup/lxc/lxc-test/ も同じ構造
  • $ cat /sys/fs/cgroup/lxc/lxc-test/tasks
    2712
    2867
    2932
    2943
    2947
    2952
    2953
    2954
    2983
    3002
    3031
    3036
    3037
    3042
    3056
    3058
    

cgroup-lite

  • Ubuntu 12.04 with cgroup-lite
    • aptitude install時に-Rとしない場合、cgroup-liteがRecommendsからインストールされる
    • /bin/cgroups-mount
      mount -t tmpfs -o uid=0,gid=0,mode=0755 cgroup /sys/fs/cgroup
      
  • $ sudo lxc-cgroup -n lxc-test tasks
    lxc-cgroup: cgroup is not mounted
    lxc-cgroup: failed to retrieve value of 'tasks' for 'lxc-test'
    
  • $ ls -1F /sys/fs/cgroup/
    blkio/
    cpu/
    cpuacct/
    cpuset/
    devices/
    freezer/
    memory/
    perf_event/
    
    • $ find /sys/fs/cgroup/ | grep lxc-test/tasks | xargs wc
      17 17 102 /sys/fs/cgroup/perf_event/lxc/lxc-test/tasks
      17 17 102 /sys/fs/cgroup/blkio/lxc/lxc-test/tasks
      17 17 102 /sys/fs/cgroup/freezer/lxc/lxc-test/tasks
      17 17 102 /sys/fs/cgroup/devices/lxc/lxc-test/tasks
      17 17 102 /sys/fs/cgroup/memory/lxc/lxc-test/tasks
      17 17 102 /sys/fs/cgroup/cpuacct/lxc/lxc-test/tasks
      17 17 102 /sys/fs/cgroup/cpu/lxc/lxc-test/tasks
      17 17 102 /sys/fs/cgroup/cpuset/lxc/lxc-test/tasks
      
  • $ cat /sys/fs/cgroup/perf_event/lxc/lxc-test/tasks
    17390
    17539
    17556
    17560
    17562
    17567
    17568
    17570
    17599
    17621
    17647
    17651
    17654
    17659
    17678
    17681
    17829
    

cgroup-bin

  • Ubuntu 12.04 with cgroup-bin
    • cgroup-binとcgroup-liteはコンフリクトする
    • /etc/init/cgconfig.conf
      mount -t tmpfs -o uid=0,gid=0,mode=0755 cgroups /sys/fs/cgroup
      
  • $ sudo lxc-cgroup -n lxc-test tasks
    lxc-cgroup: cgroup is not mounted
    lxc-cgroup: failed to retrieve value of 'tasks' for 'lxc-test'
    
  • $ ls -1F /sys/fs/cgroup/
    cpu/
    cpuacct/
    devices/
    freezer/
    memory/
    
    • $ find /sys/fs/cgroup/ | grep lxc-test/tasks | xargs wc
      16 16 80 /sys/fs/cgroup/freezer/sysdefault/lxc/lxc-test/tasks
      16 16 80 /sys/fs/cgroup/memory/sysdefault/lxc/lxc-test/tasks
      16 16 80 /sys/fs/cgroup/devices/sysdefault/lxc/lxc-test/tasks
      16 16 80 /sys/fs/cgroup/cpuacct/sysdefault/lxc/lxc-test/tasks
      16 16 80 /sys/fs/cgroup/cpu/sysdefault/lxc/lxc-test/tasks
      
  • $ cat /sys/fs/cgroup/freezer/sysdefault/lxc/lxc-test/tasks
    1296
    1390
    1434
    1461
    1470
    1471
    1473
    1474
    1502
    1524
    1552
    1562
    1563
    1566
    1580
    1582
    

mount inside container

  • コンテナの中ではAppArmorによってmountが禁止されているため、直接NFSマウントするにはAppArmorの再設定が必要

ホストで直接mountし、コンテナには--bindでアクセスさせる

  1. host# mount nfsserver:/path/to/export /mnt/somewhere
  2. /var/lib/lxc/CONTAINER/fstab
    /mnt/somewhere   path/to/mountdir    none bind 0 0
    
  3. host# lxc-start -d -n CONTAINER
  4. lxc$ mount
    nfsserver:/path/to/export on /path/to/mountdir type nfs4 (rw,relatime,...)
    
  • ホスト上でread-onlyマウントしていても、/var/lib/lxc/CONTAINER/fstabでのread/write権限設定によって上書きされるので注意
  • cifsも同じやりかたでマウントできる
    • cifsではホスト上でread-onlyマウントしていると、設定は一部(?)引き継がれる模様
    • touch hogeすると、touch: cannot touch `/path/to/mountdir/hoge': Read-only file systemとエラーになるが、実際にはファイルは生成される
    • 0バイトのファイルは生成されるが、データの書き込みは出来ない模様
    • 削除は出来ないため、削除できないファイルが作成されてしまう点に注意
  • ホスト上のマウントディレクトリを直接コンテナのディレクトリにbindする必要がある。つまり、以下のようなことは出来ない
    1. host# mount nfsserver:/path/to/export/dir1 /mnt/somewhere/dir1
    2. lxc/CONTAINER/fstab
      /mnt/somewhere   path/to/mountdir    none bind 0 0
      
    3. lxc$ ls -l /path/to/mountdir/dir1
      • nfsserver:/path/to/export/dir1 が見えることが期待されるが、実際にはマウントされていない

backup

DHCP

no valid interfaces found

  • Arch Linux 4.4.5-1
  • lxc 1:1.1.5-4
  • dhcpcd 6.10.1-1
  • # dhcpcd
    dev: loaded udev
    no valid interfaces found
    no interfaces have a carrier
    forked to background, child pid 65
    
  • # dhcpcd eth0
    DUID 00:01:00:01:1e:8a:6a:5e:ba:4a:37:94:4a:9d
    eth0: IAID 16:b6:47:44
    eth0: rebinding lease of 192.168.0.185
    eth0: NAK: address in use from 192.168.0.1
    eth0: message: address in use
    eth0: soliciting a DHCP lease
    eth0: soliciting an IPv6 router
    eth0: offered 192.168.0.172 from 192.168.0.1
    eth0: probing address 192.168.0.172/24
    eth0: leased 192.168.0.172 for 86400 seconds
    eth0: adding route to 192.168.0.0/24
    eth0: adding default route via 192.168.0.1
    forked to background, child pid 123
    
  • LXCコンテナ内で、udevdが起動しないことが原因の模様
  • strace -f dhcpcd
    • コンテナホスト(検証はKVMゲストで行った)
      access("/run/udev/control", F_OK) = 0
      
      open("/sys/devices/virtual/net/lo/uevent", O_RDONLY|O_CLOEXEC) = 9
      fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
      fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
      read(9, "INTERFACE=lo\nIFINDEX=1\n", 4096) = 23
      read(9, "", 4096)                 = 0
      read(9, "", 4096)                 = 0
      close(9)                          = 0
      open("/run/udev/data/n1", O_RDONLY|O_CLOEXEC) = 9
      
      open("/sys/devices/pci0000:00/0000:00:03.0/virtio0/net/ens3/uevent", O_RDONLY|O_CLOEXEC) = 9
      fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
      fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
      read(9, "INTERFACE=ens3\nIFINDEX=2\n", 4096) = 25
      read(9, "", 4096)                 = 0
      read(9, "", 4096)                 = 0
      close(9)                          = 0
      open("/run/udev/data/n2", O_RDONLY|O_CLOEXEC) = 9
      
    • LXCコンテナ
      access("/run/udev/control", F_OK) = -1 ENOENT (No such file or directory)
      
      open("/sys/devices/virtual/net/lo/uevent", O_RDONLY|O_CLOEXEC) = 9
      fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
      fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
      read(9, "INTERFACE=lo\nIFINDEX=1\n", 4096) = 23
      read(9, "", 4096)                 = 0
      read(9, "", 4096)                 = 0
      close(9)                          = 0
      open("/run/udev/data/n1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
      
      open("/sys/devices/virtual/net/eth0/uevent", O_RDONLY|O_CLOEXEC) = 9
      fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
      fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
      read(9, "INTERFACE=eth0\nIFINDEX=6\n", 4096) = 25
      read(9, "", 4096)                 = 0
      read(9, "", 4096)                 = 0
      close(9)                          = 0
      open("/run/udev/data/n6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
      write(2, "no valid interfaces found", 25) = 25
      write(2, "\n", 1)                 = 1
      
      write(1, "no interfaces have a carrier\n", 29) = 29
      
      • [root@test /]# systemctl status systemd-udevd
        ● systemd-udevd.service - udev Kernel Device Manager
           Loaded: loaded (/usr/lib/systemd/system/systemd-udevd.service; static; vendor preset: disabled)
           Active: inactive (dead)
        Condition: start condition failed at Sat 2016-04-02 08:30:59 UTC; 5h 9min ago
                   ConditionPathIsReadWrite=/sys was not met
             Docs: man:systemd-udevd.service(8)
                   man:udev(7)
        
  • arch:Linux_Containers#Systemd_considerations_.28required.29
    • 上記を参考にsystemdの設定を行ったが効果なし
  • arch:dhcpcd#dhcpcd_and_systemd_network_interfaces

    dhcpcd.service can be Enabled without specifying an interface. This may, however, create a race condition at boot with systemd-udevd trying to apply a predictable network interface name:

    error changing net interface name wlan0 to wlp4s0: Device or resource busy" 
    

    To avoid it, enable dhcpcd per interface it should bind to as described in dhcpcd#Running. The downside of the template unit is, however, that it does not support hot-plugging of a wired connection and will fail if the network cable is not connected. To work-around the failure, see dhcpcd#Timeout_delay.

    とのことなので、いずれにせよdhcpcd.serviceを使用しないのがよさそう

Attachments (4)

Download all attachments as: .zip