[[PageOutline]] * LinuX Container * [http://gihyo.jp/admin/serial/01/ubuntu-recipe/0226 Ubuntu Weekly Recipe：第226回　LXCで軽量仮想環境の活用｜gihyo.jp … 技術評論社] * [http://www.stgraber.org/2012/05/04/lxc-in-ubuntu-12-04-lts/ LXC in Ubuntu 12.04 LTS | Stéphane Graber's website] * [http://www.slideshare.net/mkouhei/lxc-cf201207presen LXC 再入門] * [http://www.slideshare.net/masahide_yamamoto/osc2011-nagoya LXC入門 - Osc2011 nagoya] * lxcコンテナから(外側の)ホストOSを操作できる点について具体的に解説がある * https://wiki.ubuntu.com/LxcSecurity Ubuntu 12.04での情報 * github:vajtsz/munin-plugins > Plugin to monitor LXC * mitty@precise:~$ lxc-create -t ubuntu -h {{{ usage: lxc-create -n [-f configuration] [-t template] [-h] -- [template_options] usage: lxc-create -n [-f configuration] [-t template] [-h] [fsopts] -- [template_options] fsopts: -B none fsopts: -B lvm [--lvname lvname] [--vgname vgname] [--fstype fstype] [--fssize fssize] fsopts: -B btrfs flag is not necessary, if possible btrfs support will be used creates a lxc system object. Options: name : name of the container configuration: lxc configuration template : lxc-template is an accessible template script The container backing store can be altered using '-B'. By default it is 'none', which is a simple directory tree under /var/lib/lxc//rootfs Otherwise, the following option values may be relevant: lvname : [for -lvm] name of lv in which to create lv, container-name by default vgname : [for -lvm] name of vg in which to create lv, 'lxc' by default fstype : name of filesystem to create, ext4 by default fssize : size of filesystem to create, 1G by default template-specific help follows: (these options follow '--') /usr/lib/lxc/templates/lxc-ubuntu -h|--help [-a|--arch] [-b|--bindhome ] [--trim] [-d|--debug] [-F | --flush-cache] [-r|--release ] [ -S | --auth-key ] release: the ubuntu release (e.g. precise): defaults to host release on ubuntu, otherwise uses latest LTS trim: make a minimal (faster, but not upgrade-safe) container bindhome: bind 's home into the container The ubuntu user will not be created, and will have sudo access. arch: the container architecture (e.g. amd64): defaults to host arch auth-key: SSH Public key file to inject into container }}} = Ubuntu 12.04 = * mitty@precise:~$ sudo aptitude install lxc {{{ The following NEW packages will be installed: bridge-utils{a} cgroup-lite{a} cloud-utils{a} debootstrap{a} dnsmasq-base{a} euca2ools{a} libapparmor1{a} libcap2-bin{a} libgmp10{a} libnetfilter-conntrack3{a} libpam-cap{a} libyaml-0-2{a} lxc python-boto{a} python-crypto{a} python-m2crypto{a} python-paramiko{a} python-yaml{a} 0 packages upgraded, 18 newly installed, 0 to remove and 0 not upgraded. Need to get 2,873 kB of archives. After unpacking 16.1 MB will be used. }}} == lxcbr0 == * mitty@precise:~$ ifconfig lxcbr0 {{{ lxcbr0 Link encap:Ethernet HWaddr 12:5e:23:12:4a:0f inet addr:10.0.3.1 Bcast:10.0.3.255 Mask:255.255.255.0 inet6 addr: fe80::105e:23ff:fe12:4a0f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:7 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:554 (554.0 B) }}} * /etc/default/lxc {{{ USE_LXC_BRIDGE="true" LXC_BRIDGE="lxcbr0" LXC_ADDR="10.0.3.1" LXC_NETMASK="255.255.255.0" LXC_NETWORK="10.0.3.0/24" LXC_DHCP_RANGE="10.0.3.2,10.0.3.254" LXC_DHCP_MAX="253" }}} * /etc/lxc/lxc.conf {{{ lxc.network.type=veth lxc.network.link=lxcbr0 lxc.network.flags=up }}} * /etc/dnsmasq.d/lxc {{{ bind-interfaces except-interface=lxcbr0 }}} * /etc/init/lxc-net.conf も参考になる = USE_LXC_BRIDGE="false" = * 手動でLXCコンテナのNAT設定を行う * eth0 -> br0にブリッジされていて、KVMで使用 * LXCとKVMの共存のテスト == ip forwarding == * mitty@precise:~$ cat /etc/sysctl.d/60-ip_forward.conf {{{ net.ipv4.ip_forward=1 net.ipv6.conf.all.forwarding=1 }}} * mitty@precise:~$ cat /etc/network/interfaces {{{ # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet manual auto br0 iface br0 inet dhcp bridge_ports eth0 eth0 bridge_maxwait 0 auto lxcbr0 iface lxcbr0 inet static bridge_ports none bridge_maxwait 0 address 10.0.0.254 netmask 255.255.255.0 post-up iptables -A POSTROUTING -s 10.0.0.0/24 -t nat -j MASQUERADE pre-down iptables -D POSTROUTING -s 10.0.0.0/24 -t nat -j MASQUERADE }}} == change apt repository mirror and disable auto start lxcbr0 == * デフォルトのミラーが遅いので、ftp.tsukubaに変更 * lxcの起動スクリプトによるlxcbr0の作成を抑制 * mitty@precise:~$ cat /etc/default/lxc {{{ MIRROR="http://ftp.tsukuba.wide.ad.jp/Linux/ubuntu" USE_LXC_BRIDGE="false" }}} == host settings == * mitty@precise:~$ ifconfig -a {{{ br0 Link encap:Ethernet HWaddr 52:54:00:bc:53:bc inet addr:192.168.10.172 Bcast:192.168.10.255 Mask:255.255.255.0 inet6 addr: fe80::5054:ff:febc:53bc/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1340 errors:0 dropped:0 overruns:0 frame:0 TX packets:948 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:116870 (116.8 KB) TX bytes:111171 (111.1 KB) eth0 Link encap:Ethernet HWaddr 52:54:00:bc:53:bc UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1400 errors:0 dropped:0 overruns:0 frame:0 TX packets:945 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:150975 (150.9 KB) TX bytes:110725 (110.7 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) lxcbr0 Link encap:Ethernet HWaddr 4a:5a:12:a4:0a:ac inet addr:10.0.0.254 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::485a:12ff:fea4:aac/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:408 (408.0 B) }}} * mitty@precise:~$ ip route {{{ default via 192.168.10.254 dev br0 metric 100 10.0.0.0/24 dev lxcbr0 proto kernel scope link src 10.0.0.254 192.168.10.0/24 dev br0 proto kernel scope link src 192.168.10.172 }}} * mitty@precise:~$ brctl show {{{ bridge name bridge id STP enabled interfaces br0 8000.525400bc53bc no eth0 lxcbr0 8000.000000000000 no }}} === iptables on host === * mitty@precise:~$ sudo iptables -L -t nat -vx {{{ Chain PREROUTING (policy ACCEPT 30 packets, 10827 bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 4 packets, 323 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 83 packets, 5999 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 83 packets, 5999 bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- any any 10.0.0.0/24 anywhere }}} * MASQUERADEが正しく設定されている == make LXC container with ubuntu template == * mitty@precise:~$ sudo lxc-create -t ubuntu -n lxc-test {{{ No config file specified, using the default config debootstrap is /usr/sbin/debootstrap Checking cache download in /var/cache/lxc/precise/rootfs-amd64 ... installing packages: vim,ssh Downloading ubuntu precise minimal ... I: Retrieving Release I: Retrieving Release.gpg I: Checking Release signature .... I: Checking component main on http://ftp.tsukuba.wide.ad.jp/Linux/ubuntu... .... Processing triggers for initramfs-tools ... Download complete Copy /var/cache/lxc/precise/rootfs-amd64 to /var/lib/lxc/lxc-test/rootfs ... Copying rootfs to /var/lib/lxc/lxc-test/rootfs ... ## # The default user is 'ubuntu' with password 'ubuntu'! # Use the 'sudo' command to run tasks as root in the container. ## 'ubuntu' template installed 'lxc-test' created }}} === set container IP with LXC/config === * LXCコンテナのconfigファイルからIPアドレスを指定する * 結論としては、デフォルトゲートウェイなどを設定できないので、不便 * mitty@precise:~$ sudo vim /var/lib/lxc/lxc-test/config {{{ lxc.network.ipv4 = 10.0.0.10/24 }}} * mitty@precise:~$ sudo lxc-start -n lxc-test -d * mitty@precise:~$ ping 10.0.0.10 -c 1 {{{ PING 10.0.0.10 (10.0.0.10) 56(84) bytes of data. 64 bytes from 10.0.0.10: icmp_req=1 ttl=64 time=0.060 ms --- 10.0.0.10 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.060/0.060/0.060/0.000 ms }}} * mitty@precise:~$ ssh 10.0.0.10 -l ubuntu * ubuntu@lxc-test:~$ ifconfig {{{ eth0 Link encap:Ethernet HWaddr 00:16:3e:ba:3e:ef inet addr:10.0.0.10 Bcast:10.0.0.255 Mask:255.255.255.0 }}} * ubuntu@lxc-test:~$ ip route {{{ 10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.10 }}} * ubuntu@lxc-test:~$ ping 8.8.8.8 {{{ connect: Network is unreachable }}} === set container IP with interfaces === * ホストOS上から、あらかじめコンテナのinterfacesファイルを設定して、起動する * 間違いが無く、かつ楽そう * mitty@precise:~$ sudo vim /var/lib/lxc/lxc-test/rootfs/etc/network/interfaces {{{ auto eth0 iface eth0 inet static address 10.0.0.1 netmask 255.255.255.0 gateway 10.0.0.254 }}} * mitty@precise:~$ ssh 10.0.0.1 -l ubuntu * ubuntu@lxc-test:~$ ifconfig {{{ eth0 Link encap:Ethernet HWaddr 00:16:3e:ba:3e:ef inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0 }}} * ubuntu@lxc-test:~$ ip route {{{ default via 10.0.0.254 dev eth0 metric 100 10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.1 }}} * ubuntu@lxc-test:~$ ping 8.8.8.8 -c 1 {{{ PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_req=1 ttl=52 time=7.66 ms --- 8.8.8.8 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 7.667/7.667/7.667/0.000 ms }}} = two NICs in one container = * [http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg02037.html Re: (Lxc-users) two NICs in container] == example == * 通常のbridge(veth)ではなくmacvlanであるが、vethでも同じはず * /var/lib/lxc/test/config {{{ lxc.network.type=macvlan lxc.network.macvlan.mode=bridge lxc.network.link=em1 lxc.network.flags=up lxc.network.hwaddr = 00:16:3e:85:2e:da lxc.network.type=macvlan lxc.network.macvlan.mode=bridge lxc.network.link=em1 lxc.network.flags=up lxc.network.hwaddr = 00:16:3e:85:2e:db }}} * mitty@test:~$ ifconfig -a | egrep 'addr|Link' {{{ eth0 Link encap:Ethernet HWaddr 00:16:3e:85:2e:da inet addr:192.168.83.207 Bcast:192.168.83.255 Mask:255.255.255.0 inet6 addr: fe80::216:3eff:fe85:2eda/64 Scope:Link eth1 Link encap:Ethernet HWaddr 00:16:3e:85:2e:db inet addr:192.168.83.212 Bcast:192.168.83.255 Mask:255.255.255.0 inet6 addr: fe80::216:3eff:fe85:2edb/64 Scope:Link lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host }}} * mitty@test:~$ ip route {{{ default via 192.168.83.243 dev eth0 192.168.83.0/24 dev eth0 proto kernel scope link src 192.168.83.207 192.168.83.0/24 dev eth1 proto kernel scope link src 192.168.83.212 }}} = mount bind = * /var/lib/lxc/>container name>/fstab を用いて、ホストの特定ディレクトリ以下をゲストにbind出来る * fstab {{{ /media mnt none bind 0 0 }}} * host:/media -> guest:/mnt とマウントされる * mntに「/」が'''無い'''ことに注意 * mitty@lxc:~$ mount {{{ /dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX on /mnt type ext4 (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered) }}} * mitty@lxc:~$ ls -l /mnt/ {{{ total 0 -rw-r--r-- 1 root root 0 Oct 23 22:01 host-media }}} 1. mitty@lxc:~$ sudo rm /mnt/host-media {{{ removed `/mnt/host-media' }}} 1. mitty@host:~$ ls -l /media/host-media {{{ ls: cannot access /media/host-media: No such file or directory }}} == read-only bind == * fstab {{{ /media mnt none bind,ro 0 0 }}} * mountコマンドの出力結果は「(rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)」で変わらないが、read-only化される * mitty@lxc:~$ sudo rm /mnt/host-media {{{ rm: cannot remove `/mnt/host-media': Read-only file system }}}