[[PageOutline]] * LinuX Container * [http://gihyo.jp/admin/serial/01/linux_containers LXCで学ぶコンテナ入門 -軽量仮想化環境を実現する技術:連載|gihyo.jp … 技術評論社] * 細かくまとまっていて良い * [http://www.slideshare.net/enakai/lxc-8300191 Lxc で始めるケチケチ仮想化生活?!] * [http://gihyo.jp/admin/serial/01/ubuntu-recipe/0226 Ubuntu Weekly Recipe:第226回 LXCで軽量仮想環境の活用|gihyo.jp … 技術評論社] * [http://www.stgraber.org/2012/05/04/lxc-in-ubuntu-12-04-lts/ LXC in Ubuntu 12.04 LTS | Stéphane Graber's website] * [http://www.slideshare.net/mkouhei/lxc-cf201207presen LXC 再入門] * [http://www.slideshare.net/masahide_yamamoto/osc2011-nagoya LXC入門 - Osc2011 nagoya] * lxcコンテナから(外側の)ホストOSを操作できる点について具体的に解説がある * https://wiki.ubuntu.com/LxcSecurity Ubuntu 12.04での情報 * [http://pc.amanogawa.info/archives/113 lxc仮想環境の複製とバックアップ | 素人linux] * [http://serverfault.com/questions/429461/no-tun-device-in-lxc-guest-for-openvpn No tun device in lxc guest for openvpn - Server Fault] > {{{ > # mkdir /dev/net > # mknod /dev/net/tun c 10 200 > # chmod 666 /dev/net/tun > }}} * [http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg03671.html Re: (Lxc-users) loop mount inside container] * [http://serverfault.com/questions/475922/mounting-a-network-file-system-inside-lxc-on-ubuntu-12-10 Mounting a network file system inside LXC on Ubuntu 12.10 - Server Fault] * [http://sourceforge.net/tracker/?func=detail&atid=826303&aid=3595947&group_id=163076 SourceForge.net: Linux Containers: Detail: 3595947 - lxc-ip script to get IP addresses of the container] * ホスト側のvethデバイス名を指定する * [http://osdir.com/ml/lxc-chroot-linux-containers/2011-06/msg00058.html lxc-chroot-linux-containers - Re: (Lxc-users) veth name - msg#00058 - Recent Discussion OSDir.com] * man 5 lxc.conf > By default lxc choose a name for the network device belonging to the outside of the container, this name is handled by lxc, but if you wish to handle this name yourself, you can tell lxc to set a specific name with the {{{lxc.network.veth.pair}}} option. * 複数のコンテナで同じ名前のpairを指定すると、二つ目は起動しない(下記の例ではtest1, test2コンテナで{{{lxc.network.veth.pair = test}}}と指定) * $ sudo lxc-start -n test2 {{{ lxc-start: failed to create test-vethNdql6o : File exists lxc-start: failed to create netdev lxc-start: failed to create the network lxc-start: failed to spawn 'test2' lxc-start: No such file or directory - failed to remove cgroup '/sys/fs/cgroup/cpu/sysdefault/lxc/test2' }}} * 指定しない場合、mktemp("vethXXXXXX")で決定される * [http://lxc.git.sourceforge.net/git/gitweb.cgi?p=lxc/lxc;a=blob;f=src/lxc/conf.c;h=2eb598bc6b4511a91b2122beb54db27a7c12491f;hb=6371febf3c7d181cb0f11209df67c35f9d7b233b#l1376 SourceForge - lxc/lxc/blob - src/lxc/conf.c] > {{{static int instanciate_veth(struct lxc_handler *handler, struct lxc_netdev *netdev)}}} * ホスト側の物理NIC(lxc.network.link)とリンクするveth1の名前やifindexを外部から知る手段が無い模様 * $ sudo lxc-start -d -n test -o test.log -l DEBUG {{{ lxc-start 1356615687.534 DEBUG lxc_conf - instanciated veth 'vethtgqRWu/vethnwRgNf', index is '99' }}} * {{{lxc.network.veth.pair = vtest}}} とすると以下の様になる {{{ lxc-start 1356615810.522 DEBUG lxc_conf - instanciated veth 'vtest/vethA3QwQ9', index is '102' }}} * [http://d.hatena.ne.jp/defiant/20120914/1347619058 Ubuntu 12.04 の lxc (1) - TenForwardの日記] * コンテナの中にいるかどうかをどうやって判別しているか * [http://d.hatena.ne.jp/defiant/20121204/1354611837 Ubuntu 12.04 の lxc (2) - TenForwardの日記] * AppArmorについて * [http://d.hatena.ne.jp/defiant/20130301/1362124706 Linux 3.8 で改良された Namespace 機能と lxc-attach コマンド - TenForwardの日記] > 3.7 までの名前空間をサポートした標準のカーネルではこのコマンドは動作しない状態でした. * [http://d.hatena.ne.jp/defiant/20130213/1360760602 Linux Kernel 3.8 の User Namespace 機能 (1) - TenForwardの日記] > 今までもコンテナごとに /etc/passwd などを置いて,それぞれでユーザ管理を行うことは可能でした.でもコンテナで UID=0 のユーザがいたとすると,ホストOS上や他のコンテナでもそのユーザは UID=0 でしたし,コンテナ内で root (UID=0) の権限で実行しているプロセスは,ホストOSや他のコンテナでも root (UID=0) の権限で実行している事になっていました. * [http://d.hatena.ne.jp/defiant/20130218/1361170507 Linux 3.8 の User Namespace 機能 (2) - TenForwardの日記] > /proc/PID/uid_map or gid_map を開いて文字列書いてるだけですね.ちなみに設定ファイルには > {{{ > lxc.id_map = U 100000 0 10000 > lxc.id_map = G 100000 0 10000 > }}} > こんな感じに書きます.これで名前空間内では 0-10000 の ID が,ホスト上では 100000-110000 となります. * [http://d.hatena.ne.jp/defiant/20130507/1367924268 Linux 3.8 の User Namespace 機能 (3) - TenForwardの日記] > 3.8 で実装が完了した! ということで楽しみにしていたユーザ名前空間 (User Namespace) ですが,3.8 の時点ではカーネルのかなりの機能を無効にしないと有効に出来ない状態でした. * [http://d.hatena.ne.jp/defiant/20130509/1368093420 Linux 3.8 の User Namespace 機能 (4) - TenForwardの日記] > 3.9 kernel での準備が出来たので,今回は少しだけユーザ名前空間を体験してみました. * mitty@precise:~$ lxc-create -t ubuntu -h {{{ usage: lxc-create -n [-f configuration] [-t template] [-h] -- [template_options] usage: lxc-create -n [-f configuration] [-t template] [-h] [fsopts] -- [template_options] fsopts: -B none fsopts: -B lvm [--lvname lvname] [--vgname vgname] [--fstype fstype] [--fssize fssize] fsopts: -B btrfs flag is not necessary, if possible btrfs support will be used creates a lxc system object. Options: name : name of the container configuration: lxc configuration template : lxc-template is an accessible template script The container backing store can be altered using '-B'. By default it is 'none', which is a simple directory tree under /var/lib/lxc//rootfs Otherwise, the following option values may be relevant: lvname : [for -lvm] name of lv in which to create lv, container-name by default vgname : [for -lvm] name of vg in which to create lv, 'lxc' by default fstype : name of filesystem to create, ext4 by default fssize : size of filesystem to create, 1G by default template-specific help follows: (these options follow '--') /usr/lib/lxc/templates/lxc-ubuntu -h|--help [-a|--arch] [-b|--bindhome ] [--trim] [-d|--debug] [-F | --flush-cache] [-r|--release ] [ -S | --auth-key ] release: the ubuntu release (e.g. precise): defaults to host release on ubuntu, otherwise uses latest LTS trim: make a minimal (faster, but not upgrade-safe) container bindhome: bind 's home into the container The ubuntu user will not be created, and will have sudo access. arch: the container architecture (e.g. amd64): defaults to host arch auth-key: SSH Public key file to inject into container }}} * [https://www.stgraber.org/2013/12/20/lxc-1-0-blog-post-series/ LXC 1.0: Blog post series [0/10] | Stéphane Graber's website] * [https://www.stgraber.org/2014/02/09/lxc-1-0-gui-in-containers/ LXC 1.0: GUI in containers [9/10] | Stéphane Graber's website] * [http://unix.stackexchange.com/questions/102204/executing-a-command-inside-a-running-lxc linux - Executing a command inside a running LXC - Unix & Linux Stack Exchange] > lxc-attach - start a process inside a running container. * [http://stackoverflow.com/questions/21554651/lxc-attach-failed-to-enter-the-namespace-ec2-instances amazon ec2 - lxc-attach failed to enter the namespace - EC2 Instances - Stack Overflow] > lxc-attach requires features that are not present in the native 12.04 kernel (3.5). You need at least 3.8 which IIRC is available in the backport. * [http://www.linuxquestions.org/questions/slackware-14/lxc-start-since-last-upgrade-in-current-reproducible-error-4175559143/ lxc-start since last upgrade in -current, reproducible error] > lxc-start: utils.c: setproctitle: 1461 Invalid argument - setting cmdline failed > * [https://lists.linuxcontainers.org/pipermail/lxc-users/2015-November/010411.html (lxc-users) 1.1.5 setproctitle bug] > > No, this is a non-fatal error, so you're just fine. * [http://unix.stackexchange.com/questions/71527/arch-linux-pacman-does-not-work-when-chrooting grub2 - Arch linux : pacman does not work when chrooting - Unix & Linux Stack Exchange] > Remember to {{{arch-chroot}}} and not simply {{{chroot}}} - that way {{{/proc}}} will be populated and pacman will function as expected. * [http://bryanalves.github.io/2015/07/02/enabling-lxc-archlinux-with-systemd-networkd/ Enabling LXC on Archlinux with systemd-networkd – Bryan's Blah Blog] > I’m on a laptop with wireless, so I can’t just create a bridge and give IPs to containers from an upstream DHCP server. I needed to do NAT and essentially simulate/replicate how docker networking works. = two NICs in one container = * [http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg02037.html Re: (Lxc-users) two NICs in container] == example == * 通常のbridge(veth)ではなくmacvlanであるが、vethでも同じはず * /var/lib/lxc/test/config {{{ lxc.network.type=macvlan lxc.network.macvlan.mode=bridge lxc.network.link=em1 lxc.network.flags=up lxc.network.hwaddr = 00:16:3e:85:2e:da lxc.network.type=macvlan lxc.network.macvlan.mode=bridge lxc.network.link=em1 lxc.network.flags=up lxc.network.hwaddr = 00:16:3e:85:2e:db }}} * mitty@test:~$ ifconfig -a | egrep 'addr|Link' {{{ eth0 Link encap:Ethernet HWaddr 00:16:3e:85:2e:da inet addr:192.168.83.207 Bcast:192.168.83.255 Mask:255.255.255.0 inet6 addr: fe80::216:3eff:fe85:2eda/64 Scope:Link eth1 Link encap:Ethernet HWaddr 00:16:3e:85:2e:db inet addr:192.168.83.212 Bcast:192.168.83.255 Mask:255.255.255.0 inet6 addr: fe80::216:3eff:fe85:2edb/64 Scope:Link lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host }}} * mitty@test:~$ ip route {{{ default via 192.168.83.243 dev eth0 192.168.83.0/24 dev eth0 proto kernel scope link src 192.168.83.207 192.168.83.0/24 dev eth1 proto kernel scope link src 192.168.83.212 }}} = mount bind = * /var/lib/lxc/>container name>/fstab を用いて、ホストの特定ディレクトリ以下をゲストにbind出来る * fstab {{{ /media mnt none bind 0 0 }}} * host:/media -> guest:/mnt とマウントされる * mntに「/」が'''無い'''ことに注意 * mitty@lxc:~$ mount {{{ /dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX on /mnt type ext4 (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered) }}} * mitty@lxc:~$ ls -l /mnt/ {{{ total 0 -rw-r--r-- 1 root root 0 Oct 23 22:01 host-media }}} 1. mitty@lxc:~$ sudo rm /mnt/host-media {{{ removed `/mnt/host-media' }}} 1. mitty@host:~$ ls -l /media/host-media {{{ ls: cannot access /media/host-media: No such file or directory }}} == read-only bind == * fstab {{{ /media mnt none bind,ro 0 0 }}} * mountコマンドの出力結果は「(rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)」で変わらないが、read-only化される * mitty@lxc:~$ sudo rm /mnt/host-media {{{ rm: cannot remove `/mnt/host-media': Read-only file system }}} = X11 with VNC = * [http://box.matto.nl/lxcxserver.html Set up a LXC Linux Container as Xserver howto | box.matto.nl] * [http://unix.stackexchange.com/questions/18003/linux-lxc-deploying-images-with-tiniest-possible-x11 xorg - Linux - LXC; deploying images with tiniest possible X11 - Unix and Linux] * 12.04上のLXCでは、configのcgroupの設定やmknodなどは特に必要ない模様 * vnc自体については => [../../network/vnc#vncserver] * 基本的に{{{aptitude install -R}}}でインストールしている * エラーメッセージについては ~/.vnc/ 以下に保存されるログファイルから == common packages == * vnc4server * xfonts-base {{{ could not open default font 'fixed' }}} == GNOME == * gnome * unity-2d {{{ gnome-session[5129]: WARNING: GSIdleMonitor: IDLETIME counter not found gnome-session[5129]: WARNING: Session 'ubuntu' runnable check failed: Exited with code 1 gnome-session[5129]: WARNING: Unable to find default provider 'unity-2d-panel' of required provider 'panel' gnome-session[5129]: WARNING: Unable to find default provider 'unity-2d-shell' of required provider 'shell' }}} * [https://forums.ubuntulinux.jp/viewtopic.php?id=12231 Ubuntu日本語フォーラム / 11.10でUnityを削除してみたら起動しなくなってしまいました] * [[Image(gnome.png,33%)]] * ~/.vnc/xstartup {{{#!sh xsetroot -solid grey exec gnome-session & }}} * [[Image(gnome-unity-2d.png,33%)]] == Xfce == * xfce4 * ~/.vnc/xstartup {{{#!sh xsetroot -solid grey exec startxfce4 & }}} * もしくは {{{#!sh xsetroot -solid grey exec xfce4-session & }}} * 後者だとTrashディレクトリなどがFile Managerに表示されない * [[Image(xfce4.png,33%)]] * ファイルアイコンが表示されない == LXDE == * lxde * ~/.vnc/xstartup {{{#!sh xsetroot -solid grey exec lxsession -s LXDE & }}} * -s LXDEは無くても良い模様 * [[Image(lxde.png,33%)]] = mount inside container = * コンテナの中ではAppArmorによってmountが禁止されているため、直接NFSマウントするにはAppArmorの再設定が必要 == ホストで直接mountし、コンテナには--bindでアクセスさせる == 1. host# mount nfsserver:/path/to/export /mnt/somewhere 1. /var/lib/lxc/CONTAINER/fstab {{{ /mnt/somewhere path/to/mountdir none bind 0 0 }}} 1. host# lxc-start -d -n CONTAINER 1. lxc$ mount {{{ nfsserver:/path/to/export on /path/to/mountdir type nfs4 (rw,relatime,...) }}} * ホスト上でread-onlyマウントしていても、/var/lib/lxc/CONTAINER/fstabでのread/write権限設定によって上書きされるので注意 * cifsも同じやりかたでマウントできる * cifsではホスト上でread-onlyマウントしていると、設定は一部(?)引き継がれる模様 * touch hogeすると、{{{touch: cannot touch `/path/to/mountdir/hoge': Read-only file system}}}とエラーになるが、実際にはファイルは生成される * 0バイトのファイルは生成されるが、データの書き込みは出来ない模様 * '''削除は出来ない'''ため、削除できないファイルが作成されてしまう点に注意 * ホスト上のマウントディレクトリを直接コンテナのディレクトリにbindする必要がある。つまり、以下のようなことは出来ない 1. host# mount nfsserver:/path/to/export/dir1 /mnt/somewhere/dir1 1. lxc/CONTAINER/fstab {{{ /mnt/somewhere path/to/mountdir none bind 0 0 }}} 1. lxc$ ls -l /path/to/mountdir/dir1 * nfsserver:/path/to/export/dir1 が見えることが期待されるが、実際にはマウントされていない = backup = * tarballにする場合、tarコマンドに{{{--numeric-owner}}}を付けること * [http://blog.xe.bz/archives/51645042.html AKIBA de: (メモ) tarの--numeric-ownerオプションはどう働くのか?] = DHCP = == no valid interfaces found == * Arch Linux 4.4.5-1 * lxc 1:1.1.5-4 * dhcpcd 6.10.1-1 * # dhcpcd {{{ dev: loaded udev no valid interfaces found no interfaces have a carrier forked to background, child pid 65 }}} * # dhcpcd eth0 {{{ DUID 00:01:00:01:1e:8a:6a:5e:ba:4a:37:94:4a:9d eth0: IAID 16:b6:47:44 eth0: rebinding lease of 192.168.0.185 eth0: NAK: address in use from 192.168.0.1 eth0: message: address in use eth0: soliciting a DHCP lease eth0: soliciting an IPv6 router eth0: offered 192.168.0.172 from 192.168.0.1 eth0: probing address 192.168.0.172/24 eth0: leased 192.168.0.172 for 86400 seconds eth0: adding route to 192.168.0.0/24 eth0: adding default route via 192.168.0.1 forked to background, child pid 123 }}} * LXCコンテナ内で、udevdが起動しないことが原因の模様 * strace -f dhcpcd * コンテナホスト(検証はKVMゲストで行った) {{{ access("/run/udev/control", F_OK) = 0 open("/sys/devices/virtual/net/lo/uevent", O_RDONLY|O_CLOEXEC) = 9 fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 read(9, "INTERFACE=lo\nIFINDEX=1\n", 4096) = 23 read(9, "", 4096) = 0 read(9, "", 4096) = 0 close(9) = 0 open("/run/udev/data/n1", O_RDONLY|O_CLOEXEC) = 9 open("/sys/devices/pci0000:00/0000:00:03.0/virtio0/net/ens3/uevent", O_RDONLY|O_CLOEXEC) = 9 fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 read(9, "INTERFACE=ens3\nIFINDEX=2\n", 4096) = 25 read(9, "", 4096) = 0 read(9, "", 4096) = 0 close(9) = 0 open("/run/udev/data/n2", O_RDONLY|O_CLOEXEC) = 9 }}} * LXCコンテナ {{{ access("/run/udev/control", F_OK) = -1 ENOENT (No such file or directory) open("/sys/devices/virtual/net/lo/uevent", O_RDONLY|O_CLOEXEC) = 9 fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 read(9, "INTERFACE=lo\nIFINDEX=1\n", 4096) = 23 read(9, "", 4096) = 0 read(9, "", 4096) = 0 close(9) = 0 open("/run/udev/data/n1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/sys/devices/virtual/net/eth0/uevent", O_RDONLY|O_CLOEXEC) = 9 fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 fstat(9, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 read(9, "INTERFACE=eth0\nIFINDEX=6\n", 4096) = 25 read(9, "", 4096) = 0 read(9, "", 4096) = 0 close(9) = 0 open("/run/udev/data/n6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) write(2, "no valid interfaces found", 25) = 25 write(2, "\n", 1) = 1 write(1, "no interfaces have a carrier\n", 29) = 29 }}} * [root@test /]# systemctl status systemd-udevd {{{ ● systemd-udevd.service - udev Kernel Device Manager Loaded: loaded (/usr/lib/systemd/system/systemd-udevd.service; static; vendor preset: disabled) Active: inactive (dead) Condition: start condition failed at Sat 2016-04-02 08:30:59 UTC; 5h 9min ago ConditionPathIsReadWrite=/sys was not met Docs: man:systemd-udevd.service(8) man:udev(7) }}} * arch:Linux_Containers#Systemd_considerations_.28required.29 * 上記を参考にsystemdの設定を行ったが効果なし * arch:dhcpcd#dhcpcd_and_systemd_network_interfaces > {{{dhcpcd.service}}} can be [arch:Enabled] without specifying an interface. This may, however, create a race condition at boot with ''systemd-udevd'' trying to apply a predictable network interface name: > {{{ > error changing net interface name wlan0 to wlp4s0: Device or resource busy" > }}} > To avoid it, enable dhcpcd per interface it should bind to as described in [arch:dhcpcd#Running]. The downside of the template unit is, however, that it does not support hot-plugging of a wired connection and will fail if the network cable is not connected. To work-around the failure, see [arch:dhcpcd#Timeout_delay]. とのことなので、いずれにせよ{{{dhcpcd.service}}}を使用しないのがよさそう * 古い情報 * [http://blog.oddbit.com/2013/02/01/dhcpcd-under-lxc/ Odd Bits] {{{dhcpcd[492]: eth0: if_init: Read-only file system}}} == Dependency failed for dhcpcd on eth0. == * # systemctl start dhcpcd@eth0 {{{ A dependency job for dhcpcd@eth0.service failed. See 'journalctl -xe' for details. }}} * # journalctl -xe {{{ Apr 10 20:00:56 test3 systemd[1]: Starting of sys-subsystem-net-devices-eth0.device not supported. -- Subject: Unit sys-subsystem-net-devices-eth0.device has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit sys-subsystem-net-devices-eth0.device has failed. -- -- The result is unsupported. Apr 10 20:00:56 test3 systemd[1]: Dependency failed for dhcpcd on eth0. -- Subject: Unit dhcpcd@eth0.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit dhcpcd@eth0.service has failed. -- -- The result is dependency. Apr 10 20:00:56 test3 systemd[1]: dhcpcd@eth0.service: Job dhcpcd@eth0.service/start failed with result 'dependency'. }}} * /usr/lib/systemd/system/dhcpcd@.service {{{ [Unit] Description=dhcpcd on %I Wants=network.target Before=network.target BindsTo=sys-subsystem-net-devices-%i.device After=sys-subsystem-net-devices-%i.device [Service] Type=forking PIDFile=/run/dhcpcd-%I.pid ExecStart=/usr/bin/dhcpcd -q -w %I ExecStop=/usr/bin/dhcpcd -x %I [Install] WantedBy=multi-user.target }}} * {{{BindsTo=}}}, {{{After=}}}の行を削除することで対応可能 * see also * [https://bbs.archlinux.org/viewtopic.php?id=164753 Netctl doesn't work in LXC (Linux containers) due to udev dependency / Networking, Server, and Protection / Arch Linux Forums] * [https://bugs.archlinux.org/task/35715 FS#35715 : (netctl) fails to start network inside Linux Container]