wiki:TipAndDoc/VM/KVM

Version 42 (modified by mitty, 9 years ago) (diff)

--

KVM

  • KVM自体にはIntel-VT or AMD-V が必須
    egrep '(vmx|svm)' --color=always /proc/cpuinfo
    
  • QEMUはVT/AMD-vが無くても動く(ただし非常に遅い)

Client

Windows Guest

drivers

  • デバイスマネージャ

block device

  • virtio-win-1.1.16.vfdを用いてVirtIO diskでセットアップ後、virtio-win-0.1-30.isoによってドライバを更新するとBSODを出して起動しなくなる。
    • OSセットアップ直後
    • ドライバ更新
    • BSOD

Windows 7 x64 with virtio

  • H/W
    • ASUS H97M-PLUS
    • Core i7-4790 3.60GHz
    • W3U1600HQ-8G (DDR3-1600 8GB x2)
    • RAID1 on WD60EFRX-68MYMN1 x2
  • S/W
    • ArchLinux 4.0.4-2-ARCH x86_64
    • qemu 2.3.0-2
    • seabios 1.8.1-1
    • libvirt 1.2.15-1
    • virt-manager 1.2.0-2
    • spice 0.12.5-1
    • virtio-win-0.1.104.iso

virbr0

  • libvirtdによって自動で作られる「default」のNAT
    • ps aux | grep dns
      nobody    2684  0.0  0.0  21264   792 ?        S    Feb07   0:01 dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file=  --listen-address 192.168.122.1 --except-interface lo --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-lease-max=253
      
    • lease file => /var/lib/misc/dnsmasq.leases

iptables

  • sudo iptables-save
    # Generated by iptables-save v1.4.8 on Thu Feb 16 10:31:29 2012
    *nat
    :PREROUTING ACCEPT [743892:129452770]
    :POSTROUTING ACCEPT [1179:112796]
    :OUTPUT ACCEPT [1215:115607]
    -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 
    -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 
    -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE 
    COMMIT
    # Completed on Thu Feb 16 10:31:29 2012
    # Generated by iptables-save v1.4.8 on Thu Feb 16 10:31:29 2012
    *filter
    :INPUT ACCEPT [4957020:3457557240]
    :FORWARD ACCEPT [0:0]
    :OUTPUT ACCEPT [3323525:4327643495]
    -A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT 
    -A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT 
    -A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT 
    -A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT 
    -A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT 
    -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT 
    -A FORWARD -i virbr0 -o virbr0 -j ACCEPT 
    -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable 
    -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable 
    COMMIT
    # Completed on Thu Feb 16 10:31:29 2012
    

port forwarding

Live Migration

kvm-spice

  • Ubuntu 12.04でqemu-kvm-spiceをインストールすると、デフォルトのemulatorが/usr/bin/kvm-spiceになって以下の様な問題が起きる
  • kern.logにAppArmorからDENIEDとエラーが出る
    Oct 13 22:45:08 HOSTNAME kernel: [507466.445098] type=1400 audit(1350135908.372:69): apparmor="DENIED" operation="open" parent=1 profile="libvirt-52d08545-a490-c890-654a-260caf35d27d" name="/proc/17685/auxv" pid=17685 comm="kvm-spice" requested_mask="r" denied_mask="r" fsuid=104 ouid=104
    
  • Video modelをcirrus以外にすると、画面が表示されなくなる(真っ黒のまま)

start/shutdown log for VM

  • Ubuntu 12.04
  • /var/log/libvirt/qemu 以下に、VMごとのログが残される
    -rw------- 1 root root  3940 Oct  8 00:24 quantal.log
    -rw------- 1 root root 12394 Oct 13 23:13 test.log
    
  • /var/log/libvirt/qemu/xp-vm01.log
    2012-10-28 12:46:55.328+0000: starting up
    LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name xp-vm01 -uuid XXXXXXXX-YYYY-ZZZZ-WWWW-VVVVVVVVVVVV -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xp-vm01.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-shutdown -drive file=/var/lib/libvirt/images/xp-vm01.img,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 -drive file=/home/mitty/virtio-win-0.1-30.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=16,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:a8:d0:1d,bus=pci.0,addr=0x3 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:1 -vga vmware -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
    vmsvga_value_read: Bad register 2b
    vmsvga_value_read: Bad register 2c
    qemu: terminating on signal 15 from pid 1803
    2012-10-28 12:51:56.141+0000: shutting down
    

autoshutdown

  • /var/log/libvirt/shutdownlog.log で確認できる
    libvirt: libvirt-bin: entering pre-stop at Fri Oct 19 23:22:33 JST 2012
    libvirt: libvirt-bin: attempting clean shutdown of xp-vm02 at Fri Oct 19 23:22:33 JST 2012
    libvirt: destroying xp-vm02
    libvirt: libvirt-bin: exiting pre-stop at Fri Oct 19 23:23:06 JST 2012
    libvirt: libvirt-bin: entering pre-stop at Fri Oct 26 23:37:35 JST 2012
    libvirt: libvirt-bin: exiting pre-stop at Fri Oct 26 23:37:35 JST 2012
    libvirt: libvirt-bin: entering pre-stop at Sun Oct 28 21:45:17 JST 2012
    libvirt: libvirt-bin: attempting clean shutdown of xp-vm02 at Sun Oct 28 21:45:17 JST 2012
    libvirt: libvirt-bin: attempting clean shutdown of xp-vm01 at Sun Oct 28 21:45:17 JST 2012
    libvirt: libvirt-bin: exiting pre-stop at Sun Oct 28 21:45:45 JST 2012
    
  • /etc/init/libvirt-bin.conf で制御されている模様

passthrough

  • QEMUの引数に追加されるものの例
    01:00.0 Multimedia controller [0480]: Altera Corporation Device [1172:4c15] (rev 01)
    
    -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x9
    

Marvell controller

Plextor M6e

  • PX-G128M6e
    • lspci -nnv
      02:00.0 SATA controller [0106]: Lite-On IT Corp. / Plextor M6e PCI Express SSD [Marvell 88SS9183] [1c28:0122] (rev 14) (prog-if 01 [AHCI 1.0])
              Subsystem: Marvell Technology Group Ltd. Device [1b4b:9183]
      
  • root@archiso ~ # uname -a
    Linux archiso 4.0.4-2-ARCH #1 SMP PREEMPT Fri May 22 03:05:23 UTC 2015 x86_64 GNU/Linux
    
  • root@archiso ~ # cat /proc/cmdline
    BOOT_IMAGE=boot/x86_64/vmlinuz archisobasedir=arch archisolabel=ARCH_201506 initrd=boot/intel_ucode.img,boot/x86_64/archiso.img intel_iommu=on
    
  • root@archiso ~ # dmesg|grep -i -e DMAR -e IOMMU
    [    0.000000] Command line: BOOT_IMAGE=boot/x86_64/vmlinuz archisobasedir=arch archisolabel=ARCH_201506 initrd=boot/intel_ucode.img,boot/x86_64/archiso.img intel_iommu=on
    [    0.000000] ACPI: DMAR 0x00000000DAC6CED0 0000B8 (v01 INTEL  BDW      00000001 INTL 00000001)
    [    0.000000] Kernel command line: BOOT_IMAGE=boot/x86_64/vmlinuz archisobasedir=arch archisolabel=ARCH_201506 initrd=boot/intel_ucode.img,boot/x86_64/archiso.img intel_iommu=on
    [    0.000000] Intel-IOMMU: enabled
    [    0.086964] dmar: Host address width 39
    [    0.086969] dmar: DRHD base: 0x000000fed90000 flags: 0x0
    [    0.086987] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
    [    0.086990] dmar: DRHD base: 0x000000fed91000 flags: 0x1
    [    0.087000] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008c20660462 ecap f010da
    [    0.087003] dmar: RMRR base: 0x000000dbe7b000 end: 0x000000dbe89fff
    [    0.087006] dmar: RMRR base: 0x000000dd000000 end: 0x000000df1fffff
    [    0.087011] IOAPIC id 8 under DRHD base  0xfed91000 IOMMU 1
    [    7.370829] DMAR: No ATSR found
    [    7.371048] IOMMU: dmar0 using Queued invalidation
    [    7.371051] IOMMU: dmar1 using Queued invalidation
    [    7.371056] IOMMU: Setting RMRR:
    [    7.371083] IOMMU: Setting identity map for device 0000:00:02.0 [0xdd000000 - 0xdf1fffff]
    [    7.371527] IOMMU: Setting identity map for device 0000:00:14.0 [0xdbe7b000 - 0xdbe89fff]
    [    7.371577] IOMMU: Setting identity map for device 0000:00:1a.0 [0xdbe7b000 - 0xdbe89fff]
    [    7.371616] IOMMU: Setting identity map for device 0000:00:1d.0 [0xdbe7b000 - 0xdbe89fff]
    [    7.371644] IOMMU: Prepare 0-16MiB unity mapping for LPC
    [    7.371661] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
    [    7.673577] [drm] DMAR active, disabling use of stolen memory
    [    7.673740] dmar: DRHD: handling fault status reg 2
    [    7.673744] dmar: DMAR:[DMA Write] Request device [02:00.1] fault addr fffe0000
                   DMAR:[fault reason 02] Present bit in context entry is clear
    [    7.986473] dmar: DRHD: handling fault status reg 3
    [    7.986493] dmar: DMAR:[DMA Write] Request device [02:00.1] fault addr fffe0000
                   DMAR:[fault reason 02] Present bit in context entry is clear
    [    9.337664] dmar: DRHD: handling fault status reg 2
    [    9.337683] dmar: DMAR:[DMA Write] Request device [00:02.0] fault addr 74ff240000
                   DMAR:[fault reason 05] PTE Write access is not set
    [   12.985456] dmar: DRHD: handling fault status reg 2
    [   12.985519] dmar: DMAR:[DMA Write] Request device [02:00.1] fault addr fffe0000
                   DMAR:[fault reason 02] Present bit in context entry is clear
    [   13.298243] dmar: DRHD: handling fault status reg 3
    [   13.298307] dmar: DMAR:[DMA Write] Request device [02:00.1] fault addr fffe0000
                   DMAR:[fault reason 02] Present bit in context entry is clear
    [   18.297124] dmar: DRHD: handling fault status reg 2
    [   18.297152] dmar: DMAR:[DMA Write] Request device [02:00.1] fault addr fffe0000
                   DMAR:[fault reason 02] Present bit in context entry is clear
    [   18.609924] dmar: DRHD: handling fault status reg 3
    [   18.609971] dmar: DMAR:[DMA Write] Request device [02:00.1] fault addr fffe0000
                   DMAR:[fault reason 02] Present bit in context entry is clear
    [   23.608862] dmar: DRHD: handling fault status reg 2
    [   23.608924] dmar: DMAR:[DMA Write] Request device [02:00.1] fault addr fffe0000
                   DMAR:[fault reason 02] Present bit in context entry is clear
    
  • PX-G128M6eを取り外すと以下のようになる
    [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=0f566cb2-aec8-4122-a80a-f2e5f48e4426 rw intel_iommu=on
    [    0.000000] ACPI: DMAR 0x00000000DAC6CED0 0000B8 (v01 INTEL  BDW      00000001 INTL 00000001)
    [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=0f566cb2-aec8-4122-a80a-f2e5f48e4426 rw intel_iommu=on
    [    0.000000] Intel-IOMMU: enabled
    [    0.107383] dmar: Host address width 39
    [    0.107388] dmar: DRHD base: 0x000000fed90000 flags: 0x0
    [    0.107406] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
    [    0.107409] dmar: DRHD base: 0x000000fed91000 flags: 0x1
    [    0.107419] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008c20660462 ecap f010da
    [    0.107422] dmar: RMRR base: 0x000000dbe7b000 end: 0x000000dbe89fff
    [    0.107425] dmar: RMRR base: 0x000000dd000000 end: 0x000000df1fffff
    [    0.107430] IOAPIC id 8 under DRHD base  0xfed91000 IOMMU 1
    [    0.677861] DMAR: No ATSR found
    [    0.678093] IOMMU: dmar0 using Queued invalidation
    [    0.678096] IOMMU: dmar1 using Queued invalidation
    [    0.678100] IOMMU: Setting RMRR:
    [    0.678127] IOMMU: Setting identity map for device 0000:00:02.0 [0xdd000000 - 0xdf1fffff]
    [    0.678608] IOMMU: Setting identity map for device 0000:00:14.0 [0xdbe7b000 - 0xdbe89fff]
    [    0.678655] IOMMU: Setting identity map for device 0000:00:1a.0 [0xdbe7b000 - 0xdbe89fff]
    [    0.678697] IOMMU: Setting identity map for device 0000:00:1d.0 [0xdbe7b000 - 0xdbe89fff]
    [    0.678725] IOMMU: Prepare 0-16MiB unity mapping for LPC
    [    0.678743] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
    [    4.326005] [drm] DMAR active, disabling use of stolen memory
    

patch

Attachments (17)

Download all attachments as: .zip