wiki:TipAndDoc/VM/KVM

  1. KVM
    1. Client
  2. Windows Guest
    1. drivers
      1. block device
    2. Windows 7 x64 with virtio
    3. Windows 10
      1. Heavy load on playing sound
        1. Heavy load problem is fixed
    4. power efficiency
  3. virbr0
    1. iptables
      1. port forwarding
  4. Live Migration
  5. kvm-spice
    1. transfer rate
  6. start/shutdown log for VM
    1. autoshutdown
  7. passthrough
  8. Clock
  9. QoS
  10. OVMF

KVM

  • KVM自体にはIntel-VT or AMD-V が必須 
    egrep '(vmx|svm)' --color=always /proc/cpuinfo
  • QEMUはVT/AMD-vが無くても動く(ただし非常に遅い)

Client

Windows Guest

drivers

  • デバイスマネージャ

block device

  • virtio-win-1.1.16.vfdを用いてVirtIO diskでセットアップ後、virtio-win-0.1-30.isoによってドライバを更新するとBSODを出して起動しなくなる。
    • OSセットアップ直後
    • ドライバ更新
    • BSOD

Windows 7 x64 with virtio

  • H/W
    • ASUS H97M-PLUS
    • Core i7-4790 3.60GHz
    • W3U1600HQ-8G (DDR3-1600 8GB x2)
    • RAID1 on WD60EFRX-68MYMN1 x2
  • S/W
    • ArchLinux 4.0.4-2-ARCH x86_64
    • qemu 2.3.0-2
    • seabios 1.8.1-1
    • libvirt 1.2.15-1
    • virt-manager 1.2.0-2
    • spice 0.12.5-1
    • virtio-win-0.1.104.iso

Windows 10

Heavy load on playing sound

  • ver.1511 (TH2)で音声を再生すると、ホストCPUに非常に高い負荷がかかる 解消された模様(後述)
    • ,
Windows 10 TH1 mp3 playingWindows 10 TH2 no soundWindows 10 TH2 mp3 playing

Heavy load problem is fixed

  • 以下の環境下では発生しなくなっていた
  • ホスト
    • ArchLinux 4.6.2-1
    • qemu 2.6.0-1
    • seabios 1.9.2-1
    • libvirt 1.3.5-1
    • virt-manager 1.3.2-5
    • spice 0.12.7-1
    • virtio-win 0.1.118.1-1
  • ゲスト

power efficiency

  • H/W: wiki:TipAndDoc/Hardware#ASUSH97M-PLUS
  • S/W
    • ArchLinux 4.2.5-1-ARCH x86_64
    • qemu 2.4.0.1-1
    • seabios 1.8.2-1
    • libvirt 1.2.20-3
    • virt-manager 1.2.1-1
    • spice 0.12.6-1
    • virtio-win 0.1.110.2-1
  • VM
    • CPU: 2core 2HT
    • Mem: 8GB
    • HDD: qcow2 60GB
  1. no VM (idle): 39W～40W
  2. Win 7 x64 SP1 (idle): 40W
  3. Win 7 x64 SP1 (Chrome x64 with 艦これ): 48W～50W
    • リモートデスクトップ接続
    • 音飛び(ブツブツ音)が若干入る(RDP 8にアップグレードするも効果なし)
  4. Win 10 x64 (idle): 40W
  5. Win 10 x64 (Chrome x64 with 艦これ): 49W～54W
    • リモートデスクトップ接続

virbr0

  • libvirtdによって自動で作られる「default」のNAT
    • ps aux | grep dns 
      nobody    2684  0.0  0.0  21264   792 ?        S    Feb07   0:01 dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file=  --listen-address 192.168.122.1 --except-interface lo --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-lease-max=253
    • lease file => /var/lib/misc/dnsmasq.leases
  • ​Re: [libvirt-users virtual networking - virbr0-nic interface]

    Any pointers on how virbr0 and virbr0-nic work would be really helpful.

    It's a workaround for kernel bug/feature. The bridge's MAC is copied from the first NIC attached. So if one detach all interfaces from a bridge and then attach just one, the bridge will lost previous MAC and gain a new one - just the same as the attached interface has.

    So if libvirt has to ensure a MAC for virtual bridge - it creates this dummy device (no traffic is routed through though) and just attach it to the virtual bridge.

iptables

  • sudo iptables-save 
    # Generated by iptables-save v1.4.8 on Thu Feb 16 10:31:29 2012
*nat
:PREROUTING ACCEPT [743892:129452770]
:POSTROUTING ACCEPT [1179:112796]
:OUTPUT ACCEPT [1215:115607]
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE 
COMMIT
# Completed on Thu Feb 16 10:31:29 2012
# Generated by iptables-save v1.4.8 on Thu Feb 16 10:31:29 2012
*filter
:INPUT ACCEPT [4957020:3457557240]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [3323525:4327643495]
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT 
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT 
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT 
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT 
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT 
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT 
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable 
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable 
COMMIT
# Completed on Thu Feb 16 10:31:29 2012

port forwarding

Live Migration

kvm-spice

  • Ubuntu 12.04でqemu-kvm-spiceをインストールすると、デフォルトのemulatorが/usr/bin/kvm-spiceになって以下の様な問題が起きる
  • kern.logにAppArmorからDENIEDとエラーが出る 
    Oct 13 22:45:08 HOSTNAME kernel: [507466.445098] type=1400 audit(1350135908.372:69): apparmor="DENIED" operation="open" parent=1 profile="libvirt-52d08545-a490-c890-654a-260caf35d27d" name="/proc/17685/auxv" pid=17685 comm="kvm-spice" requested_mask="r" denied_mask="r" fsuid=104 ouid=104
  • Video modelをcirrus以外にすると、画面が表示されなくなる(真っ黒のまま)

transfer rate

  • ネットワーク転送量がRDPに比較して極端に多くなるので注意
    • RDP: 100Kbps以下 (idle) ～ 10Mbps (動画再生)
    • SPICE: 100Kbps以下 (idle) ～ 100Mbps以上 (動画再生)、400Mbps近くなることもあった

start/shutdown log for VM

  • Ubuntu 12.04
  • /var/log/libvirt/qemu 以下に、VMごとのログが残される 
    -rw------- 1 root root  3940 Oct  8 00:24 quantal.log
-rw------- 1 root root 12394 Oct 13 23:13 test.log
  • /var/log/libvirt/qemu/xp-vm01.log 
    2012-10-28 12:46:55.328+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name xp-vm01 -uuid XXXXXXXX-YYYY-ZZZZ-WWWW-VVVVVVVVVVVV -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/xp-vm01.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-shutdown -drive file=/var/lib/libvirt/images/xp-vm01.img,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 -drive file=/home/mitty/virtio-win-0.1-30.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=16,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:a8:d0:1d,bus=pci.0,addr=0x3 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:1 -vga vmware -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
vmsvga_value_read: Bad register 2b
vmsvga_value_read: Bad register 2c
qemu: terminating on signal 15 from pid 1803
2012-10-28 12:51:56.141+0000: shutting down

autoshutdown

  • /var/log/libvirt/shutdownlog.log で確認できる 
    libvirt: libvirt-bin: entering pre-stop at Fri Oct 19 23:22:33 JST 2012
libvirt: libvirt-bin: attempting clean shutdown of xp-vm02 at Fri Oct 19 23:22:33 JST 2012
libvirt: destroying xp-vm02
libvirt: libvirt-bin: exiting pre-stop at Fri Oct 19 23:23:06 JST 2012
libvirt: libvirt-bin: entering pre-stop at Fri Oct 26 23:37:35 JST 2012
libvirt: libvirt-bin: exiting pre-stop at Fri Oct 26 23:37:35 JST 2012
libvirt: libvirt-bin: entering pre-stop at Sun Oct 28 21:45:17 JST 2012
libvirt: libvirt-bin: attempting clean shutdown of xp-vm02 at Sun Oct 28 21:45:17 JST 2012
libvirt: libvirt-bin: attempting clean shutdown of xp-vm01 at Sun Oct 28 21:45:17 JST 2012
libvirt: libvirt-bin: exiting pre-stop at Sun Oct 28 21:45:45 JST 2012
  • /etc/init/libvirt-bin.conf で制御されている模様

passthrough

=> passthrough

Clock

QoS

  • ​libvirt: Domain XML format
  • ex) 
        <interface type='network'>
      <mac address='--:--:--:--:--:--'/>
      <source network='network'/>
      <bandwidth>
        <inbound average='1024' peak='1024' burst='1024'/>
        <outbound average='1024' peak='1024' burst='1024'/>
      </bandwidth>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    • virt-managerでは設定できないが、virt-managerでNICの設定を変更しても<bandwidth></bandwidth>の部分は維持されるので、virsh editで設定すればよい
  • Virtual Network設定で指定できるQoS機能は、ざっと試した限りでは反映されない感じだった

OVMF

  • mitty@kvmhost:~ (master)$ pacaur -S ovmf 
    ovmf /usr/
ovmf /usr/share/
ovmf /usr/share/licenses/
ovmf /usr/share/licenses/ovmf/
ovmf /usr/share/licenses/ovmf/License.txt
ovmf /usr/share/ovmf/
ovmf /usr/share/ovmf/ovmf_ia32.bin
ovmf /usr/share/ovmf/ovmf_x64.bin
  • mitty@kvmhost:~/works (master)$ wget ​https://www.kraxel.org/repos/jenkins/edk2/edk2.git-ovmf-x64-0-20160316.b1616.g6fe9d9c.noarch.rpm
  • mitty@kvmhost:~/works (master)$ wget ​https://www.kraxel.org/repos/jenkins/edk2/edk2.git-ovmf-ia32-0-20160316.b1616.g6fe9d9c.noarch.rpm
  • mitty@kvmhost:~/works (master)$ rpmextract.sh edk2.git-ovmf-x64-0-20160316.b1616.g6fe9d9c.noarch.rpm
  • mitty@kvmhost:~/works (master)$ rpmextract.sh edk2.git-ovmf-ia32-0-20160316.b1616.g6fe9d9c.noarch.rpm
  • mitty@kvmhost:~/works (master)$ sudo cp -R --preserve=timestamp usr/share/* /usr/share/
  • mitty@kvmhost:~ (master)$ sudo vim /etc/libvirt/qemu.conf

    • etc/libvirt/qemu.conf

      a b group="78" 
      515515#   "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd", 
      516516#   "/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd" 
      517517#] 
       518nvram = [ 
       519   "/usr/share/edk2.git/ovmf-ia32/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-ia32/OVMF_VARS-pure-efi.fd", 
       520   "/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd" 
       521] 
      518522 
      519523# The backend to use for handling stdout/stderr output from 
      520524# QEMU processes. 
  • EFI自体は起動するが、OSのブートができない (archlinux-2016.03.01-dual.iso, Win10_1511_Japanese_x64.isoでテスト)

Last modified 7 years ago Last modified on Apr 17, 2018 10:03:34 PM

Attachments (17)

Download all attachments as: .zip