Hi,
Since I want to use discard, I changed the BUS device from VirtIO to SCSI in my template.
Unfortunately I noticed, that SCSI is much slower compared to VirtIO.
VirtIO: 380MB/s
SCSI: 180MB/S
I heard that SCSI is more complex, but the differences are still too great in my opinion.
Hi,
Could you answer the following questions:
What OS is the hypervisor?
What version is qemu-kvm? (check the domain XML for the emulator element and query for the program version. For example
[root@one1 ~]# virsh dumpxml one-181 | grep emulator
<emulator>/usr/libexec/qemu-kvm</emulator>
[root@one1 ~]# /usr/libexec/qemu-kvm --version
QEMU emulator version 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.14.1)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
What is the command line of the qemu-kvm process on the hypervisor? (for example ps axwuf | grep one-181)
Best Regards,
Anton Todorov
Hi,
Thank you for your answer.
I use Debian Stretch as OS.
/usr/bin/qemu-system-x86_64 --version
QEMU emulator version 2.8.1(Debian 1:2.8+dfsg-6+deb9u3)
command line:
root 22999 0.0 0.0 15540 936 pts/0 S+ 23:55 0:00 \_ grep one-13
oneadmin 20157 0.2 6.6 2969960 2165076 ? Sl Apr05 4:37 /usr/bin/qemu-system-x86_64 -name guest=one-13,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-14-one-13/master-key.aes -machine pc-i440fx-2.8,accel=kvm,usb=off,dump-guest-core=off -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid cd852157-cef3-47eb-a676-de96b28ccade -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-14-one-13/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device lsi,id=scsi0,bus=pci.0,addr=0x4 -drive file=/var/lib/one//datastores/0/13/disk.0,format=qcow2,if=none,id=drive-scsi0-0-0,cache=none -device scsi-hd,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0,bootindex=1 -drive file=/var/lib/one//datastores/0/13/disk.1,format=raw,if=none,id=drive-ide0-0-0,readonly=on -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:00:5e:82:d6:99,bus=pci.0,addr=0x3 -vnc 0.0.0.0:13 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on-device lsi,id=scsi0,bus=pci.0,addr=0x4
It is emulated LSI SCSI controller. You should figure out how to use virtio-scsi. AFAIK it is the default controller for SCSI on Debian/Ubuntu but it must be possible to define other in OpenNebula.
-drive file=/var/lib/one//datastores/0/13/disk.0,format=qcow2,if=none,id=drive-scsi0-0-0,cache=none
You should have discard=unmap for TRIM to actually work. Set DISCARD=unmap in the VM template or IMAGE template.
Probably you are using the default threaded IO model, to use aio=native you must set the IO=native in the VM template(or image template).
Enabling virsio-scsi queues should help too, but keep in mind that for optimal performance the number of queues should match the number of VCPUs though.
Changing these should have performance close to virtio-blk if not better.
There are further optimizations possible but they need to alter the VM’s XML outside of OpenNebula though.
Hope this helps.
Best Regards,
Anton Todorov
Hi,
Thank you, I have set discard to unmap in the template settings.
But I can’t figure out how to change to virtio-scsi instead of lsi scsi.
I tried setting the bus and bus adapter controler to custom and entering virtio-scsi, but after I safe it, it gets changed to SCSI which is the lsi model.
Looking at the sources, if you set “virtio-scsi queues” the virtio-scsi controller will be used 
BR,
Anton Todorov
This should be right, or?
Unfortunately If I update the template it jumps back to normal scsi
EDIT: Seems to be a known Issue:
virtio-scsi queues are set in the VM Template → OS & CPU → Features
Try using the cli , (asuming your image id is 10)
cat >vmtemplate <<EOF
DISK = [ CACHE="none", DEV_PREFIX="sd", DISCARD="unmap", IMAGE_ID="10", IO="native" ]
FEATURES = [ VIRTIO_SCSI_QUEUES = "1" ]
EOF
onetemplate update $TEMPLATE_ID --append ./vmtemplate
Best Regards,
Anton Todorov
Seems to work, thank you for your effort!
Just for clarification, If I want to use virtio-scsi I have to select SCSI as BUS and under OS&CPU > Features select the number of virtio-scsi queues?
I set it to one, but I hope I don`t have to change this value every time I create a VM with more cores.
Thank you!
Yes.
Setting the BUS=SCSI in sunstone will set only DEV_PREFIX=sd in the VM template which is a hint to libvirt to use default SCSI controller definition in libvirt. On RedHat/CentOS/etc default model is virtio-scsi, but on Debian/Ubuntu it is lsi.
Setting virtio-scsi queues will create a controller definition in the VM XML with model “virtio-scsi”.
I am curious did you managed to measure the disk performance after the changes?
Best Regards,
Anton Todorov
Hi,
Yes, the disk performance is now great.
Host: 387 MB/s
VM: 378 MB/s
Didn’t play with the number of queues yet.