Select a ssh cipher to improve migration speed

Community Support
1

Hello,
I have 2 KVM nodes, no shared storage
The VM disks are stored locally on the servers, which is fine for our purpose.

I noticed that during VM migration, the network speed is capped at 33MByte/sec (around 280Mbit/sec)
Same result with a scp between nodes
I’ve checked with iperf (1Gbit - good), I’ve checked disks with dd (over 350MByte/sec - good)
Finally I find the culprit: it’s the ssh encryption

I’m using CentOS 8. I changed the cipher, and I had best results using chacha20-poly1305@openssh.com

scp -c chacha20-poly1305@openssh.com bigfile.iso node2:
105.8MB/s

There is a way to force OpenNebula to use a specific cipher during migration tasks?

Thank you
Marco

2

Hi @Marco,

You could export a variable in the shell environment of the oneadmin user (usually in the /var/lib/one/.bashrc file) named SCP="scp -c chacha20-poly1305@openssh.com".
Note that if you are using opennebula 5.12+ you should export one more variable SSH_FWD="ssh -c chacha20-poly1305@openssh.com -o ForwardAgent=yes -o ControlMaster=no -o ControlPath=none"

I think that should be done on both the frontend and the HV hosts…

Edit: Or add a Ciphers definition in /var/lib/one/.ssh/config…

Hope this helps.

Best Regards,
Anton Todorov

1 Like
3

Hello, I think that better is to use libvirt with TCP communication rather that ssh. Search nebula docs for ‘qemu+tcp’

I cannot add link, I don’t know why.

4

Hi @atodorov_storpool
I’ve added a line in oneadmin’s .ssh/config file to force the cipher
Now migrations are way more fast!

This problem is related to CentOs8: some months ago I tested OpenNebula on CentOS7 and the migration speed was fast. Never faced this issue before

Hello @feldsam - I didn’t catch your hint
I searched in OpenNebula’s docs about qemu and tcp but I didn’t found nothing related to this issue (or almost i think)

Thank you!
Marco

5

Hello @Marco, I mean this docs

6

@Marco I can recommend use of Oracle Linux 7.8 with Kernel UEK 5 (it is free) and KVM utils repo. It is pretty stable.