TL;DR: are there plans to allow cold-plugging disk to a VM in an UNDEOPLOYED state, or even merge POWEROFF and UNDEPLOYED to one state?
Details: differences between POWEROFF and UNDEPLOYED are not very clear to my users - having VMs in the POWEROFF state holds the provisioned capacity of a physical host,so I would prefer to hide the POWEROFF state from my users altogether. It is partly possible to hide it from Sunstone menus, but when shutting down a VM from the inside (e.g. using the shutdown -h command), it enters the POWEROFF state nevertheless.
But on the other hand, some actions are not possible in an UNDEPLOYED state. One of them is attaching a new disk. From my point of view, there shouldn’t be a problem with attaching disk to an UNDEPLOYED VM. But Sunstone refuses to do it. For CDROMs, which cannot be attached or detached to a running VM, because of QEMU cdrom limitations, it is a problem. I usually undeploy my VMs, but then discover that I need to use POWEROFF instead, but there is no way to go from UNDEPLOY to POWEROFF without starting the VM first.
Do a separate POWEROFF and UNDEPLOYED states still serve any purpose in current setups?
As you said the main difference is if the VM resources (including the VM folder and disks) remain in the host or not. There are actually 2 pairs of states POWEROFF/UNEPLOY and STOP/SUSPENED that do the same but keeping the VM in the host (POWEROFF and SUSPEND) or in the front-end (UNDEPLOY/STOP).
This means that going back to RUNNING from STOP and UNDEPLOY needs to go through a scheduling round and may end up in a different host. That’s not the case for POWEROFF or SUSPEND.
You can see UNDEPLOY and STOP as long term way to put a VM in standby, while POWEROFF/SUSPEND is more a short/medium term. The price you pay is that resources are reserved for the VM to allow a fast boot.
So we see both situations are useful for different scenarios so we plan to keep both options.
Now on to the attach operation, this is somehow storage related. If you are using a shared storage there is probably no difference, but in SSH mode attaching a disk in UNDEPLOY will have 2 drawbacks: (1) waste storage of the central location that is not going to be used + (2) additional transfer to the central location (i.e. instead of moving the disk directly where the VM is running).
This is also align to the intention of UNDEPLOY/STOP as long term pause states for the VM.
I hope this gives at least some rationale behind the design decisions on these four states.
As for attaching the disk in the UNDEPLOYED state - for shared and read-only CDROM, I don’t see why this would be a problem. If I understand it correctly, in SSH mode, these images already reside on a central node. And they don’t need to be “attached” per se, just marked for attachment during the next DEPLOY. Also, let the user decide, whether he wants to waste a storage or a central server or not. Having to go through deploy-shutdown-diskattach instead of allowing cold-plug in the UNDEPLOYED state is not a good alternative.
Anyway, in my opinion OpenNebula could be made better by not treating all storage models the same and acting as greatest common denominator, using only the subset of features, which can be provided by all backends. It could be better to support some actions depending on features provided by an underlying TM/VMM/etc. It could make sense to support disk attach and detach even in UNDEPLOYED state, provided that the underlying storage is shared (e.g. Ceph). It could also make sense to use native snapshots in Ceph, as I wrote here:
I know this would be a big architectural change, but anyway, thanks in advance for considering this.
Yes I agree, in fact some of the flags we use on the TMs are used to better adapt to the storage system. For example DS_MIGRATE (to allow or not datastore migrations), ALLOW_ORPHANS (to better represent the snapshot mode) etc… So I see no problem in adding other like UNDEPLOY_ATTACH and implment that operation for the TM drivers that make sense to do so.