We (my team) is exploring: would OpenNebula KVM/LXD/Firecracker provide a feasible, “drop in” replacement for Proxmox VE?
The key point: we do not, currently have a large, “enterprise” configuration. For us, a single, small (64GB, 6 core) ProxmoxVE (PVE) hardware-server “just works” for our production usage; PVE is a relatively-simple install, the administration is not always that straightforward (OpenNebula seems to have far better documentation), but after we dig around a while, we can usually figure it out and/or get support from PVE’s forums and/or corporate support.
Our initial OpenNebula KVM install ( https://docs.opennebula.io/minione/ ) did not yield a “we can get this work work out-of-the-box in a few mins or hours” experience. First problem: it required “firewall hole punching (within the hypervisor OS) of OpenNebula’s firewall” to make KVM’s visible. (No firewall management is required in PVE until you want it; again, when you being a PVE kvm online it “just works” without much if any additional configuration, for better or worse.) This gives us the feeling that it’s going to be a longer, harder effort to make OpenNebula work for us.
Long term, as our project scales, we suspect we may want to move beyond PVE to a OpenStack-ish, OpenNebula-ish “enterprise” environment. If we could do it now and save a “migration” in the future, that’d be nice. For now: it seems like a lot more work for us to spinup.
But before we discard OpenNebula for our near-term purposes, we ask: would anyone care to offer some perspective as to why we should/would consider OpenNebula a reasonable PVE replacement in our (above) context/evironment?
MiniONE was created as an evaluation tool for a single host deployment. The networking setup it creates is supposed to be simple, universal and applicable to the various OS, versions, configs etc, thus the limitation.
For real deployments it’s supposed to create some fw802.1q or vxlan networking.
One thing that I wanted to take the opportunity to point out is that OpenNebula and Proxmox are completely different animals. Proxmox is a tool for the orchestration of VMs on small clusters, while OpenNebula provides more features for data center virtualization management features and moreover cloud management features. In general, these are some of the features that are not offered by Proxmox:
• Strong multi-tenancy that includes ACLs, VDCs, quotes, accounting, showback…
• Cloud orchestration and provisioning, self-service portal, marketplace for users…
• Multi-VM applications with auto-scaling
• Powerful hooking system for integration and automation
• GUI, CLI and APIs (Ruby, Python, Go and JAVA or XML-RPC)
• Cloud federation and multi-site…
• Embrace hybrid and edge cloud computing
• Support for VMware and Firecracker (besides KVM and LXD)
• Wider support for application containers, Docker, Kubernetes…
• Proxmox does not allow you to scale, while OpenNebula has been tested on 1,250 hosts running 20,000 VMs.
Moreover when you begin to consider the options for commercial support and you compare the Enterprise subscriptions, the Proxmox subscriptions are more expensive than OpenNebula’s.
All seems to make sense, thanks much @jorel and @mabdou. Very helpful, timely, and thorough info.
Our current take:
PromoxVE might (most easily) “get us by” for our (rather) small configuration for now. We suspect OpenNebula (or something like it) might be a more-effective path to scale in the future.
