Since I can now start a k8s easily, I would like to use it but I’m challenged with the k8s world, mostly the traefik ingress which configure a reverse proxy automatically.
In it’s default configuration, the traefik is configured as a LoadBalancer service but this requires an external piece, generally it’s provided by the cloud provider.
Do you know if OpenNebula provides such LoadBalancer service?
On the other hand, I find kube-vip to create an in k8s LoadBalancer but I need to reserve some public IP pools and make them usable by any of the k8s VMs (all of them cloud be a VIP leader and the VIP could move from one VM to another).
I only find references of the VIP mechanism in ONE for vrouters.
Hey, sorry, but they doesn’t care. I did some contributtion in 2017, but not accepted and since that no development on load-ballancer functionality which is essential for cloud. They invest time and money to new virtual router appliance (Virtual Network Function), but again without load ballancers.
thank you for trying the Kubernetes appliance. As of now there is no built-in LoadBalancer or cloud-provider integrating OpenNebula and Kubernetes but we do intend to develop it eventually (or maybe someone from the community will contribute their own). Unfortunately we cannot commit to any timeframe as of this time.
Vrouter/VNF does implement VIP (floating ip) so your intentions with kube-api (sorry, no experience with it) might work as you described.
LoadBalancer as a new additional VNF for the VNF appliance is also on the roadmap but again without timeframe but this one should not be too hard to add (IMHO) if it is just a generic loadbalancer - of course if the goal is to somehow integrate it with Kubernetes then that is completely different story.
I understand that you are missing the loadbalancer feature in the old and the new vrouter (which was complete rewrite with new additions) - but your contribution (although appreciated) was implemented in PHP and the PR was refused mostly for that reason. We are using shell or ruby and even python if cannot be avoided in the appliances. But we would not want to have each and every part of the functionality in a different language.
I hope that you understand.
The loadbalancer will be added to the VNF appliance for sure (and hopefully other virtual functions) but no time commitment - the roadmap is still in the making for the 6.2 release.
I am sorry for inconveniences and we’ll try to add a loadbalancer soon enough even if you feel it is already long overdue.
Hi Petr, yes, I understand and know that PHP was problem, but it is four years ago. I also offered help with integration and testing LB functionality. I was also asking for sharing OpenNebula Appliances source code, but was refused. If they are open, then we can contribute to appliances, help with development and make ecosystem better. Last summer I want to invest to development of k8s cluster-api driver, but LB is essential functionality.
Deploy kube-vip and configure it to use the holded IP of step 2 (i.e. kubectl create configmap --namespace kube-system plndr --from-literal cidr-global=192.168.0.200/32)
Deploy the traefik ingress controller
That way, the Kubernetes cluster can use the IP 192.168.0.200, I just need to define a domain name pointing to it and then go.
configure kube-vip with CIDR from that pool of IPs
If you plan to create a LoadBalancer, it should have an IP for being able to interact with it, this way the community could write controllers (like the openstack one).
thank you for write up! I think that it is useful for other users!
Can you please clarify more this part for me:
If you plan to create a LoadBalancer , it should have an IP for being able to interact with it, this way the community could write controllers (like the openstack one).
If I am understanding correctly the loadbalancer in your case is kube-api running inside the kubernetes appliance directly. The idea is to have generic loadbalancer in VNF appliance.
Your deployment could be automated by teaching VNF loadbalancing with the support for onegate (there is already keepalived for vrrp which could do loadbalancing as Kristian proposed) and deploy it together with Kubernetes appliance as a service template (as kubernetes now can be). The keepalived/loadbalancer config could even be dynamic enough (by polling the state of onegate) as is done now similarly for SDNAT.
When all this is in place (loadbalancer in VNF, support for onegate and deployment of Kubernetes and VNF as a service) then we still need to implement at least a stub of cloud-provider which will be able to feed onegate. The kube-api might still be needed as the complement to the VNF.
IMHO a simple hacking inside one or two appliances is not enough there must be some support directly inside the OpenNebula itself - to provide API - config in onegate.
Static loadbalancing as Kristian wanted can still be supported but it would be useless for this use-case.
If I misunderstood something then let me know.
EDIT: Actually when SDNAT is already there then that dynamic part is probably already taken care off… What is missing is the preconfigured pool of ip addresses and loadbalancer.
I’m quite new to the k8s world and that beast miss a from baby k8s to god of planetary deployment in 2000 steps documentation
Yes, I need to use kube-vip (not kube-api ;-)) for the LoadBalancer service types to get IPs reachable from the outside but this requires to avoid network protection to let my IPs fly from k8s node VM to k8s node VM as needed (kube-vip use ARP or BGP).
I never tried the VNF (nor Vrouter), but their loadbalancing is an IP flying from one VNF to another, right? Here, the flying IPs will be bound by the kubernetes nodes running the k8s LoadBalancer service type (for example traefik ingress controller).
I must admit that I’m a bit lost here since I don’t master every parts, I need to take time to better express my use case from a user point of view before even thinking about technical details.
I would like to let you know that the appliance was updated. I did evaluate the kube-vip but it did not worked too well (my experience anyway) - but the MetalLB project seemed to be a better choice and more popular.
