OpenNebula Marketplace cleanup

This week, we went through the appliances in the OpenNebula Marketplace (https://marketplace.opennebula.org/appliance) and removed those which didn’t work anymore or were too old. We’ll do such checks and cleanups from time to time.

Unfortunately, if you have the OpenNebula 5.x (currently 5.0 - 5.2.1) deployed, this cleanup can require also an action on your side. These versions don’t clean locally synced appliances metadata properly, appliances are still seen and must be deleted manually. The bug will be fixed in following ONE stable releases.

There are 2 options how to workaround the problem:

Fix database

Fix directly your database by cleaning the table marketplaceapp_pool and restart OpenNebula daemons. Please, don’t forget to make a database backup always before any database changes.

  • connect the database configured in the /etc/one/oned.conf (section DB=[]) and clean the table. Examples:

for MySQL: mysql -u oneadmin -p -e 'DELETE FROM marketplaceapp_pool' opennebula_db
for SQLite3: sqlite3 /var/lib/one/one.db 'DELETE FROM marketplaceapp_pool'

  • check there is no activity in the OpenNebula (running action on VM, image etc.) and restart the daemon:
systemctl restart opennebula
or
/etc/init.d/opennebula restart

Use CLI

If you don’t have direct access to the database, you can disable and rename obsolete appliances via the CLI (onemarketapp). Appliances will be still listed, but users won’t be able to use it.

Before we did the cleanup, we injected the obsoleting metadata so that the appliances can be easily identified on a client side. On first sight, these appliances have zero size and placeholder “DELETED” instead of version. E.g.:

$ onemarketapp list
  ID NAME                         VERSION  SIZE STAT TYPE  REGTIME MARKET               ZONE
...
  11 CentOS 6.5 - Xen             DELETED    0M  rdy  img 08/12/14 OpenNebula Public       0

The following commands take these appliances and

  1. disable them
  2. put "DELETED - " prefix into the appliance name
onemarketapp list --csv --filter VERSION=DELETED | \
    grep -v ID,NAME | cut -d, -f2 | \
    xargs -d"\n" -I% -n1 bash -c 'onemarketapp disable "%"; onemarketapp rename "%" "DELETED - %" '

If you are done with any of previous 2 choices, you can check the results in the Sunstone (Storage -> Apps) or via CLI (onemarketapp list). The list should be reduced or obsolete appliances are disabled.

We are sorry for any inconveniences. Please, leave us a message here in case you experience any problems and/or would like to discuss any steps.

Best regards,
Vlastimil Holer

1 Like

I hit an error after using your CLI method:

CentOS 6 - KVM

Wed Feb 26 21:44:12 2020 : Error copying image in the datastore: Error copying http://marketplace.opennebula.systems//appliance/51a7685d-b9a6-43a4-a160-41ab0f6caa5e/download/0 to /var/lib/one//datastores/105/ee875ab2605524a685d983cc0dab50ce

2 problems I faced, actually:

  1. seems the image Centos 6.5 KVM is still in our instance after CLI command, while not in the market place (sorry I lost that error trace, but it was the same as above ). So I tried again with the above (Centos 6 KVM), then hit the second problem:
  2. the downloads server do not deliver sufficient bandwidth: it starts well (10 Mbps then drops at 10k after a short while, yes that’s 1000 times slower). Probably a problem with your hoster or CDN, AWS it seems. Maybe a good old bittorrent would be more practical for those big files. Even without any Sunstone integration whatsoever.