Cannot populate linux container marketplace

Please, describe the problem here and provide additional information below (if applicable) …
Running on debian 11 , I have all the maket places populated except linux containers. The only error in my log is

ommand execution failed (exit code: 255): /var/lib/one/remotes/market/linuxcontainers/monitor PE1BUktFVF9EUklWRVJfQUNUSU9OX0RBVEE+PE1BUktFVFBMQUNFPjxJRD4xMDM8L0lEPjxVSUQ+MDwvVUlEPjxHSUQ+MDwvR0lEPjxVTkFNRT5vbmVhZG1pbjwvVU5BTUU+PEdOQU1FPm9uZWFkbWluPC9HTkFNRT48TkFNRT5MaW51eCBDb250YWluZXJzPC9OQU1FPjxTVEFURT4wPC9TVEFURT48TUFSS0VUX01BRD48IVtDREFUQVtsaW51eGNvbnRhaW5lcnNdXT48L01BUktFVF9NQUQ+PFpPTkVfSUQ+PCFbQ0RBVEFbMF1dPjwvWk9ORV9JRD48VE9UQUxfTUI+MDwvVE9UQUxfTUI+PEZSRUVfTUI+MDwvRlJFRV9NQj48VVNFRF9NQj4wPC9VU0VEX01CPjxNQVJLRVRQTEFDRUFQUFM+PC9NQVJLRVRQTEFDRUFQUFM+PFBFUk1JU1NJT05TPjxPV05FUl9VPjE8L09XTkVSX1U+PE9XTkVSX00+MTwvT1dORVJfTT48T1dORVJfQT4wPC9PV05FUl9BPjxHUk9VUF9VPjA8L0dST1VQX1U+PEdST1VQX00+MDwvR1JPVVBfTT48R1JPVVBfQT4wPC9HUk9VUF9BPjxPVEhFUl9VPjA8L09USEVSX1U+PE9USEVSX00+MDwvT1RIRVJfTT48T1RIRVJfQT4wPC9PVEhFUl9BPjwvUEVSTUlTU0lPTlM+PFRFTVBMQVRFPjxFTkRQT0lOVD48IVtDREFUQVtodHRwczovL2ltYWdlcy5saW51eGNvbnRhaW5lcnMub3JnXV0+PC9FTkRQT0lOVD48RklMRVNZU1RFTT48IVtDREFUQVtleHQ0XV0+PC9GSUxFU1lTVEVNPjxGT1JNQVQ+PCFbQ0RBVEFbcmF3XV0+PC9GT1JNQVQ+PElNQUdFX1NJWkVfTUI+PCFbQ0RBVEFbMTAyNF1dPjwvSU1BR0VfU0laRV9NQj48TUFSS0VUX01BRD48IVtDREFUQVtsaW51eGNvbnRhaW5lcnNdXT48L01BUktFVF9NQUQ+PFNLSVBfVU5URVNURUQ+PCFbQ0RBVEFbeWVzXV0+PC9TS0lQX1VOVEVTVEVEPjwvVEVNUExBVEU+PC9NQVJLRVRQTEFDRT48L01BUktFVF9EUklWRVJfQUNUSU9OX0RBVEE+ 103

the market is 103 because I deleted the market place and recreated hoping that would work. Anyone have ideas what to troubleshoot next ?

Versions of the related components and OS (frontend, debian 11 6.6.0):

Steps to reproduce:

Current results:

Expected results:

You are most likely facing this issue. We have already fixed it and it should be available on the next maintenance release.

If you want to apply the patch. Replace the file /var/lib/one/remotes/market/linuxcontainers/monitor with the following content

#!/usr/bin/env ruby

# -------------------------------------------------------------------------- #
# Copyright 2002-2023, OpenNebula Project, OpenNebula Systems                #
#                                                                            #
# Licensed under the Apache License, Version 2.0 (the "License"); you may    #
# not use this file except in compliance with the License. You may obtain    #
# a copy of the License at                                                   #
#                                                                            #
# http://www.apache.org/licenses/LICENSE-2.0                                 #
#                                                                            #
# Unless required by applicable law or agreed to in writing, software        #
# distributed under the License is distributed on an "AS IS" BASIS,          #
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.   #
# See the License for the specific language governing permissions and        #
# limitations under the License.                                             #
# -------------------------------------------------------------------------- #

require 'net/http'
require 'uri'
require 'json'
require 'base64'
require 'rexml/document'
require 'time'
require 'digest/md5'

require_relative '../common/lxd'

#-------------------------------------------------------------------------------
#
#-------------------------------------------------------------------------------
class LinuxContainersMarket

    #---------------------------------------------------------------------------
    # Default Configuration parameters for the Driver
    #   :url of linuxcontainers market place
    #   :sizemb default size for container images
    #   :fs filesystem for the image file
    #   :format for the image file, qcow2, raw
    #   :agent for HTTP client
    #   :memory Guest RAM
    #   :cpu Guest cpu share with host
    #   :vcpu Guest cores
    #   :privileged container idmap
    #---------------------------------------------------------------------------
    DEFAULTS = {
        :url => 'https://images.linuxcontainers.org',
        :sizemb => 1024,
        :fs => 'ext4',
        :format => 'raw',
        :agent => 'OpenNebula',
        :tested_apps => ['alpine', 'alt', 'centos', 'rockylinux', 'almalinux', 'oracle', 'debian',
                         'ubuntu', 'fedora', 'devuan', 'opensuse', 'amazonlinux'],
        :skip_untested => 'yes',
        :memory => '768',
        :cpu => 1,
        :vcpu => 2,
        :privileged => true
    }

    def initialize(options = {})
        @options = DEFAULTS
        @options.merge!(options)

        version_path = File.dirname(__FILE__) + '/../../VERSION'
        @options[:agent] = "OpenNebula #{File.read(version_path)}" \
            if File.exist? version_path

        @http_client = nil
    end

    # Get container information
    def get(path)
        uri = URI(@options[:url] + path)

        req = Net::HTTP::Get.new(uri.request_uri)
        req['User-Agent'] = @options[:agent]

        response = @http_client.request(req)

        return 0, response.body if response.is_a? Net::HTTPSuccess

        [response.code.to_i, response.msg]
    end

    # Get the list of appliances
    def fetch_appliances
        first_level = '/images/'

        open_connection

        rc, body = get(first_level)

        return rc, body if rc != 0

        distros = body.scan(%r{a href="([a-z].*/)">})
        distros.map! {|d| d.join.chomp('/') }

        if @options[:skip_untested] == 'yes'
            distros.delete_if {|d| !@options[:tested_apps].include?(d) }
        end

        tree = {}

        distros.each do |distro|
            d_suf = "#{distro}/"
            rc, body = get(first_level + d_suf)

            next if rc != 0

            versions = body.scan(%r{a href="(.*/)">})
            versions.shift # Remove first entry ("Parent Directory")

            version_path = {}
            versions.each do |version|
                path = "#{first_level}#{d_suf}#{version[0]}amd64/default/"
                rc, body = get(path)

                next if rc != 0

                release_dates = body.scan(%r{a href="(.*/)">})

                # Previous release_dates array leaves a point in the html page
                release_date  = release_dates.last[0]
                version_path[version[0]] = "#{path}#{release_date}rootfs.tar.xz"
            end

            tree[d_suf] = version_path
        end

        appstr = ''

        #-----------------------------------------------------------------------
        # Generate the container app information
        #-----------------------------------------------------------------------
        tree.each do |distro, value|
            value.each do |version, path|
                path = URI.decode_www_form_component(path)

                description = "Downloaded from #{@options[:url]}"
                regtime = app_time(path).to_s

                data = {
                    'NAME' => "#{distro[0...-1]}_#{version[0...-1]} - LXD",
                    'SIZE' => @options[:sizemb],
                    'PUBLISHER' => @options[:url],
                    'FORMAT' => @options[:format],
                    'REGTIME' => regtime,
                    'SOURCE' => app_url(path).to_s,
                    'MD5' => md5(regtime),
                    'IMPORT_ID' => '-1',
                    'ORIGIN_ID' => '-1',
                    'VERSION' => '1.0',
                    'TYPE' => 'IMAGE',
                    'DESCRIPTION' => description,
                    'TAGS' => '',
                    'LINK' => "#{@options[:url]}#{path}"
                       .gsub('/rootfs.tar.xz', '')
                }

                tmpl = ''
                data.each {|key, val| print_var(tmpl, key, val) }

                tmpl64 = ''
                print_var(tmpl64, 'DRIVER', 'raw')

                data = { 'APPTEMPLATE64' => tmpl64,
                         'VMTEMPLATE64' => LXDMarket.template(@options) }
                data.each do |key, val|
                    print_var(tmpl, key, Base64.strict_encode64(val))
                end

                appstr << "APP=\"#{Base64.strict_encode64(tmpl)}\"\n"
            end
        end

        appstr
    ensure
        @http_client.finish if @http_client
    end

    private

    # Generate the URL for the appliance path of the container. Example:
    #
    # lxd://https://images.linuxcontainers.org/images/ubuntu/xenial/amd64/default/\
    #    ./20181214_07:42/rootfs.tar.xz?size=5120&filesystem=ext4&format=raw
    def app_url(path)
        "lxd://#{@options[:url]}#{path}?size=#{@options[:sizemb]}" \
        "&filesystem=#{@options[:fs]}&format=#{@options[:format]}"
    end

    # Returns build date based on image path
    def app_time(path)
        buildate = path.split('/')[6]

        buildate = DateTime.strptime(buildate, '%Y%m%d_%H:%M')
        buildate.to_time.to_i
    end

    # Print variable in an APP template
    def print_var(str, name, val)
        return if val.nil?
        return if val.class == String && val.empty?

        str << "#{name}=\"#{val}\"\n"
    end

    # Returns an md5 from a combination of the @option hash and an input string
    def md5(string)
        Digest::MD5.hexdigest("#{@options} #{string}")
    end

    # Opens a TCP connection to @options[:url]
    def open_connection
        @http_client.finish if @http_client

        http_proxy = ENV['http_proxy'] || ENV['HTTP_PROXY'] # for ruby 1.9.3

        if http_proxy
            http_proxy = 'http://' + http_proxy if http_proxy !~ /^http/

            p_uri   = URI(http_proxy)
            p_host  = p_uri.host
            p_port  = p_uri.port
        else
            p_host  = nil
            p_port  = nil
        end

        uri = URI(@options[:url])

        response = Net::HTTP.get_response(uri)

        if response.is_a? Net::HTTPRedirection
            @options[:url] = response['location']
            uri = URI(@options[:url])
        end

        @http_client = Net::HTTP.start(uri.hostname, uri.port, p_host, p_port,
                                       :use_ssl => uri.scheme == 'https')
    end

end

################################################################################
# Main Program. Outpust the list of marketplace appliances
################################################################################
def set_option(option, doc, name, path)
    option[name] = doc.elements[path].text if doc.elements[path]
end

begin
    options     = {}
    drv_message = Base64.decode64(ARGV[0])
    doc = REXML::Document.new(drv_message).root

    pre = 'MARKETPLACE/TEMPLATE'

    data = {
        :url => "#{pre}/ENDPOINT",
        :sizemb => "#{pre}/IMAGE_SIZE_MB",
        :fs => "#{pre}/FILESYSTEM",
        :format => "#{pre}/FORMAT",
        :skip_untested => "#{pre}/SKIP_UNTESTED",
        :memory => "#{pre}/MEMORY",
        :cpu => "#{pre}/CPU",
        :vcpu => "#{pre}/VCPU",
        :privileged => "#{pre}/PRIVILEGED"
    }

    data.each {|key, value| set_option(options, doc, key, value) }

    str = LinuxContainersMarket.new(options).fetch_appliances
    puts str
rescue StandardError
    STDERR.puts str
    exit(-1)
end

After that you might need to disable/enable the marketplace, or create a new one, maybe even restart OpenNebula.

Ok That worked !!! Thank you!!! :vulcan_salute: