AWS provisions have stopped working

Samu · February 1, 2022, 10:27am

Without any change on my part, either in my code or in OpenNebula the AWS provisions have stopped working. So i’m guessing it has something to do with either terraform or ansible.

I’m getting this error when provisioning from either CLI or FireEdge, I’ve tried different providers on different AWS zones with the same result

WARNING: This operation can take tens of minutes. Please be patient.
- Waiting until host(s) are ready in OpenNebula
ERROR: Failed to configure hosts
- 3.71.16.23 : TASK[Install libselinux-python3 (Centos8 8)] - non-zero return code
- 18.192.49.157 : TASK[Install libselinux-python3 (Centos8 8)] - non-zero return code
1. quit
2. retry
3. skip
4. cleanupe

Also tried removing the providers and following the tutorial, but i’m getting the same error Provisioning an Edge Cluster — OpenNebula 6.2.0 documentation

I’m using OpenNebula 6.2 CE

Samu · February 2, 2022, 10:41am

a more detailed log is below, seems like ansible can’t install linux dependencies

detailed log, fail at the end

Executing command: create

2022-02-02 10:32:49 +0000

Command options: batch debug [json, true] [user, serveradmin:oneadmin] [password…

2022-02-02 10:32:49 +0000

Creating provision objects

2022-02-02 10:32:49 +0000

Creating OpenNebula cluster: ${provision}

2022-02-02 10:32:49 +0000

Cluster created with ID: 173

2022-02-02 10:32:49 +0000

Creating datastore ${provision}-image

2022-02-02 10:32:50 +0000

datastore created with ID: 246

2022-02-02 10:32:50 +0000

Creating datastore ${provision}-system

2022-02-02 10:32:50 +0000

datastore created with ID: 247

2022-02-02 10:32:50 +0000

Creating network ${provision}-public

2022-02-02 10:32:51 +0000

network created with ID: 73

2022-02-02 10:32:51 +0000

Creating OpenNebula host: provision-5e830b79ed9282688ae1c99af6509bc70ce819cae381…

2022-02-02 10:32:51 +0000

host created with ID: 240

WARNING: This operation can take tens of minutes. Please be patient.

2022-02-02 10:32:52 +0000

Deploying

2022-02-02 10:32:52 +0000

Command run: terraform version --json

ID: 89

2022-02-02 10:32:52 +0000

Command STDOUT: { “terraform_version”: “0.14.7”, “terraform_revision”: “”, …

2022-02-02 10:32:52 +0000

Command succeeded

2022-02-02 10:32:52 +0000

Command run: cd /tmp/tf20220202-8551-txls9h; terraform 0.13upgrade -yes

2022-02-02 10:32:53 +0000

Command STDOUT: The 0.13upgrade command has been removed. You must run this comm…

2022-02-02 10:32:53 +0000

Command succeeded

2022-02-02 10:32:53 +0000

Command run: cd /tmp/tf20220202-8551-txls9h; terraform init

2022-02-02 10:32:56 +0000

Command STDOUT: Initializing the backend… Initializing pr…

2022-02-02 10:32:56 +0000

Command succeeded

2022-02-02 10:32:56 +0000

Command run: cd /tmp/tf20220202-8551-txls9h; export TF_LOG=DEBUG; terraform appl…

2022-02-02 10:33:55 +0000

Command STDOUT: aws_vpc.device_173: Creating… aws_vpc.device_173: Creation complete after 1s [id=vpc-08d8eeeeec5ca5e0d] aws_subnet.device_173: Creating… aws_internet_gateway.device_173: Creating… aws_security_group.device_173_all: Creating… aws_internet_gateway.device_173: Creation complete after 1s [id=igw-0ed3a0bdba512f09d] aws_route.device_173: Creating… aws_route.device_173: Creation complete after 0s [id=r-rtb-06feb7e13cb0648bb1080289494] aws_security_group.device_173_all: Creation complete after 3s [id=sg-0dc91292750db29b6] aws_subnet.device_173: Still creating… [10s elapsed] aws_subnet.device_173: Creation complete after 11s [id=subnet-06362b8a76415af47] aws_instance.device_240: Creating… aws_instance.device_240: Still creating… [10s elapsed] aws_instance.device_240: Still creating… [20s elapsed] aws_instance.device_240: Still creating… [30s elapsed] aws_instance.device_240: Still creating… [40s elapsed] aws_instance.device_240: Creation complete after 42s [id=i-0be25dc1fd96243e4] Warning: Interpolation-only expressions are deprecated on deploy.tf line 83, in data “aws_ec2_instance_type_offerings” “available”: 83: values = ["${each.key}"] Terraform 0.11 and earlier required all non-constant expressions to be provided via interpolation syntax, but this pattern is now deprecated. To silence this warning, remove the “${ sequence from the start and the }” sequence from the end of this expression, leaving just the inner expression. Template interpolation syntax is still used to construct strings from expressions when the template includes multiple interpolation sequences or a mixture of literal strings and interpolations. This deprecation applies only to templates that consist entirely of a single interpolation sequence. Apply complete! Resources: 6 added, 0 changed, 0 destroyed. Outputs: device_id_240 = “i-0be25dc1fd96243e4” ip_240 = “18.170.218.85”

2022-02-02 10:33:55 +0000

Command STDERR: 2022/02/02 10:32:56 [WARN] Log levels other than TRACE are curre…

2022-02-02 10:33:55 +0000

Command succeeded

2022-02-02 10:33:55 +0000

Command run: cd /tmp/tf20220202-8551-txls9h; terraform output

2022-02-02 10:33:56 +0000

Command STDOUT: device_id_240 = “i-0be25dc1fd96243e4” ip_240 = “18.170.218.85”

2022-02-02 10:33:56 +0000

Command succeeded

2022-02-02 10:33:56 +0000

Monitoring hosts

2022-02-02 10:33:56 +0000

Generating Ansible configurations into /tmp/d20220202-8551-wilnbj

2022-02-02 10:33:56 +0000

Creating /tmp/d20220202-8551-wilnbj/inventory: [nodes] 18.170.218.85 [targets] 18.170.218.85 ansible_connection=ssh ansible_ssh_private_key_file=/var/lib/one/.ssh-oneprovision/id_rsa ansible_user=centos ansible_port=22

2022-02-02 10:33:56 +0000

Creating /tmp/d20220202-8551-wilnbj/group_vars.yml: — sys_ds_ids: - 247 first_host: 18.170.218.85

2022-02-02 10:33:56 +0000

Creating /tmp/d20220202-8551-wilnbj/host_vars/18.170.218.85.yml: — oneprovision_hypervisor: qemu opennebula_ssh_deploy_private_key: true opennebula_node_kvm_use_ev: true opennebula_node_kvm_param_nested: true opennebula_node_kvm_manage_kvm: false

2022-02-02 10:33:56 +0000

Creating /tmp/d20220202-8551-wilnbj/ansible.cfg: [defaults] retry_files_enabled = False deprecation_warnings = False display_skipped_hosts = False callback_whitelist = stdout_callback = skippy host_key_checking = False roles_path = /usr/share/one/oneprovision/ansible/roles [privilege_escalation] become = yes become_user = root

2022-02-02 10:33:56 +0000

Checking working SSH connection

2022-02-02 10:33:56 +0000

Command run: ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg ANSIBLE_BECOM…

2022-02-02 10:34:07 +0000

Command STDOUT: 18.170.218.85 | UNREACHABLE! => { “changed”: false, "msg…

2022-02-02 10:34:07 +0000

Command STDERR: [WARNING]: Skipping callback plugin ‘’, unable to load

2022-02-02 10:34:07 +0000

Command FAILED (code=4): ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg A…

2022-02-02 10:34:27 +0000

Command run: ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg ANSIBLE_BECOM…

2022-02-02 10:34:28 +0000

Command STDOUT: 18.170.218.85 | UNREACHABLE! => { “changed”: false, "msg…

2022-02-02 10:34:28 +0000

Command STDERR: [WARNING]: Skipping callback plugin ‘’, unable to load

2022-02-02 10:34:28 +0000

Command FAILED (code=4): ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg A…

2022-02-02 10:34:48 +0000

Command run: ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg ANSIBLE_BECOM…

2022-02-02 10:34:48 +0000

Command STDOUT: 18.170.218.85 | UNREACHABLE! => { “changed”: false, "msg…

2022-02-02 10:34:48 +0000

Command STDERR: [WARNING]: Skipping callback plugin ‘’, unable to load

2022-02-02 10:34:48 +0000

Command FAILED (code=4): ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg A…

2022-02-02 10:35:08 +0000

Command run: ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg ANSIBLE_BECOM…

2022-02-02 10:35:19 +0000

Command STDOUT: 18.170.218.85 | UNREACHABLE! => { “changed”: false, "msg…

2022-02-02 10:35:19 +0000

Command STDERR: [WARNING]: Skipping callback plugin ‘’, unable to load

2022-02-02 10:35:19 +0000

Command FAILED (code=4): ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg A…

2022-02-02 10:35:39 +0000

Command run: ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg ANSIBLE_BECOM…

2022-02-02 10:35:42 +0000

Command STDOUT: 18.170.218.85 | CHANGED | rc=0 >> Warning: Permanently added '18…

2022-02-02 10:35:42 +0000

Command STDERR: [WARNING]: Skipping callback plugin ‘’, unable to load

2022-02-02 10:35:42 +0000

Command succeeded

2022-02-02 10:35:42 +0000

Configuring hosts

2022-02-02 10:35:42 +0000

Command run: ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg ansible-playb…

2022-02-02 10:35:51 +0000

Command STDOUT: PLAY [all] *****************************************************…

2022-02-02 10:35:51 +0000

Command STDERR: [WARNING]: Skipping callback plugin ‘’, unable to load

2022-02-02 10:35:51 +0000

Command FAILED (code=2): ANSIBLE_CONFIG=/tmp/d20220202-8551-wilnbj/ansible.cfg ansible-playbook --ssh-common-args=’-o UserKnownHostsFile=/dev/null’ -i /tmp/d20220202-8551-wilnbj/inventory -e @/tmp/d20220202-8551-wilnbj/group_vars.yml /usr/share/one/oneprovision/ansible/aws.yml

ERROR: Failed to configure hosts

18.170.218.85 : TASK[Install libselinux-python3 (Centos8 8)] - non-zero return…

ahuertas · February 2, 2022, 11:14am

Hello @Samu,

The provision is based on CentOS which is EOL. We have switched it to ubuntu2004 so the next OpenNebula release will come with it.

Best,
Álex.

Samu · February 2, 2022, 11:30am

Thanks @ahuertas

How could I make the switch before the next release is publised? I’m guessing I would have to change the provision AMI and adapt the ansible playbook to the new OS

ahuertas · February 7, 2022, 12:22pm

Hello @Samu,

Yes the default AMI needs to be changed, but also another changes are needed, so I would recommend you to wait until the version is out.

Best,
Álex.

Topic		Replies	Views
Multiple errors when creating edge clusters on AWS Community Support oneprovision	10	445	September 15, 2023
Oneprovision failing Community Support	2	523	March 18, 2022
Error monitoring Host : Error executing poll [FIXED] Community Support	1	1720	May 27, 2015
Deploy High Availability on version 5.12.x HA / Federation	8	1344	May 10, 2021
Running into errors, when a Hook is executed HA / Federation	1	659	September 22, 2021

AWS provisions have stopped working

Related Topics