Upgrading onedb database from 4.10 to 4.12 fails

Community Support
1

Hello, i have been upgrading from 4.10.2 to 4.12.0 following the documentation and when i’ve tried to upgrade the sqlite one.db with the command “onedb upgrade -v --sqlite /var/lib/one/one.db” i’ve got the follow error…

Version read:
Shared tables 4.6.0 : OpenNebula 4.10.2 daemon bootstrap
Local tables  4.9.80 : OpenNebula 4.10.2 daemon bootstrap

Sqlite database backup stored in /var/lib/one/one.db_2015-3-11_10:43:59.bck
Use 'onedb restore' or copy the file back to restore the DB.

>>> Running migrators for shared tables
  > Running migrator /usr/lib/one/ruby/onedb/shared/4.6.0_to_4.11.80.rb
  > Done in 0.02s

Database migrated from 4.6.0 to 4.11.80 (OpenNebula 4.11.80) by onedb command.

>>> Running migrators for local tables
  > Running migrator /usr/lib/one/ruby/onedb/local/4.9.80_to_4.10.3.rb
  > Done in 0.00s

  > Running migrator /usr/lib/one/ruby/onedb/local/4.10.3_to_4.11.80.rb

ArgumentError: invalid byte sequence in US-ASCII
/usr/lib/ruby/vendor_ruby/sqlite3/database.rb:48:in `gsub'
/usr/lib/ruby/vendor_ruby/sqlite3/database.rb:48:in `quote'
/usr/lib/ruby/vendor_ruby/sequel/adapters/sqlite.rb:398:in `literal_string_append'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:86:in `literal_append'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:881:in `block in expression_list_append'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:879:in `each'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:879:in `expression_list_append'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:309:in `array_sql_append'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:1029:in `literal_array_append'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:105:in `literal_append'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:991:in `insert_values_sql'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:819:in `block in clause_sql'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:819:in `each'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:819:in `clause_sql'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:723:in `_insert_sql'
/usr/lib/ruby/vendor_ruby/sequel/dataset/sql.rb:62:in `insert_sql'
/usr/lib/ruby/vendor_ruby/sequel/dataset/actions.rb:334:in `insert'
/usr/lib/one/ruby/onedb/local/4.10.3_to_4.11.80.rb:86:in `block (2 levels) in up'
/usr/lib/ruby/vendor_ruby/sequel/dataset/actions.rb:144:in `block in each'
/usr/lib/ruby/vendor_ruby/sequel/adapters/sqlite.rb:371:in `block (2 levels) in fetch_rows'
/usr/lib/ruby/vendor_ruby/sqlite3/resultset.rb:138:in `each'
/usr/lib/ruby/vendor_ruby/sequel/adapters/sqlite.rb:362:in `block in fetch_rows'
/usr/lib/ruby/vendor_ruby/sqlite3/database.rb:266:in `query'
/usr/lib/ruby/vendor_ruby/sequel/adapters/sqlite.rb:179:in `block (2 levels) in _execute'
/usr/lib/ruby/vendor_ruby/sequel/database/logging.rb:33:in `log_yield'
/usr/lib/ruby/vendor_ruby/sequel/adapters/sqlite.rb:179:in `block in _execute'
/usr/lib/ruby/vendor_ruby/sequel/database/connecting.rb:229:in `block in synchronize'
/usr/lib/ruby/vendor_ruby/sequel/connection_pool/threaded.rb:91:in `hold'
/usr/lib/ruby/vendor_ruby/sequel/database/connecting.rb:229:in `synchronize'
/usr/lib/ruby/vendor_ruby/sequel/adapters/sqlite.rb:172:in `_execute'
/usr/lib/ruby/vendor_ruby/sequel/adapters/sqlite.rb:122:in `execute'
/usr/lib/ruby/vendor_ruby/sequel/dataset/actions.rb:794:in `execute'
/usr/lib/ruby/vendor_ruby/sequel/adapters/sqlite.rb:356:in `fetch_rows'
/usr/lib/ruby/vendor_ruby/sequel/dataset/actions.rb:144:in `each'
/usr/lib/ruby/vendor_ruby/sequel/database/dataset.rb:46:in `fetch'
/usr/lib/one/ruby/onedb/local/4.10.3_to_4.11.80.rb:76:in `block in up'
/usr/lib/ruby/vendor_ruby/sequel/database/transactions.rb:118:in `_transaction'
/usr/lib/ruby/vendor_ruby/sequel/database/transactions.rb:93:in `block in transaction'
/usr/lib/ruby/vendor_ruby/sequel/database/connecting.rb:229:in `block in synchronize'
/usr/lib/ruby/vendor_ruby/sequel/connection_pool/threaded.rb:104:in `hold'
/usr/lib/ruby/vendor_ruby/sequel/database/connecting.rb:229:in `synchronize'
/usr/lib/ruby/vendor_ruby/sequel/database/transactions.rb:86:in `transaction'
/usr/lib/one/ruby/onedb/local/4.10.3_to_4.11.80.rb:75:in `up'
/usr/lib/one/ruby/onedb/onedb.rb:230:in `apply_migrators'
/usr/lib/one/ruby/onedb/onedb.rb:177:in `upgrade'
/usr/bin/onedb:291:in `block (2 levels) in <main>'
/usr/lib/one/ruby/cli/command_parser.rb:449:in `call'
/usr/lib/one/ruby/cli/command_parser.rb:449:in `run'
/usr/lib/one/ruby/cli/command_parser.rb:76:in `initialize'
/usr/bin/onedb:200:in `new'
/usr/bin/onedb:200:in `<main>'


The database will be restored
Sqlite database backup restored in /var/lib/one/one.db
```

Any idea?
2

Hi,

What ruby version are you using? Do you know if you have any network name with non-ascii characters?

3

Ubuntu 14.04

root@Carter ~ # ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]

I’ve check the sqlite db

sqlite> select * from network_pool;
0|Public|<VNET><ID>0</ID><UID>0</UID><GID>0</GID><UNAME>oneadmin</UNAME><GNAME>oneadmin</GNAME><NAME>Public</NAME><PERMISSIONS><OWNER_U>1</OWNER_U><OWNER_M>1</OWNER_M><OWNER_A>0</OWNER_A><GROUP_U>0</GROUP_U><GROUP_M>0</GROUP_M><GROUP_A>0</GROUP_A><OTHER_U>0</OTHER_U><OTHER_M>0</OTHER_M><OTHER_A>0</OTHER_A></PERMISSIONS><CLUSTER_ID>-1</CLUSTER_ID><CLUSTER></CLUSTER><BRIDGE>ovsbr0</BRIDGE><VLAN>0</VLAN><PARENT_NETWORK_ID/><PHYDEV/><VLAN_ID/><USED_LEASES>0</USED_LEASES><TEMPLATE><BRIDGE><![CDATA[ovsbr0]]></BRIDGE><DESCRIPTION><![CDATA[Red Pública]]></DESCRIPTION><DNS><![CDATA[8.8.8.8]]></DNS><GATEWAY><![CDATA[172.16.0.1]]></GATEWAY><NETWORK_ADDRESS><![CDATA[172.16.0.0]]></NETWORK_ADDRESS><NETWORK_MASK><![CDATA[255.255.255.0]]></NETWORK_MASK><PHYDEV><![CDATA[]]></PHYDEV><VLAN><![CDATA[NO]]></VLAN><VLAN_ID><![CDATA[]]></VLAN_ID></TEMPLATE><AR_POOL><AR><ALLOCATED/><AR_ID><![CDATA[0]]></AR_ID><IP><![CDATA[172.16.0.2]]></IP><MAC><![CDATA[02:00:ac:10:00:02]]></MAC><SIZE><![CDATA[250]]></SIZE><TYPE><![CDATA[IP4]]></TYPE></AR></AR_POOL></VNET>|0|0|1|0|0|-1|-1
1|Corp|<VNET><ID>1</ID><UID>0</UID><GID>0</GID><UNAME>oneadmin</UNAME><GNAME>oneadmin</GNAME><NAME>Corp</NAME><PERMISSIONS><OWNER_U>1</OWNER_U><OWNER_M>1</OWNER_M><OWNER_A>0</OWNER_A><GROUP_U>0</GROUP_U><GROUP_M>0</GROUP_M><GROUP_A>0</GROUP_A><OTHER_U>0</OTHER_U><OTHER_M>0</OTHER_M><OTHER_A>0</OTHER_A></PERMISSIONS><CLUSTER_ID>-1</CLUSTER_ID><CLUSTER></CLUSTER><BRIDGE>ovsbr0</BRIDGE><VLAN>1</VLAN><PARENT_NETWORK_ID/><PHYDEV/><VLAN_ID/><USED_LEASES>0</USED_LEASES><TEMPLATE><BRIDGE><![CDATA[ovsbr0]]></BRIDGE><DESCRIPTION><![CDATA[Red Corporativa]]></DESCRIPTION><PHYDEV><![CDATA[]]></PHYDEV><VLAN><![CDATA[YES]]></VLAN><VLAN_ID><![CDATA[]]></VLAN_ID></TEMPLATE><AR_POOL><AR><ALLOCATED/><AR_ID><![CDATA[0]]></AR_ID><IP><![CDATA[172.16.1.1]]></IP><MAC><![CDATA[02:00:ac:10:01:01]]></MAC><SIZE><![CDATA[254]]></SIZE><TYPE><![CDATA[IP4]]></TYPE></AR></AR_POOL></VNET>|0|0|1|0|0|-1|-1
```

Can be the accent in "Pública"?
4

That should help us to reproduce it. I’ll post the fix here as soon as it’s ready.

Thank you.

5

Hi,

That’s a bug, there are some encoding problems with ruby 1.9. Can you replace your file /usr/lib/one/ruby/onedb/local/4.9.80_to_4.10.3.rb with this, and try again?

Thanks.

6

But the original file has this:

module Migrator
    def db_version
        "4.10.3"
    end

    def one_version
        "OpenNebula 4.10.3"
    end

    def up

        init_log_time()

        @db.run "CREATE TABLE IF NOT EXISTS vm_import (deploy_id VARCHAR(128), vmid INTEGER, PRIMARY KEY(deploy_id));"

        log_time()

        return true
    end
end

And the new one doesn’t contains the “CREATE TABLE IF NOT EXISTS vm_import…”, is this ok?

7

My bad, sorry! It should be the file /usr/lib/one/ruby/onedb/local/4.10.3_to_4.11.80.rb

8

It works, will be included this change in the next revision?
If i reinstall from scratch the 4.12 version i will find this bug in the future?

9

We will post a workaround in the docs for now, and a more robust solution will be included in future versions. Probably in a 4.12.1 release.

Thanks for reporting it.