# Ceph error monitoring datastore **URL:** \ **Category:** Product Support\ **Created:** [May 26, 2016, 10:37am UTC](https://forum.opennebula.io/t/ceph-error-monitoring-datastore/2295 "2016-05-26T10:37:55Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![UAnton](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/uanton/32/6309_2.png) [@UAnton](https://forum.opennebula.io/u/UAnton)\ **Post date:** [May 26, 2016, 10:37am UTC](https://forum.opennebula.io/t/ceph-error-monitoring-datastore/2295/1 "2016-05-26T10:37:55Z")
Hi! I have ceph cluster and want use it with opennebula but every time have problem with ceph + opennebula. Opennebula don’t see ceph storage. In log I have: ``` Command execution fail: /var/lib/one/remotes/datastore/ceph/monitor PERTX0RSSVZFUl9BQ1RJT05fREFUQT48REFUQVNUT1JFPjxJRD4xMDA8L0lEPjxVSUQ+MDwvVUlEPjxHSUQ+MD wvR0lEPjxVTkFNRT5vbmVhZG1pbjwvVU5BTUU+PEdOQU1FPm9uZWFkbWluPC9HTkFNRT48TkFNRT5jZXBoZHM8L05BTUU+PFBFUk1JU1NJT05TPjxPV05FUl9VPjE8L09XTkVSX1U+PE9XTkVSX00+MTwvT1dORVJfTT48T1dORVJfQT4wPC9PV05FUl9BPjx HUk9VUF9VPjE8L0dST1VQX1U+PEdST1VQX00+MDwvR1JPVVBfTT48R1JPVVBfQT4wPC9HUk9VUF9BPjxPVEhFUl9VPjA8L09USEVSX1U+PE9USEVSX00+MDwvT1RIRVJfTT48T1RIRVJfQT4wPC9PVEhFUl9BPjwvUEVSTUlTU0lPTlM+PERTX01BRD48IVtD REFUQVtjZXBoXV0+PC9EU19NQUQ+PFRNX01BRD48IVtDREFUQVtjZXBoXV0+PC9UTV9NQUQ+PEJBU0VfUEFUSD48IVtDREFUQVsvdmFyL2xpYi9vbmUvL2RhdGFzdG9yZXMvMTAwXV0+PC9CQVNFX1BBVEg+PFRZUEU+MDwvVFlQRT48RElTS19UWVBFPjM8L 0RJU0tfVFlQRT48U1RBVEU+MDwvU1RBVEU+PENMVVNURVJTPjxJRD4wPC9JRD48L0NMVVNURVJTPjxUT1RBTF9NQj4wPC9UT1RBTF9NQj48RlJFRV9NQj4wPC9GUkVFX01CPjxVU0VEX01CPjA8L1VTRURfTUI+PElNQUdFUz48L0lNQUdFUz48VEVNUExBVE U+PEJSSURHRV9MSVNUPjwhW0NEQVRBWzE3Mi4xNi4xMC4yMTEgMTcyLjE2LjEwLjIxMl1dPjwvQlJJREdFX0xJU1Q+PENFUEhfSE9TVD48IVtDREFUQVsxNzIuMTYuMTAuMjExOjY3ODkgMTcyLjE2LjEwLjIxMjo2Nzg5XV0+PC9DRVBIX0hPU1Q+PENFUEh fU0VDUkVUPjwhW0NEQVRBW2UwNzAwZDUwLWIzMzgtNDc1OC1hNTk3LTg5NWU0MTY3Y2I2Yl1dPjwvQ0VQSF9TRUNSRVQ+PENFUEhfVVNFUj48IVtDREFUQVtvbmVhZG1pbl1dPjwvQ0VQSF9VU0VSPjxDTE9ORV9UQVJHRVQ+PCFbQ0RBVEFbU0VMRl1dPjwv Q0xPTkVfVEFSR0VUPjxESVNLX1RZUEU+PCFbQ0RBVEFbUkJEXV0+PC9ESVNLX1RZUEU+PERTX01BRD48IVtDREFUQVtjZXBoXV0+PC9EU19NQUQ+PExOX1RBUkdFVD48IVtDREFUQVtOT05FXV0+PC9MTl9UQVJHRVQ+PFBPT0xfTkFNRT48IVtDREFUQVtvb mVdXT48L1BPT0xfTkFNRT48VE1fTUFEPjwhW0NEQVRBW2NlcGhdXT48L1RNX01BRD48L1RFTVBMQVRFPjwvREFUQVNUT1JFPjwvRFNfRFJJVkVSX0FDVElPTl9EQVRBPg== 100 Thu May 26 13:34:05 2016 [Z0][ImM][I]: ExitCode: 255 Thu May 26 13:34:05 2016 [Z0][ImM][E]: Error monitoring datastore 100: - Thu May 26 13:34:05 2016 [Z0][ImM][I]: Command execution fail: /var/lib/one/remotes/datastore/ceph/monitor PERTX0RSSVZFUl9BQ1RJT05fREFUQT48REFUQVNUT1JFPjxJRD4xMDE8L0lEPjxVSUQ+MDwvVUlEPjxHSUQ+MD wvR0lEPjxVTkFNRT5vbmVhZG1pbjwvVU5BTUU+PEdOQU1FPm9uZWFkbWluPC9HTkFNRT48TkFNRT5jZXBoZHMxPC9OQU1FPjxQRVJNSVNTSU9OUz48T1dORVJfVT4xPC9PV05FUl9VPjxPV05FUl9NPjE8L09XTkVSX00+PE9XTkVSX0E+MDwvT1dORVJfQT4 8R1JPVVBfVT4xPC9HUk9VUF9VPjxHUk9VUF9NPjA8L0dST1VQX00+PEdST1VQX0E+MDwvR1JPVVBfQT48T1RIRVJfVT4wPC9PVEhFUl9VPjxPVEhFUl9NPjA8L09USEVSX00+PE9USEVSX0E+MDwvT1RIRVJfQT48L1BFUk1JU1NJT05TPjxEU19NQUQ+PCFb Q0RBVEFbY2VwaF1dPjwvRFNfTUFEPjxUTV9NQUQ+PCFbQ0RBVEFbY2VwaF1dPjwvVE1fTUFEPjxCQVNFX1BBVEg+PCFbQ0RBVEFbL3Zhci9saWIvb25lLy9kYXRhc3RvcmVzLzEwMV1dPjwvQkFTRV9QQVRIPjxUWVBFPjA8L1RZUEU+PERJU0tfVFlQRT4zP C9ESVNLX1RZUEU+PFNUQVRFPjA8L1NUQVRFPjxDTFVTVEVSUz48SUQ+MDwvSUQ+PC9DTFVTVEVSUz48VE9UQUxfTUI+MDwvVE9UQUxfTUI+PEZSRUVfTUI+MDwvRlJFRV9NQj48VVNFRF9NQj4wPC9VU0VEX01CPjxJTUFHRVM+PC9JTUFHRVM+PFRFTVBMQV RFPjxCUklER0VfTElTVD48IVtDREFUQVsxNzIuMTYuMTAuMjExIDE3Mi4xNi4xMC4yMTJdXT48L0JSSURHRV9MSVNUPjxDRVBIX0hPU1Q+PCFbQ0RBVEFbMTcyLjE2LjEwLjIxMTo2Nzg5IDE3Mi4xNi4xMC4yMTI6Njc4OV1dPjwvQ0VQSF9IT1NUPjxDRVB IX1NFQ1JFVD48IVtDREFUQVtlMDcwMGQ1MC1iMzM4LTQ3NTgtYTU5Ny04OTVlNDE2N2NiNmJdXT48L0NFUEhfU0VDUkVUPjxDRVBIX1VTRVI+PCFbQ0RBVEFbbGlidmlydF1dPjwvQ0VQSF9VU0VSPjxDTE9ORV9UQVJHRVQ+PCFbQ0RBVEFbU0VMRl1dPjwv Q0xPTkVfVEFSR0VUPjxESVNLX1RZUEU+PCFbQ0RBVEFbUkJEXV0+PC9ESVNLX1RZUEU+PERTX01BRD48IVtDREFUQVtjZXBoXV0+PC9EU19NQUQ+PExOX1RBUkdFVD48IVtDREFUQVtOT05FXV0+PC9MTl9UQVJHRVQ+PFBPT0xfTkFNRT48IVtDREFUQVtvb mVdXT48L1BPT0xfTkFNRT48VE1fTUFEPjwhW0NEQVRBW2NlcGhdXT48L1RNX01BRD48VFlQRT48IVtDREFUQVtJTUFHRV9EU11dPjwvVFlQRT48L1RFTVBMQVRFPjwvREFUQVNUT1JFPjwvRFNfRFJJVkVSX0FDVElPTl9EQVRBPg== 101 Thu May 26 13:34:05 2016 [Z0][ImM][I]: ExitCode: 255 Thu May 26 13:34:05 2016 [Z0][ImM][E]: Error monitoring datastore 101: - ``` [root@node01 one]# onedatastore list ID NAME SIZE AVAIL CLUSTERS IMAGES TYPE DS TM STAT 0 system 50G 95% 0 0 sys - shared on 1 default 50G 95% 0 0 img fs shared on 2 files 50G 95% 0 0 fil fs ssh on 100 cephds 0M - 0 0 img ceph ceph on 101 cephds1 0M - 0 0 img ceph ceph on What I do wrong? Use manual from [http://docs.opennebula.org/5.0/deployment/open\_cloud\_storage\_setup/ceph\_ds.html](http://docs.opennebula.org/5.0/deployment/open_cloud_storage_setup/ceph_ds.html) ---
**Author:** ![UAnton](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/uanton/32/6309_2.png) [@UAnton](https://forum.opennebula.io/u/UAnton)\ **Post date:** [May 28, 2016, 8:51pm UTC](https://forum.opennebula.io/t/ceph-error-monitoring-datastore/2295/2 "2016-05-28T20:51:55Z")
I try fresh install 3 times. Used ver. 4.14.2 and 4.90 but every time I got error: > Sat May 28 23:48:25 2016 [Z0][ImM][I]: /var/lib/one/remotes/tm/ceph/…/…/datastore/libfs.sh: line 252: RANDOM % 0: division by 0 (error token is “0”) > Sat May 28 23:48:25 2016 [Z0][ImM][E]: Datastore template missing ‘BRIDGE\_LIST’ attribute. > Sat May 28 23:48:25 2016 [Z0][ImM][I]: ExitCode: 255 > Sat May 28 23:48:25 2016 [Z0][ImM][E]: Error monitoring datastore 100: Datastore template missing ‘BRIDGE\_LIST’ attribute. But I have BRIDGE\_LIST! Please help me. Thanks! ---
**Author:** ![jfontan](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/jfontan/32/18_2.png) [@jfontan](https://forum.opennebula.io/u/jfontan)\ **Post date:** [May 28, 2016, 9:34pm UTC](https://forum.opennebula.io/t/ceph-error-monitoring-datastore/2295/3 "2016-05-28T21:34:59Z")
Can you send the output of: ```auto $ onedatastore show 100 ``` ---
**Author:** ![UAnton](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/uanton/32/6309_2.png) [@UAnton](https://forum.opennebula.io/u/UAnton)\ **Post date:** [May 29, 2016, 4:59am UTC](https://forum.opennebula.io/t/ceph-error-monitoring-datastore/2295/4 "2016-05-29T04:59:23Z")
> root@node01:/var/log/one# onedatastore show 103 > DATASTORE 103 INFORMATION > ID : 103 > NAME : cephds > USER : oneadmin > GROUP : oneadmin > CLUSTERS : 0 > TYPE : IMAGE > DS\_MAD : ceph > TM\_MAD : ceph > BASE PATH : /var/lib/one//datastores/103 > DISK\_TYPE : RBD > STATE : READY > DATASTORE CAPACITY > TOTAL: : 0M > FREE: : 0M > USED: : 0M > LIMIT: : - > PERMISSIONS > OWNER : um- > GROUP : u– > OTHER : — > DATASTORE TEMPLATE > BRIDGE\_LIST=“172.16.10.211 172.16.10.212” > CEPH\_HOST=“172.16.10.211 172.16.10.212” > CEPH\_SECRET=“c1e33b0a-860a-4230-b19b-368eebbb87f4” > CEPH\_USER=“libvirt” > CLONE\_TARGET=“SELF” > DISK\_TYPE=“RBD” > DS\_MAD=“ceph” > LN\_TARGET=“NONE” > POOL\_NAME=“one” > RESTRICTED\_DIRS=“/” > SAFE\_DIRS=“/var/tmp” > TM\_MAD=“ceph” > IMAGES ---
**Author:** ![anandharaj](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@anandharaj](https://forum.opennebula.io/u/anandharaj)\ **Post date:** [May 31, 2016, 6:44am UTC](https://forum.opennebula.io/t/ceph-error-monitoring-datastore/2295/5 "2016-05-31T06:44:06Z")
Below are my comments based on your setup / configuration: 1. I dont recommend using Ceph’s MON server for bridge list also. In my setup, i just use the few of the ONE’s node 2. Does the ceph authentication has been setup properly? Please verify you can run the following command: > rbd --id libvirt ls -l -p one You may refer [my guide](http://www.raj2u.net/opennebula-frontend-with-ceph-datastore/) also and see if it help resolve the issue. ---
**Author:** ![UAnton](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/uanton/32/6309_2.png) [@UAnton](https://forum.opennebula.io/u/UAnton)\ **Post date:** [May 31, 2016, 12:21pm UTC](https://forum.opennebula.io/t/ceph-error-monitoring-datastore/2295/6 "2016-05-31T12:21:15Z")
Hi! I read you guide. Fresh install - and all ok!! ))) Thanks! But I can’t understand how use ceph as systemDS. ImageDS work good. Thank you very much! ---
**Author:** ![anandharaj](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@anandharaj](https://forum.opennebula.io/u/anandharaj)\ **Post date:** [June 1, 2016, 2:15am UTC](https://forum.opennebula.io/t/ceph-error-monitoring-datastore/2295/7 "2016-06-01T02:15:38Z")
Great to hear the problem is solved. For System DS, you may need to use NFS or CephFS. But if your ONE is 5.0 beta, you may try System DS on ceph directly - this i have not tested