# Incorrect ceph datastore capacity in sunstone

**URL:** <https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165>\
**Category:** Operations\
**Created:** [April 23, 2019, 3:09pm UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165 "2019-04-23T15:09:20Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![nalexandrov](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/nalexandrov/32/6391_2.png) [@nalexandrov](https://forum.opennebula.io/u/nalexandrov)\
**Post date:** [April 23, 2019, 3:09pm UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165/1 "2019-04-23T15:09:20Z")

</div>

Hi there,  
After many searches I can’t figure out why our ONE(5.6.1) GUI is displaying different capacity from that in Ceph(13.2.5) #ceph df  
 ![DS2](https://canada1.discourse-cdn.com/flex031/uploads/opennebula/original/2X/6/66772c9f5037452bcb3aedcb726acc51e81cac13.png)

 ![DS1](https://canada1.discourse-cdn.com/flex031/uploads/opennebula/original/2X/5/54425f43d6116772f84c1d99cd0867ce43230c22.png)  
Ceph is set with 2 replicas, so should it be 19TB max size (not 15.9TB) in sunstone ?  
BR,  
N.Alexandrov

---

<div class="post-metadata">

**Author:** ![kvanals](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/kvanals/32/5967_2.png) [@kvanals](https://forum.opennebula.io/u/kvanals)\
**Post date:** [April 23, 2019, 8:45pm UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165/2 "2019-04-23T20:45:31Z")

</div>

I believe this is probably related to the “Max Avail” in the “one” pool. The capacity shown will be what you can actually use in that pool, not what the Ceph cluster actually has available. To increase max avail, be sure to check that OSDs are balanced. Check out the “ceph osd reweight-by-utilization” command.

- Kenny

---

<div class="post-metadata">

**Author:** ![foysal](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/foysal/32/6112_2.png) [@foysal](https://forum.opennebula.io/u/foysal)\
**Post date:** [June 25, 2019, 8:57am UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165/3 "2019-06-25T08:57:13Z")

</div>

HI Nalexandrov,

I have a query to you which is not directly related to your post but hope you can help. We are tring to integrate ceph with opennebula and already have a working ceph cluster. But when we add ceph datastore in front end it shows no size of the ceph pool and oned.log shows unable to monitor the DS with following trace.  
"Command execution failed (exit code: 255): /var/lib/one/remotes/datastore/ceph/monitor PERTX0RSSVZFU… a big hash string.

Please be note that we are able to access the ceph pool using the user authentication we used in front-end and ceph DS.

Regards  
Foysal

---

<div class="post-metadata">

**Author:** ![nalexandrov](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/nalexandrov/32/6391_2.png) [@nalexandrov](https://forum.opennebula.io/u/nalexandrov)\
**Post date:** [June 25, 2019, 11:05am UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165/4 "2019-06-25T11:05:54Z")

</div>

Hi,  
What versions are your ONE and CEPH?  
I’ve got these lines in datastore configuration section of /etc/one/oned.conf:  
DATASTORE\_LOCATION = /var/lib/one/datastores  
DATASTORE\_CAPACITY\_CHECK = “yes”  
There are the docs for One5.6:  
[http://docs.opennebula.org/5.6/deployment/open\_cloud\_storage\_setup/ceph\_ds.html](http://docs.opennebula.org/5.6/deployment/open_cloud_storage_setup/ceph_ds.html)

---

<div class="post-metadata">

**Author:** ![IowaOrganics](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/iowaorganics/32/5974_2.png) [@IowaOrganics](https://forum.opennebula.io/u/IowaOrganics)\
**Post date:** [June 25, 2019, 1:16pm UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165/6 "2019-06-25T13:16:37Z")

</div>

The hash string is the command that failed to execute. I recommend you to copy the entire string starting with the file path through the end of the hash (include those end trailing numbers I think) and push enter. You will actually get the error that the command throws when you execute it.

---

<div class="post-metadata">

**Author:** ![foysal](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/foysal/32/6112_2.png) [@foysal](https://forum.opennebula.io/u/foysal)\
**Post date:** [June 25, 2019, 3:35pm UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165/7 "2019-06-25T15:35:30Z")

</div>

Thanks for your suggestion. We are getting below error after running the command manually.

ERROR: monitor: Command “ceph --id libvirt --keyfile /etc/ceph/ceph.client.libvirt.keyring df detail --format xml” failed:

---

<div class="post-metadata">

**Author:** ![IowaOrganics](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/iowaorganics/32/5974_2.png) [@IowaOrganics](https://forum.opennebula.io/u/IowaOrganics)\
**Post date:** [June 25, 2019, 3:52pm UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165/8 "2019-06-25T15:52:42Z")

</div>

It appears your user is failing to authenticate.  
Are you able to execute the following command from your frontend?

> su oneadmin -c ‘rbd ls -p yourpoolname’

If not, please step through [Ceph Datastore — OpenNebula 5.8.5 documentation](https://docs.opennebula.org/5.8/deployment/open_cloud_storage_setup/ceph_ds.html#frontend-and-node-setup)

Make sure your ceph keys are owned by oneadmin in /var/lib/one

---

<div class="post-metadata">

**Author:** ![FoZo](https://avatars.discourse-cdn.com/v4/letter/f/53a042/32.png) [@FoZo](https://forum.opennebula.io/u/FoZo)\
**Post date:** [October 21, 2019, 2:57pm UTC](https://forum.opennebula.io/t/incorrect-ceph-datastore-capacity-in-sunstone/7165/9 "2019-10-21T14:57:44Z")

</div>

Hello,  
I have the same problem.  
Long time ago I added some extra disks to the ceph cluster and the cluster size was extended, but the “TOTAL” value inside opennebula is not updated¸ or most likely the information is misleading.

Used is taken from the ceph’s value for “used”, but the value for the OpenNebula’s total is coming from ceph’s “used” plus “max avail”.  
OpenNebula should use the value from “Stored” instead of the one in “Used”.

[oneadmin@OpenNebula ~]$ onedatastore show 999  
DATASTORE CAPACITY  
TOTAL: : 47.6T  
FREE: : 8.5T  
USED: : 39.1T  
LIMIT: : -

[oneadmin@OpenNebula ~]$ ceph --user libvirt df  
RAW STORAGE:  
CLASS SIZE AVAIL USED RAW USED %RAW USED  
hdd 71 TiB 32 TiB 39 TiB 39 TiB 55.30  
TOTAL 71 TiB 32 TiB 39 TiB 39 TiB 55.30

POOLS:  
POOL ID STORED OBJECTS USED %USED MAX AVAIL  
.rgw.root 1 3.2 KiB 8 1.5 MiB 0 8.5 TiB  
default.rgw.control 2 0 B 8 0 B 0 8.5 TiB  
default.rgw.meta 3 0 B 0 0 B 0 8.5 TiB  
default.rgw.log 4 0 B 207 0 B 0 8.5 TiB  
one 5 13 TiB 4.02M 39 TiB 60.53 8.5 TiB

OpenNebula 5.8.5  
ceph version 14.2.4 nautilus (stable)
