You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Indra Pramana (JIRA)" <ji...@apache.org> on 2013/07/17 18:40:49 UTC
[jira] [Commented] (CLOUDSTACK-3542) Ceph RBD provides wrong usage
info to CloudStack
[ https://issues.apache.org/jira/browse/CLOUDSTACK-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13711255#comment-13711255 ]
Indra Pramana commented on CLOUDSTACK-3542:
-------------------------------------------
I managed to resolve the problem by installing latest version of libvirt. Apparently, the libvirt version that I was using (1.0.2), which I got from here:
http://blog.widodh.nl/2013/06/a-quick-note-on-running-cloudstack-with-rbd-on-ubuntu-12-04/
was the culprit. I downloaded the latest libvirt version (version 1.1.0) from their FTP site:
ftp://libvirt.org/libvirt/libvirt-1.1.0.tar.gz
Install the latest librbd-dev package:
apt-get install librbd-dev
Then compile libvirt with storage RBD enabled:
./autogen.sh --prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --with-storage-rbd
make
make install
Restart the KVM host after it's done.
virsh pool-info is now showing the correct "Allocation" amount:
root@hv-kvm-02:~# virsh pool-info d433809b-01ea-3947-ba0f-48077244e4d6
Name: d433809b-01ea-3947-ba0f-48077244e4d6
UUID: d433809b-01ea-3947-ba0f-48077244e4d6
State: running
Persistent: no
Autostart: no
Capacity: 5.47 TiB
Allocation: 328.00 B
Available: 5.47 TiB
Tried to create a VM instance, and now the VM can utilise the RBD storage pool. No more "insufficient space on pool" error message on the log.
> Ceph RBD provides wrong usage info to CloudStack
> ------------------------------------------------
>
> Key: CLOUDSTACK-3542
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3542
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Storage Controller
> Affects Versions: 4.1.0
> Environment: Ubuntu 12.04.2
> Reporter: Indra Pramana
> Labels: ceph, rbd
>
> I am installing CloudStack 4.1.0 to use Ceph RBD as primary storage. I failed to create VM instance using the RBD as primary storage because Ceph provides wrong disk usage information. See below:
> root@hv-kvm-02:~# virsh pool-info bab81ce8-d53f-3a7d-b8f6-841702f65c89
> Name: bab81ce8-d53f-3a7d-b8f6-841702f65c89
> UUID: bab81ce8-d53f-3a7d-b8f6-841702f65c89
> State: running
> Persistent: no
> Autostart: no
> Capacity: 5.47 TiB
> Allocation: 34819.02 TiB <-- wrong information
> Available: 5.47 TiB
> Record on the "storage_pool" table on "cloud" database:
> +-----+-----------------+--------------------------------------+-------------------+------+----------------+--------+------------+-------------------+----------------+---------------------------+------------------------------------------------+------------------------+---------------------+---------------------+-------------+-------------+---------------------+-------+
> | id | name | uuid | pool_type | port | data_center_id | pod_id | cluster_id | available_bytes | capacity_bytes | host_address | user_info | path | created | removed | update_time | status | storage_provider_id | scope |
> +-----+-----------------+--------------------------------------+-------------------+------+----------------+--------+------------+-------------------+----------------+---------------------------+------------------------------------------------+------------------------+---------------------+---------------------+-------------+-------------+---------------------+-------+
> | 209 | sc-image | bab81ce8-d53f-3a7d-b8f6-841702f65c89 | RBD | 6789 | 6 | 6 | 6 | 38283921137336466 | 6013522722816 | ceph-mon.xxx.com | admin:xxx | sc1 | 2013-07-13 08:58:27 | NULL | NULL | Up | NULL | NULL |
> ===
> 2013-07-15 11:27:31,632 DEBUG [cloud.alert.AlertManagerImpl] (CapacityChecker:null) System Alert: Low Available Storage in cluster Cluster-01 pod Pod-01 of availability zone 01
> 2013-07-15 11:27:31,632 DEBUG [cloud.alert.AlertManagerImpl] (CapacityChecker:null) Available storage space is low, total: 13282342 MB, used: 36510398411 MB (274879.22%)
> ===
> As a result, VM instance creation fails since it's not able to find available storage pool.
> ===
> 2013-07-15 11:15:28,313 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Checking pool: 208 for volume allocation [Vol[227|vm=225|ROOT]], maxSize : 15828044742656, totalAllocatedSize : 1769538048, askingSize : 8589934592, allocated disable threshold: 0.85
> 2013-07-15 11:15:28,313 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Checking if storage pool is suitable, name: sc-image ,poolId: 209
> 2013-07-15 11:15:28,313 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Is localStorageAllocationNeeded? false
> 2013-07-15 11:15:28,313 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Is storage pool shared? true
> 2013-07-15 11:15:28,317 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Checking pool 209 for storage, totalSize: 6013522722816, usedBytes: 38283921137336466, usedPct: 6366.305226067051, disable threshold: 0.85
> 2013-07-15 11:15:28,317 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Insufficient space on pool: 209 since its usage percentage: 6366.305226067051 has crossed the pool.storage.capacity.disablethreshold: 0.85
> 2013-07-15 11:15:28,317 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-3:job-168) FirstFitStoragePoolAllocator returning 1 suitable storage pools
> 2013-07-15 11:15:28,317 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) Checking suitable pools for volume (Id, Type): (228,DATADISK)
> 2013-07-15 11:15:28,317 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) We need to allocate new storagepool for this volume
> 2013-07-15 11:15:28,319 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) Calling StoragePoolAllocators to find suitable pools
> 2013-07-15 11:15:28,319 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-3:job-168) Looking for pools in dc: 6 pod:6 cluster:6 having tags:[rbd]
> 2013-07-15 11:15:28,322 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-3:job-168) FirstFitStoragePoolAllocator has 1 pools to check for allocation
> 2013-07-15 11:15:28,322 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Checking if storage pool is suitable, name: sc-image ,poolId: 209
> 2013-07-15 11:15:28,322 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Is localStorageAllocationNeeded? false
> 2013-07-15 11:15:28,322 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-3:job-168) Is storage pool shared? true
> 2013-07-15 11:15:28,326 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Checking pool 209 for storage, totalSize: 6013522722816, usedBytes: 38283921137336466, usedPct: 6366.305226067051, disable threshold: 0.85
> 2013-07-15 11:15:28,326 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-3:job-168) Insufficient space on pool: 209 since its usage percentage: 6366.305226067051 has crossed the pool.storage.capacity.disablethreshold: 0.85
> 2013-07-15 11:15:28,326 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-3:job-168) FirstFitStoragePoolAllocator returning 0 suitable storage pools
> 2013-07-15 11:15:28,326 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) No suitable pools found for volume: Vol[228|vm=225|DATADISK] under cluster: 6
> 2013-07-15 11:15:28,326 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) No suitable pools found
> 2013-07-15 11:15:28,326 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) No suitable storagePools found under this Cluster: 6
> 2013-07-15 11:15:28,326 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-3:job-168) Could not find suitable Deployment Destination for this VM under any clusters, returning.
> 2013-07-15 11:15:28,332 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-3:job-168) Destroying vm VM[User|Indra-Test-3] as it failed to create on Host with Id:null
> 2013-07-15 11:15:28,498 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-3:job-168) VM state transitted from :Stopped to Error with event: OperationFailedToErrorvm's original host id: null new host id: null host id before state transition: null
> 2013-07-15 11:15:29,125 INFO [user.vm.DeployVMCmd] (Job-Executor-3:job-168) com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|Indra-Test-3]Scope=interface com.cloud.dc.DataCenter; id=6
> ===
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira