You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2021/07/28 12:35:04 UTC
[GitHub] [cloudstack] ravening opened a new pull request #5249: Global setting to select preferred storage pool
ravening opened a new pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249
### Description
Currently all the volumes are allocated on storage pools
based on the capacity or the algorithm selected. Sometimes
we need to deploy all volumes of particular account in a
specific storage pool and in that case its not possible.
with this change, we can specify the uuid of the preferred
storage pool, so that all volumes of the account will be
deployed in this pool
<!--- Describe your changes in DETAIL - And how has behaviour functionally changed. -->
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
<!--- ********************************************************************************* -->
<!--- NOTE: AUTOMATATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. -->
<!--- PLEASE PUT AN 'X' in only **ONE** box -->
<!--- ********************************************************************************* -->
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [X] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [X] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] ravening commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
ravening commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-889688723
> > > @ravening, one nit from functional perspective; wouldn't a user want to use a storage pool name instead of a uuid?
> >
> >
> > @DaanHoogland This will be configured mostly by admins. Also if you see similar other global settings , they all use uuid's rather than actual name of the resource. For eg: router service offering
>
> Yes but this is a preferred pool for an account. wouldn't the account holders have to/want to configure that themselves? And wouldn't the admin want to not be bothered and leave it to the users?
@DaanHoogland customers wont have access or see the list of storage pools in the platform. So they cant select it anyway. its useful only for admins.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-896465396
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-888974911
> > @ravening, one nit from functional perspective; wouldn't a user want to use a storage pool name instead of a uuid?
>
> @DaanHoogland This will be configured mostly by admins. Also if you see similar other global settings , they all use uuid's rather than actual name of the resource. For eg: router service offering
Yes but this is a preferred pool for an account. wouldn't the account holders have to/want to configure that themselves? And wouldn't the admin want to not be bothered and leave it to the users?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] ravening commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
ravening commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-888868727
> @ravening, one nit from functional perspective; wouldn't a user want to use a storage pool name instead of a uuid?
@DaanHoogland This will be configured mostly by admins. Also if you see similar other global settings , they all use uuid's rather than actual name of the resource. For eg: router service offering
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-895045594
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-895077533
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian. SL-JID 800
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-895044895
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-889978322
> > > > @ravening, one nit from functional perspective; wouldn't a user want to use a storage pool name instead of a uuid?
> > >
> > >
> > > @DaanHoogland This will be configured mostly by admins. Also if you see similar other global settings , they all use uuid's rather than actual name of the resource. For eg: router service offering
> >
> >
> > Yes but this is a preferred pool for an account. wouldn't the account holders have to/want to configure that themselves? And wouldn't the admin want to not be bothered and leave it to the users?
>
> @DaanHoogland customers wont have access or see the list of storage pools in the platform. So they cant select it anyway. its useful only for admins.
too bad
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#discussion_r686121729
##########
File path: engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java
##########
@@ -321,9 +347,13 @@ public StoragePool findStoragePool(DiskProfile dskCh, DataCenter dc, Pod pod, Lo
}
DataCenterDeployment plan = new DataCenterDeployment(dc.getId(), podId, clusterId, hostId, null, null);
- final List<StoragePool> poolList = allocator.allocateToPool(dskCh, profile, plan, avoidList, 1);
+ final List<StoragePool> poolList = allocator.allocateToPool(dskCh, profile, plan, avoidList, StoragePoolAllocator.RETURN_UPTO_ALL);
if (poolList != null && !poolList.isEmpty()) {
- return (StoragePool)dataStoreMgr.getDataStore(poolList.get(0).getId(), DataStoreRole.Primary);
+ // Check if the preferred storage pool can be used. If yes, use it.
+ Optional<StoragePool> storagePool = getPreferredStoragePool(poolList, vm);
+
+ return (storagePool.isPresent()) ? (StoragePool) this.dataStoreMgr.getDataStore(storagePool.get().getId(), DataStoreRole.Primary) :
+ (StoragePool)dataStoreMgr.getDataStore(poolList.get(0).getId(), DataStoreRole.Primary);
Review comment:
We can add the availability to the loop condition and break once found, can't we? I see your point though. Though the principle stands, I'm not making a big issue of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-896465715
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-896120919
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-888325758
@ravening, one nit from functional perspective; wouldn't a user want to use a storage pool name instead of a uuid?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] ravening commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
ravening commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-895194403
> Overall code LGTM, I see that @DaanHoogland made some interesting suggestions. I would appreciate it if you take them into account. But either way, LGTM.
>
> Thanks for the PR @ravening.
> Has this been tested? Manual tests might be interesting in this one with some alternatives for the global settings.
@GabrielBrascher will work on it to make changes.
Yes I tested them manually by passing uuid of particular pool in the global setting. All of the vm deployment, volume attachment to a vm will happen on this pool.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] ravening commented on a change in pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
ravening commented on a change in pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#discussion_r685172349
##########
File path: engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java
##########
@@ -321,9 +347,13 @@ public StoragePool findStoragePool(DiskProfile dskCh, DataCenter dc, Pod pod, Lo
}
DataCenterDeployment plan = new DataCenterDeployment(dc.getId(), podId, clusterId, hostId, null, null);
- final List<StoragePool> poolList = allocator.allocateToPool(dskCh, profile, plan, avoidList, 1);
+ final List<StoragePool> poolList = allocator.allocateToPool(dskCh, profile, plan, avoidList, StoragePoolAllocator.RETURN_UPTO_ALL);
if (poolList != null && !poolList.isEmpty()) {
- return (StoragePool)dataStoreMgr.getDataStore(poolList.get(0).getId(), DataStoreRole.Primary);
+ // Check if the preferred storage pool can be used. If yes, use it.
+ Optional<StoragePool> storagePool = getPreferredStoragePool(poolList, vm);
+
+ return (storagePool.isPresent()) ? (StoragePool) this.dataStoreMgr.getDataStore(storagePool.get().getId(), DataStoreRole.Primary) :
+ (StoragePool)dataStoreMgr.getDataStore(poolList.get(0).getId(), DataStoreRole.Primary);
Review comment:
@DaanHoogland
This code is inside the for loop. So as soon as we found the first suitable pool, we need to return it. If I have just one exit point after the for loop then the first selected might be overwritten by the pool which can be selected later in the for loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez merged pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
nvazquez merged pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#discussion_r679004476
##########
File path: server/src/main/java/com/cloud/deploy/DeploymentPlanningManagerImpl.java
##########
@@ -1674,7 +1675,12 @@ protected boolean hostCanAccessSPool(Host host, StoragePool pool) {
for (StoragePoolAllocator allocator : _storagePoolAllocators) {
final List<StoragePool> suitablePools = allocator.allocateToPool(diskProfile, vmProfile, plan, avoid, returnUpTo);
if (suitablePools != null && !suitablePools.isEmpty()) {
- suitableVolumeStoragePools.put(toBeCreated, suitablePools);
+ List<StoragePool> pools = new ArrayList<>();
+ Optional<StoragePool> storagePool = getPreferredStoragePool(suitablePools, vmProfile.getVirtualMachine());
+ storagePool.ifPresent(pools::add);
+
+ pools.addAll(suitablePools);
+ suitableVolumeStoragePools.put(toBeCreated, pools);
Review comment:
can you extract this bit as to not enlarge an already too long method, please?
##########
File path: engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java
##########
@@ -321,9 +347,13 @@ public StoragePool findStoragePool(DiskProfile dskCh, DataCenter dc, Pod pod, Lo
}
DataCenterDeployment plan = new DataCenterDeployment(dc.getId(), podId, clusterId, hostId, null, null);
- final List<StoragePool> poolList = allocator.allocateToPool(dskCh, profile, plan, avoidList, 1);
+ final List<StoragePool> poolList = allocator.allocateToPool(dskCh, profile, plan, avoidList, StoragePoolAllocator.RETURN_UPTO_ALL);
if (poolList != null && !poolList.isEmpty()) {
- return (StoragePool)dataStoreMgr.getDataStore(poolList.get(0).getId(), DataStoreRole.Primary);
+ // Check if the preferred storage pool can be used. If yes, use it.
+ Optional<StoragePool> storagePool = getPreferredStoragePool(poolList, vm);
+
+ return (storagePool.isPresent()) ? (StoragePool) this.dataStoreMgr.getDataStore(storagePool.get().getId(), DataStoreRole.Primary) :
+ (StoragePool)dataStoreMgr.getDataStore(poolList.get(0).getId(), DataStoreRole.Primary);
Review comment:
can you extract this into a `private StoragePool checkPreferredStorageForUse(..)` and than store the result in a `StoragePool poolToReturn = null;` so we have only one exit point. I.E. end the method with `return poolToReturn`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-896160015
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian. SL-JID 830
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-897102726
<b>Trillian test result (tid-1585)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 57440 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5249-t1585-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 85 look OK, 4 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_02_deploy_and_upgrade_kubernetes_cluster | `Failure` | 3609.38 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 128.72 | test_kubernetes_clusters.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Failure` | 349.86 | test_routers_network_ops.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 613.35 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | `Failure` | 521.11 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Failure` | 581.72 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Failure` | 509.06 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Error` | 509.07 | test_vpc_redundant.py
test_01_vpc_site2site_vpn_multiple_options | `Failure` | 640.53 | test_vpc_vpn.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5249: Global setting to select preferred storage pool
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5249:
URL: https://github.com/apache/cloudstack/pull/5249#issuecomment-896138890
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org