You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2021/12/18 17:08:43 UTC
[GitHub] [cloudstack] mlsorensen opened a new pull request #5789: Randomize managed volume copy host
mlsorensen opened a new pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789
* Managed volume copy was always returning first host that could see storage pools
* Fix null value in logging for ScaleIOPrimaryDataStoreDriver due to if/else logic error
Signed-off-by: Marcus Sorensen <ml...@apple.com>
### Description
This PR fixes #5788
It shuffles the list of valid host candidates that are used for processing volume migrations.
It also fixes a logic error in debug logging that caused 'null' to be printed. Originally concatenating a static string with potentially null value and then testing to see if the result is null, which would never be true.
A more ideal solution might be to track the jobs on each host and attempt to balance the load according to how busy each host is. This was investigated and there's currently no obvious way to look up job assignments to hosts via the DB, the Commands sent to Agents are kept live in memory via the AgentAttache. The ideal solution would be a much larger feature to coordinate Command assignments across the management server cluster, and possibly need improved agent comms to provide live status of Command progress.
I additionally spent some time looking into how to isolate and unit test `findUpAndEnabledHostWithAccessToStoragePools()`. Currently it would require mocking three Daos, the DataStoreProviderManager, a DataStoreProvider, and a DataStoreDriver. It's possible to write a test for this, however I was concerned that the unit test would be coding implementation across three methods into the test. These mocked Daos exist up to three methods deep - if the implementation of any of these methods changes then the test needs to be rewritten to code in the new implementation. I wasn't sure this was the right thing to do just to add a simple one-line shuffle but I'm happy to go that route if needed.
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [X] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [X] Minor
- [ ] Trivial
### How Has This Been Tested?
Have run this code against a live environment and tested volume migration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] mlsorensen commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
mlsorensen commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r774050292
##########
File path: plugins/storage/volume/scaleio/src/main/java/org/apache/cloudstack/storage/datastore/driver/ScaleIOPrimaryDataStoreDriver.java
##########
@@ -629,7 +629,7 @@ public void copyAsync(DataObject srcData, DataObject destData, Host destHost, As
private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO template to volume
- LOGGER.debug("Initiating copy from PowerFlex template volume on host " + destHost != null ? destHost.getId() : "");
+ LOGGER.debug(String.format("Initiating copy from PowerFlex template volume on host %s", destHost != null ? destHost.getId() : ""));
Review comment:
Sure, why not?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000070346
@blueorangutan test keepEnv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998487221
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 1971
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000681695
@blueorangutan test keepEnv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998552837
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1973
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
shwstppr commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998510989
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997656276
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998466292
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000032668
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997608823
Hi @mlsorensen I think, this is good to go in 4.16.1. Can you rebase with 4.16 branch. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998562948
@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] mlsorensen commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
mlsorensen commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r773533044
##########
File path: plugins/storage/volume/scaleio/src/main/java/org/apache/cloudstack/storage/datastore/driver/ScaleIOPrimaryDataStoreDriver.java
##########
@@ -629,7 +629,8 @@ public void copyAsync(DataObject srcData, DataObject destData, Host destHost, As
private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO template to volume
- LOGGER.debug("Initiating copy from PowerFlex template volume on host " + destHost != null ? destHost.getId() : "");
+ String debugMessage = "Initiating copy from PowerFlex template volume on host ";
+ LOGGER.debug(destHost != null ? debugMessage + destHost.getId() : debugMessage);
Review comment:
Good call!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997688539
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1956
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998475905
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r773161741
##########
File path: plugins/storage/volume/scaleio/src/main/java/org/apache/cloudstack/storage/datastore/driver/ScaleIOPrimaryDataStoreDriver.java
##########
@@ -629,7 +629,8 @@ public void copyAsync(DataObject srcData, DataObject destData, Host destHost, As
private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO template to volume
- LOGGER.debug("Initiating copy from PowerFlex template volume on host " + destHost != null ? destHost.getId() : "");
+ String debugMessage = "Initiating copy from PowerFlex template volume on host ";
+ LOGGER.debug(destHost != null ? debugMessage + destHost.getId() : debugMessage);
Review comment:
We could use `String.format` here:
```suggestion
LOGGER.debug(String.format("Initiating copy from PowerFlex template volume on host %s", destHost != null ? destHost.getId() : ""));
```
##########
File path: server/src/main/java/com/cloud/storage/StorageManagerImpl.java
##########
@@ -2136,6 +2136,7 @@ public Host findUpAndEnabledHostWithAccessToStoragePools(List<Long> poolIds) {
if (hostIds.isEmpty()) {
return null;
}
+ Collections.shuffle(hostIds);
Review comment:
I think we could add some log to this method, about which host was selected (or not).
##########
File path: plugins/storage/volume/scaleio/src/main/java/org/apache/cloudstack/storage/datastore/driver/ScaleIOPrimaryDataStoreDriver.java
##########
@@ -648,7 +649,8 @@ private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Hos
private Answer copyVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO volume
- LOGGER.debug("Initiating copy from PowerFlex volume on host " + destHost != null ? destHost.getId() : "");
+ String debugMessage = "Initiating copy from PowerFlex volume on host ";
+ LOGGER.debug(destHost != null ? debugMessage + destHost.getId() : debugMessage);
Review comment:
We also could use `String.format` here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000228053
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_multiplication_x: debian :heavy_check_mark: suse15. SL-JID 2003
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000263960
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000032668
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r773707614
##########
File path: plugins/storage/volume/scaleio/src/main/java/org/apache/cloudstack/storage/datastore/driver/ScaleIOPrimaryDataStoreDriver.java
##########
@@ -629,7 +629,7 @@ public void copyAsync(DataObject srcData, DataObject destData, Host destHost, As
private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO template to volume
- LOGGER.debug("Initiating copy from PowerFlex template volume on host " + destHost != null ? destHost.getId() : "");
+ LOGGER.debug(String.format("Initiating copy from PowerFlex template volume on host %s", destHost != null ? destHost.getId() : ""));
Review comment:
```suggestion
LOGGER.debug(String.format("Initiating copy from PowerFlex template volume on host %s", destHost != null ? destHost.getId() : "<unknown>"));
```
##########
File path: plugins/storage/volume/scaleio/src/main/java/org/apache/cloudstack/storage/datastore/driver/ScaleIOPrimaryDataStoreDriver.java
##########
@@ -648,7 +648,7 @@ private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Hos
private Answer copyVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO volume
- LOGGER.debug("Initiating copy from PowerFlex volume on host " + destHost != null ? destHost.getId() : "");
+ LOGGER.debug(String.format("Initiating copy from PowerFlex template volume on host %s", destHost != null ? destHost.getId() : ""));
Review comment:
same here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997622231
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 1950
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti removed a comment on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti removed a comment on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997606954
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] mlsorensen commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
mlsorensen commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r774065861
##########
File path: plugins/storage/volume/scaleio/src/main/java/org/apache/cloudstack/storage/datastore/driver/ScaleIOPrimaryDataStoreDriver.java
##########
@@ -629,7 +629,7 @@ public void copyAsync(DataObject srcData, DataObject destData, Host destHost, As
private Answer copyTemplateToVolume(DataObject srcData, DataObject destData, Host destHost) {
// Copy PowerFlex/ScaleIO template to volume
- LOGGER.debug("Initiating copy from PowerFlex template volume on host " + destHost != null ? destHost.getId() : "");
+ LOGGER.debug(String.format("Initiating copy from PowerFlex template volume on host %s", destHost != null ? destHost.getId() : ""));
Review comment:
Implementing this. Going to use "not specified" @DaanHoogland since that sounds less like an error. There is code later to resolve a destination host if this is not specified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000309174
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2007
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000878190
<b>Trillian test result (tid-2738)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 30871 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5789-t2738-kvm-centos7.zip
Smoke tests completed. 91 look OK, 0 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GabrielBrascher commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
GabrielBrascher commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r775944212
##########
File path: server/src/main/java/com/cloud/storage/StorageManagerImpl.java
##########
@@ -2136,6 +2136,7 @@ public Host findUpAndEnabledHostWithAccessToStoragePools(List<Long> poolIds) {
if (hostIds.isEmpty()) {
return null;
}
+ Collections.shuffle(hostIds);
Review comment:
I might be wrong on this one, but I think that there are not many logs regarding the chosen host to migrate.
Call hierarchy that I've taken a quick check does not present logs. Assuming your comment, this would be logged later on in the call hierarchy ( I haven't checked all the way). Even if it is, you can see that many exceptions can be thrown not too late after the host ID is retrieved.
```
> StorageManagerImpl.findUpAndEnabledHostWithAccessToStoragePools(List<Long>) (com.cloud.storage)
> VolumeServiceImpl.copyManagedVolume(VolumeInfo, DataStore) (org.apache.cloudstack.storage.volume)
> VolumeServiceImpl.copyVolume(VolumeInfo, DataStore) (org.apache.cloudstack.storage.volume)
> VolumeOrchestrator.copyVolumeFromSecToPrimary(VolumeInfo, VirtualMachine, VirtualMachineTemplate, DataCenter, Pod, Long, ServiceOffering, ...) (org.apache.cloudstack.engine.orchestration)
...
> VolumeOrchestrator.migrateVolume(Volume, StoragePool) (org.apache.cloudstack.engine.orchestration)
...
> VolumeApiServiceImpl.orchestrateExtractVolume(long, long) (com.cloud.storage)
...
```
Additionally, the `canHostAccessStoragePools` called inside the `findUpAndEnabledHostWithAccessToStoragePools` can return false. If the host cannot access, not much is logged as well.
Please, let us know in case we are missing something here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-999280274
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000188616
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti removed a comment on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti removed a comment on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000032161
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r776651377
##########
File path: server/src/main/java/com/cloud/storage/StorageManagerImpl.java
##########
@@ -2136,6 +2136,7 @@ public Host findUpAndEnabledHostWithAccessToStoragePools(List<Long> poolIds) {
if (hostIds.isEmpty()) {
return null;
}
+ Collections.shuffle(hostIds);
Review comment:
The logging is not of the randomised list but the selected host is logged by the deployment planner and startvm cmd.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997607744
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998511417
@shwstppr a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000681976
@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000032161
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000264607
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000188397
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud merged pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
rohityadavcloud merged pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000065100
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_multiplication_x: debian :heavy_check_mark: suse15. SL-JID 1993
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000070417
@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997606954
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997655499
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-997622231
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998561971
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] mlsorensen commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
mlsorensen commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r773531132
##########
File path: server/src/main/java/com/cloud/storage/StorageManagerImpl.java
##########
@@ -2136,6 +2136,7 @@ public Host findUpAndEnabledHostWithAccessToStoragePools(List<Long> poolIds) {
if (hostIds.isEmpty()) {
return null;
}
+ Collections.shuffle(hostIds);
Review comment:
There's logging elsewhere, where the returned value is used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-999280500
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] mlsorensen commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
mlsorensen commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998074511
> Hi @mlsorensen I think, this is good to go in 4.16.1. Can you rebase with 4.16 branch. Thanks.
Done. Do I need a separate PR for main or will this get into main?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998476156
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-998964434
<b>Trillian test result (tid-2698)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31743 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5789-t2698-kvm-centos7.zip
Smoke tests completed. 91 look OK, 0 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-999299192
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1985
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000070417
@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti removed a comment on pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
sureshanaparti removed a comment on pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#issuecomment-1000070346
@blueorangutan test keepEnv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a change in pull request #5789: Randomize managed volume copy host
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on a change in pull request #5789:
URL: https://github.com/apache/cloudstack/pull/5789#discussion_r776651377
##########
File path: server/src/main/java/com/cloud/storage/StorageManagerImpl.java
##########
@@ -2136,6 +2136,7 @@ public Host findUpAndEnabledHostWithAccessToStoragePools(List<Long> poolIds) {
if (hostIds.isEmpty()) {
return null;
}
+ Collections.shuffle(hostIds);
Review comment:
The logging is not of the randomised list but the selected host is logged by the deployment planner or during provisioning or volume operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org