You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "Damans227 (via GitHub)" <gi...@apache.org> on 2026/01/14 19:22:55 UTC
[PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Damans227 opened a new pull request, #12426:
URL: https://github.com/apache/cloudstack/pull/12426
### Description
This PR fixes an issue where the SystemVM template is not automatically downloaded to S3 secondary storage when adding it to a CloudStack zone.
**Root Cause:**
S3 stores use `REGION` scope but `DefaultEndPointSelector` only returned `LocalHostEndpoint` for `ZONE` scope, so no endpoint was found to download the SystemVM template.
**Fix:**
Allow `LocalHostEndpoint` to handle SYSTEM template downloads for `REGION`-scoped stores, plus added null checks for S3 stores without URLs and enabled path-style access for S3-compatible storage.
Fixes: #9002
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] Build/CI
- [ ] Test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [ ] Minor
#### Bug Severity
- [ ] BLOCKER
- [x] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
**Broken:**
<img width="2134" height="465" alt="image" src="https://github.com/user-attachments/assets/e8334b38-1649-46ff-b3ca-2e4f60c70777" />
**Fixed:**
https://github.com/user-attachments/assets/c513f2bf-35f7-4452-aba4-31624200826e
### How Has This Been Tested?
**Test Environment:**
- CloudStack 4.20.3.0
- KVM hypervisor (Ubuntu 22.04)
- MinIO S3-compatible storage
**Test Steps:**
1. Started with a fresh CloudStack zone with no secondary storage configured
2. Added NFS staging store (required for S3)
3. Added S3 image store (MinIO) via UI
4. Verified SystemVM template automatically downloaded to S3
5. Verified SSVM booted successfully using the template from S3
6. Verified Console Proxy VM also booted successfully
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3837077553
> Tested with the pr packages and on oracle linux 8.6
Reproduced this issue around Ceph S3 being used as secondary storage. I too get the empty error - logs show `downloadStatus: DOWNLOAD_ERROR` but `errorString` is blank. Looking into it now.
logs:
```
2026-02-02 18:53:10,866 DEBUG [c.c.a.t.Request] Seq 3-6421288643700195481: Sending { Cmd , DownloadCommand ... "_store":{"com.cloud.agent.api.to.S3TO":{"endPoint":"http://10.0.33.100","bucketName":"cloudstack-secondary"}} ... }
2026-02-02 18:53:13,956 DEBUG [c.c.a.t.Request] Seq 3-6421288643700195481: Processing: { Ans: , DownloadAnswer {"errorString":"","downloadStatus":"DOWNLOAD_ERROR","templateSize":"(0 bytes) 0"} }
2026-02-02 18:53:13,978 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl] Failed to register template: 36d0a884-0055-11f1-bd19-1e004d0003d0 with error:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3898109666
<b>[SF] Trillian Build Failed (tid-15465)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "vishesh92 (via GitHub)" <gi...@apache.org>.
vishesh92 commented on code in PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#discussion_r2697074463
##########
utils/src/main/java/com/cloud/utils/storage/S3/S3Utils.java:
##########
@@ -114,6 +114,8 @@ public static TransferManager getTransferManager(final ClientOptions clientOptio
LOGGER.debug(format("Setting the end point for S3 client with access key %1$s to %2$s.", clientOptions.getAccessKey(), clientOptions.getEndPoint()));
client.setEndpoint(clientOptions.getEndPoint());
+ // Enable path-style access for S3-compatible storage
+ client.setS3ClientOptions(com.amazonaws.services.s3.S3ClientOptions.builder().setPathStyleAccess(true).build());
Review Comment:
Why is this required?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5622278300
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✖️ debian ✔️ suse15. SL-JID 19210
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5176918323
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18757
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3752819001
<b>[SF] Trillian Build Failed (tid-15185)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#discussion_r2697091918
##########
engine/storage/src/main/java/org/apache/cloudstack/storage/endpoint/DefaultEndPointSelector.java:
##########
@@ -400,7 +400,9 @@ public EndPoint select(DataObject object) {
}
if (object instanceof TemplateInfo) {
TemplateInfo tmplInfo = (TemplateInfo)object;
- if (store.getScope().getScopeType() == ScopeType.ZONE && store.getScope().getScopeId() == null && tmplInfo.getTemplateType() == TemplateType.SYSTEM) {
+ if (tmplInfo.getTemplateType() == TemplateType.SYSTEM &&
+ (store.getScope().getScopeType() == ScopeType.REGION ||
+ (store.getScope().getScopeType() == ScopeType.ZONE && store.getScope().getScopeId() == null))) {
Review Comment:
The modified endpoint selection logic for REGION-scoped SYSTEM templates lacks test coverage. Consider adding unit tests in the engine/storage module to verify that LocalHostEndpoint is correctly returned for REGION-scoped stores with SYSTEM templates, similar to the existing test coverage in SecondaryStorageManagerImplTest.
##########
utils/src/main/java/com/cloud/utils/storage/S3/S3Utils.java:
##########
@@ -114,6 +114,8 @@ public static TransferManager getTransferManager(final ClientOptions clientOptio
LOGGER.debug(format("Setting the end point for S3 client with access key %1$s to %2$s.", clientOptions.getAccessKey(), clientOptions.getEndPoint()));
client.setEndpoint(clientOptions.getEndPoint());
+ // Enable path-style access for S3-compatible storage
+ client.setS3ClientOptions(com.amazonaws.services.s3.S3ClientOptions.builder().setPathStyleAccess(true).build());
Review Comment:
Path-style access is being enabled unconditionally for all S3 endpoints, including AWS S3 which deprecated path-style access in favor of virtual-hosted-style. This could cause compatibility issues with AWS S3. Consider making path-style access configurable through ClientOptions, or only enabling it when a custom endpoint is detected (non-AWS S3).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3752799827
## [Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/12426?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `33.33333%` with `4 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 17.09%. Comparing base ([`6a324da`](https://app.codecov.io/gh/apache/cloudstack/commit/6a324da27a6005959932d2ddf108ed76f6878aae?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`9a2ba84`](https://app.codecov.io/gh/apache/cloudstack/commit/9a2ba84d427618dcf8e7c1072ce9da25f263ec30?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on 4.20.
| [Files with missing lines](https://app.codecov.io/gh/apache/cloudstack/pull/12426?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...tack/storage/endpoint/DefaultEndPointSelector.java](https://app.codecov.io/gh/apache/cloudstack/pull/12426?src=pr&el=tree&filepath=engine%2Fstorage%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fcloudstack%2Fstorage%2Fendpoint%2FDefaultEndPointSelector.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZW5naW5lL3N0b3JhZ2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Nsb3Vkc3RhY2svc3RvcmFnZS9lbmRwb2ludC9EZWZhdWx0RW5kUG9pbnRTZWxlY3Rvci5qYXZh) | 0.00% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/cloudstack/pull/12426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../main/java/com/cloud/utils/storage/S3/S3Utils.java](https://app.codecov.io/gh/apache/cloudstack/pull/12426?src=pr&el=tree&filepath=utils%2Fsrc%2Fmain%2Fjava%2Fcom%2Fcloud%2Futils%2Fstorage%2FS3%2FS3Utils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-dXRpbHMvc3JjL21haW4vamF2YS9jb20vY2xvdWQvdXRpbHMvc3RvcmFnZS9TMy9TM1V0aWxzLmphdmE=) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/cloudstack/pull/12426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## 4.20 #12426 +/- ##
============================================
+ Coverage 16.23% 17.09% +0.85%
- Complexity 13382 13385 +3
============================================
Files 5657 5255 -402
Lines 498999 466300 -32699
Branches 60566 54743 -5823
============================================
- Hits 81035 79714 -1321
+ Misses 408928 377704 -31224
+ Partials 9036 8882 -154
```
| [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/12426/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/12426/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/12426/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `17.09% <33.33%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/cloudstack/pull/12426?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3855439506
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3843427634
@kiranchavala Bucket details shown in the secondary storage details:
<img width="1888" height="1257" alt="image" src="https://github.com/user-attachments/assets/6114a38c-7b7c-4e7e-8071-d0e8e5025cee" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3816493221
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16615
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3842362999
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16690
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3840532477
> Thanks @Damans227
>
> The issue is solved when I point it to a HTTP s3 link
>
> Create a zone from scratch , in the zone creation wizard add s3 as the secondary storage
>
>
> 1. Te system template that is getting download is a old one
>
> 2. The db also points to a older record
>
> ```
> mysql> select name,url,hypervisor_type from vm_template where type="SYSTEM";
> +-------------------------------+-------------------------------------------------------------------------------------------+-----------------+
> | name | url | hypervisor_type |
> +-------------------------------+-------------------------------------------------------------------------------------------+-----------------+
> | SystemVM Template (XenServer) | http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-xen.vhd.bz2 | XenServer |
> | SystemVM Template (KVM) | http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 | KVM |
> | SystemVM Template (vSphere) | http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-vmware.ova | VMware |
> | SystemVM Template (HyperV) | http://download.cloudstack.org/templates/4.3/systemvm64template-2013-12-23-hyperv.vhd.bz2 | Hyperv |
> | SystemVM Template (LXC) | http://download.cloudstack.org/templates/acton/acton-systemvm-02062012.qcow2.bz2 | LXC |
> | SystemVM Template (Ovm3) | http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-ovm.raw.bz2 | Ovm3 |
> +-------------------------------+-------------------------------------------------------------------------------------------+-----------------+
> ```
>
>
>
> ```
> aws --profile kirans3 --endpoint-url http://10.0.33.100 s3 ls s3://testcskiran --recursive
> 2026-02-03 15:16:34 319401369 template/tmpl/1/3/routing-3/systemvm64template-4.6.0-kvm.qcow2.bz2
>
> aws --profile daman --endpoint-url http://10.0.33.100 s3 ls s3://testbucket --recursive
> 2026-02-03 00:36:02 5 template/test-upload.txt
> 2026-02-03 00:47:34 319401369 template/tmpl/1/3/routing-3/systemvm64template-4.6.0-kvm.qcow2.bz2
> 2026-01-09 11:21:48 14 test.txt
>
> ```
>
> Will check again with a fresh deployment
Got it. Thanks for checking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3897932233
<b>[SF] Trillian Build Failed (tid-15462)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3945695625
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5621790023
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-6488981246
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19821
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3752811421
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on code in PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#discussion_r2699309890
##########
utils/src/main/java/com/cloud/utils/storage/S3/S3Utils.java:
##########
@@ -114,6 +114,8 @@ public static TransferManager getTransferManager(final ClientOptions clientOptio
LOGGER.debug(format("Setting the end point for S3 client with access key %1$s to %2$s.", clientOptions.getAccessKey(), clientOptions.getEndPoint()));
client.setEndpoint(clientOptions.getEndPoint());
+ // Enable path-style access for S3-compatible storage
+ client.setS3ClientOptions(com.amazonaws.services.s3.S3ClientOptions.builder().setPathStyleAccess(true).build());
Review Comment:
hmm, path-style access code is already inside the `if (StringUtils.isNotBlank(clientOptions.getEndPoint()))` block - which means path-style access is only enabled when a custom endpoint is specified. The code is already does what Copilot is asking for.
##########
utils/src/main/java/com/cloud/utils/storage/S3/S3Utils.java:
##########
@@ -114,6 +114,8 @@ public static TransferManager getTransferManager(final ClientOptions clientOptio
LOGGER.debug(format("Setting the end point for S3 client with access key %1$s to %2$s.", clientOptions.getAccessKey(), clientOptions.getEndPoint()));
client.setEndpoint(clientOptions.getEndPoint());
+ // Enable path-style access for S3-compatible storage
+ client.setS3ClientOptions(com.amazonaws.services.s3.S3ClientOptions.builder().setPathStyleAccess(true).build());
Review Comment:
hmm, path-style access code is already inside the `if (StringUtils.isNotBlank(clientOptions.getEndPoint()))` block - which means path-style access is only enabled when a custom endpoint is specified. The code is already doing what Copilot is asking for.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3860241729
Thanks @Damans227 of the fixes
is the scope set to Region by Default or can Zone be set ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3840508300
Thanks @Damans227
The issue is solved when I point it to a HTTP s3 link
Create a zone from scratch , in the zone creation wizard add s3 as the secondary storage
1. Te system template that is getting download is a old one
2. The db also points to a older record
```
mysql> select name,url,hypervisor_type from vm_template where type="SYSTEM";
+-------------------------------+-------------------------------------------------------------------------------------------+-----------------+
| name | url | hypervisor_type |
+-------------------------------+-------------------------------------------------------------------------------------------+-----------------+
| SystemVM Template (XenServer) | http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-xen.vhd.bz2 | XenServer |
| SystemVM Template (KVM) | http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 | KVM |
| SystemVM Template (vSphere) | http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-vmware.ova | VMware |
| SystemVM Template (HyperV) | http://download.cloudstack.org/templates/4.3/systemvm64template-2013-12-23-hyperv.vhd.bz2 | Hyperv |
| SystemVM Template (LXC) | http://download.cloudstack.org/templates/acton/acton-systemvm-02062012.qcow2.bz2 | LXC |
| SystemVM Template (Ovm3) | http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-ovm.raw.bz2 | Ovm3 |
+-------------------------------+-------------------------------------------------------------------------------------------+-----------------+
```
```
aws --profile kirans3 --endpoint-url http://10.0.33.100 s3 ls s3://testcskiran --recursive
2026-02-03 15:16:34 319401369 template/tmpl/1/3/routing-3/systemvm64template-4.6.0-kvm.qcow2.bz2
aws --profile daman --endpoint-url http://10.0.33.100 s3 ls s3://testbucket --recursive
2026-02-03 00:36:02 5 template/test-upload.txt
2026-02-03 00:47:34 319401369 template/tmpl/1/3/routing-3/systemvm64template-4.6.0-kvm.qcow2.bz2
2026-01-09 11:21:48 14 test.txt
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3916447010
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3947611224
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5198801425
<b>[SF] Trillian test result (tid-16698)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 135436 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12426-t16698-kvm-ol8.zip
Smoke tests completed. 114 look OK, 27 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_CRUD_operations_userdata | `Error` | 1525.05 | test_register_userdata.py
test_deploy_vm_with_registered_userdata | `Error` | 10.00 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow | `Error` | 11.09 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append | `Error` | 10.94 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_deny | `Error` | 10.68 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params | `Error` | 10.48 | test_register_userdata.py
test_link_and_unlink_userdata_to_template | `Error` | 11.40 | test_register_userdata.py
test_user_userdata_crud | `Error` | 9.53 | test_register_userdata.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
ContextSuite context=TestRAMCPUResourceAccounting>:setup | `Error` | 0.00 | test_resource_accounting.py
test_03_register_template | `Error` | 1.14 | test_resource_names.py
ContextSuite context=TestRestoreVM>:setup | `Error` | 0.00 | test_restore_vm.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestRouterIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestIpv6Vpc>:setup | `Error` | 0.00 | test_vpc_ipv6.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestCpuCapServiceOfferings>:setup | `Error` | 0.00 | test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 0.29 | test_service_offerings.py
test_06_download_detached_volume | `Error` | 307.40 | test_volumes.py
ContextSuite context=TestSetSourceNatIp>:setup | `Error` | 0.00 | test_set_sourcenat.py
test_04_deploy_vnf_appliance | `Error` | 312.22 | test_vnf_templates.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
ContextSuite context=TestSnapshotStandaloneBackup>:setup | `Error` | 0.00 | test_snapshots.py
test_11_destroy_ssvm | `Error` | 954.54 | test_ssvm.py
ContextSuite context=TestVMWareStoragePolicies>:setup | `Error` | 0.00 | test_storage_policy.py
test_01_create_template | `Error` | 25.90 | test_templates.py
test_CreateTemplateWithDuplicateName | `Error` | 41.56 | test_templates.py
test_02_create_template_with_checksum_sha1 | `Error` | 65.73 | test_templates.py
test_03_create_template_with_checksum_sha256 | `Error` | 65.66 | test_templates.py
test_04_create_template_with_checksum_md5 | `Error` | 65.68 | test_templates.py
test_05_create_template_with_no_checksum | `Error` | 65.70 | test_templates.py
test_03_delete_template | `Error` | 1.08 | test_templates.py
test_04_extract_template | `Error` | 1.11 | test_templates.py
test_09_list_templates_download_details | `Failure` | 0.13 | test_templates.py
ContextSuite context=TestISOUsage>:setup | `Error` | 0.00 | test_usage.py
test_01_template_usage | `Error` | 24.95 | test_usage.py
test_01_volume_usage | `Error` | 191.19 | test_usage.py
test_10_attachAndDetach_iso | `Failure` | 1513.25 | test_vm_life_cycle.py
ContextSuite context=TestMigrateVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestRestoreVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestScaleVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestVMDeploymentPlannerStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestVPCRedundancy>:setup | `Error` | 0.00 | test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setup | `Error` | 0.00 | test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
test_disable_oobm_ha_state_ineligible | `Error` | 1516.56 | test_hostha_kvm.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-6487419579
@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3752798767
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16368
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3816178926
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3855381575
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3842315654
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3891468887
> @Damans227 i see the zone creation is struck though the correct template got downloaded to s3 storage
>
> ```
> aws --profile ceph-s3 --endpoint-url http://10.0.33.100 s3 ls s3://cloudstack-storage3 --recursive
> 2026-02-11 12:39:13 528175117 template/tmpl/1/3/routing-3/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2
> ```
>
> logs
>
> ```
> 2026-02-11 07:09:07,191 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] (pool-11-thread-1:[ctx-4c665486]) (logid:ff3c1fc5) Executing command "DownloadProgressCommand" [{"jobId":"0d1856f7-08fa-4e78-a644-109350cedcf3","request":"GET_STATUS","hvm":false,"description":"SystemVM Template (KVM)","checksum":"0520ffc83b6b8aa412e55056509cc70b","maxDownloadSizeInBytes":53687091200,"id":3,"resourceType":"TEMPLATE","installPath":"template/tmpl/1/3/routing-3","_store":{"id":1,"uuid":"9b63bee0-04ab-4975-b592-78974f81fb3f","accessKey":"7G2194SICMBG834PBGMJ","secretKey":"6VzoXhXoz58ScOPzenXbugnaAGSkCgg8A4gz9mQC","endPoint":"http://10.0.33.100","bucketName":"cloudstack-storage3","httpsFlag":false,"created":"Feb 11, 2026, 7:08:34 AM","enableRRS":false,"maxSingleUploadSizeInBytes":5368709120},"followRedirects":false,"url":"https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2","format":"QCOW2","accountId":1,"name":"routing-3","contextMap":{},"wait":0,"bypassHostM
aintenance":false}].
> ```
>
> <img alt="Screenshot 2026-02-11 at 12 40 41 PM" width="1202" height="838" src="https://private-user-images.githubusercontent.com/1401014/548092554-ebb2e350-2f2e-40ab-9115-a95cab8413a3.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzA5MDg3NDUsIm5iZiI6MTc3MDkwODQ0NSwicGF0aCI6Ii8xNDAxMDE0LzU0ODA5MjU1NC1lYmIyZTM1MC0yZjJlLTQwYWItOTExNS1hOTVjYWI4NDEzYTMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDIxMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjAyMTJUMTUwMDQ1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MjA2NTNlYWM2MjY1Zjc5OTBmZmZjZjRjNGQ3NzhmNDdlMmIzYzA2ZmVmNGI2NDM2Y2NlZGQ2YmM5ZmFkNGNmNyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.1TYtQZ83CKNrxEXtbiTvmcxODtzpTB6CTnYNaZoxjig"> <img alt="Screenshot 2026-02-11 at 12 43 40 PM" width="1122" height="179" src="https://private-user-images.githubusercontent.com/14
01014/548092826-0aebfc0a-975c-43ab-b8c0-4180d997bc25.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzA5MDg3NDUsIm5iZiI6MTc3MDkwODQ0NSwicGF0aCI6Ii8xNDAxMDE0LzU0ODA5MjgyNi0wYWViZmMwYS05NzVjLTQzYWItYjhjMC00MTgwZDk5N2JjMjUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDIxMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjAyMTJUMTUwMDQ1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTgyMjY3ZTdiYmU4YjhiYzVkNjNiYzMyYWVmY2I2ZGMwMzMzNTQxZTg4YmFiOTA1MjQ4NmM0ZjUwZWVmZDg4YSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.MNqG8mHfPGDRzuorbOf0CfNzZAZjlrJbNKmSPEyq4OE"> <img alt="Screenshot 2026-02-11 at 12 43 33 PM" width="1140" height="223" src="https://private-user-images.githubusercontent.com/1401014/548092833-fcab5541-240f-4311-ae31-9c596215a94d.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF
3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzA5MDg3NDUsIm5iZiI6MTc3MDkwODQ0NSwicGF0aCI6Ii8xNDAxMDE0LzU0ODA5MjgzMy1mY2FiNTU0MS0yNDBmLTQzMTEtYWUzMS05YzU5NjIxNWE5NGQucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDIxMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjAyMTJUMTUwMDQ1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MjFjZTU5ZWUyNTNjMWIyN2ZlYzE1MWQyNDY4ZjQ3ZmFlMmNmMWM1NzMyMDI5ZjM3MzI0ZGFjOTRiMDBmMDRkYiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.N4Xbvr6uYjpcSQAeXdAM1ZYV9WM5BthkL1niK6X7JZA">
I am not able to reproduce this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3926279035
@Damans227 still hitting the issue with the latest packages
<img width="1089" height="744" alt="Screenshot 2026-02-19 at 3 51 24 PM" src="https://github.com/user-attachments/assets/3ab440e2-d0d2-46e8-ae81-fdc15d462750" />
```
2026-02-19 10:16:54,498 DEBUG [c.c.a.ApiServlet] (qtp1390913202-20:[ctx-a987051b]) (logid:aaab27b7) ===START=== 10.0.3.251 -- GET name=sec&provider=S3&zoneid=5c2ae5a5-23e6-4016-9ded-d0d98197f359&details[2].key=bucket&details[2].value=cloudstack-storage4&details[3].key=usehttps&details[3].value=false&details[4].key=endpoint&details[4].value=http:%2F%2F10.0.33.100&command=addImageStore&response=json&sessionkey=2F2ygFipVavNRBLZozAjuh6kMHk
2026-02-19 10:16:54,498 DEBUG [c.c.a.ApiServlet] (qtp1390913202-20:[ctx-a987051b]) (logid:aaab27b7) Two factor authentication is already verified for the user 2, so skipping
2026-02-19 10:16:54,507 DEBUG [c.c.a.ApiServer] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"c387b0c4-0d6e-11f1-98e4-1e003d000366"}]' is allowed to perform API calls: 0.0.0.0/0,::/0
2026-02-19 10:16:54,509 INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) Account for user id c3889ae9-0d6e-11f1-98e4-1e003d000366 is Root Admin or Domain Admin, all APIs are allowed.
2026-02-19 10:16:54,509 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2026-02-19 10:16:54,509 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) API rate limiting is disabled. We will not use ApiRateLimitService.
2026-02-19 10:16:54,513 INFO [o.a.c.s.d.l.S3ImageStoreLifeCycleImpl] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) Trying to add a S3 store with endpoint: http://10.0.33.100
2026-02-19 10:16:54,579 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) template Template {"format":"QCOW2","id":3,"name":"SystemVM Template (KVM)","uniqueName":"routing-3","uuid":"813c771e-0d6e-11f1-98e4-1e003d000366"} with id 3 is already in store:ImageStore {"id":1,"name":"sec","uuid":"165c480d-1bd1-44bb-93a0-77cc3f4be5a3"}, type: Image
2026-02-19 10:16:54,601 DEBUG [o.a.c.s.i.BaseImageStoreDriverImpl] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) Downloading template to data store ImageStore {"id":1,"name":"sec","uuid":"165c480d-1bd1-44bb-93a0-77cc3f4be5a3"}
2026-02-19 10:16:54,610 WARN [c.c.s.d.DownloadMonitorImpl] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) There is no secondary storage VM for downloading template to image store sec
2026-02-19 10:16:54,613 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) template Template {"format":"QCOW2","id":3,"name":"SystemVM Template (KVM)","uniqueName":"routing-3","uuid":"813c771e-0d6e-11f1-98e4-1e003d000366"} with id 3 is already in store:ImageStore {"id":1,"name":"sec","uuid":"165c480d-1bd1-44bb-93a0-77cc3f4be5a3"}, type: Image
2026-02-19 10:16:54,678 DEBUG [c.c.a.ApiServlet] (qtp1390913202-20:[ctx-a987051b, ctx-f7ca308e]) (logid:aaab27b7) ===END=== 10.0.3.251 -- GET name=sec&provider=S3&zoneid=5c2ae5a5-23e6-4016-9ded-d0d98197f359&details[2].key=bucket&details[2].value=cloudstack-storage4&details[3].key=usehttps&details[3].value=false&details[4].key=endpoint&details[4].value=http:%2F%2F10.0.33.100&command=addImageStore&response=json&sessionkey=2F2ygFipVavNRBLZozAjuh6kMHk
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3916746484
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16858
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-6038889032
@kiranchavala can you check this again? the issues you found in jan are fixed, retested on 4.20.4 above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5615002122
@Damans227 is the issue of downloading the older system vm template version resolved of still present
That was my last comment of my testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3945694680
@kiranchavala Shall be fixed now.
<img width="1135" height="398" alt="image" src="https://github.com/user-attachments/assets/ea222c80-ef79-4583-b7f8-d7df70a86201" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3842322426
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3907564187
<b>[SF] Trillian Build Failed (tid-15479)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3922583589
<b>[SF] Trillian test result (tid-15493)</b>
Environment: kvm-ol8 (x1), zone: Advanced Networking with Mgmt server ol8
Total time taken: 103473 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12426-t15493-kvm-ol8.zip
Smoke tests completed. 99 look OK, 29 have errors, 13 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_DeployVmAntiAffinityGroup_in_project | `Error` | 97.19 | test_affinity_groups_projects.py
test_DeployVmAntiAffinityGroup | `Error` | 43.91 | test_affinity_groups.py
ContextSuite context=TestInternalLb>:setup | `Error` | 0.00 | test_internal_lb.py
ContextSuite context=TestIpv4Routing>:setup | `Error` | 0.00 | test_ipv4_routing.py
ContextSuite context=TestCreateIpv6NetworkVpcOffering>:setup | `Error` | 0.00 | test_ipv6_infra.py
ContextSuite context=TestIpv6GuestPrefix>:setup | `Error` | 0.00 | test_ipv6_infra.py
ContextSuite context=TestIpv6PublicIpRange>:setup | `Error` | 0.00 | test_ipv6_infra.py
test_01_1_create_iso_with_checksum_sha1_negative | `Error` | 0.01 | test_iso.py
test_01_create_iso_with_checksum_sha1 | `Error` | 0.00 | test_iso.py
test_02_1_create_iso_with_checksum_sha256_negative | `Error` | 0.00 | test_iso.py
test_02_create_iso_with_checksum_sha256 | `Error` | 0.00 | test_iso.py
test_03_1_create_iso_with_checksum_md5_negative | `Error` | 0.01 | test_iso.py
test_03_create_iso_with_checksum_md5 | `Error` | 0.00 | test_iso.py
test_04_create_iso_with_no_checksum | `Error` | 0.00 | test_iso.py
test_01_create_iso | `Error` | 0.01 | test_iso.py
ContextSuite context=TestISO>:setup | `Error` | 0.01 | test_iso.py
ContextSuite context=TestListIdsParams>:setup | `Error` | 0.00 | test_list_ids_parameter.py
ContextSuite context=TestListServiceOfferings>:setup | `Error` | 0.00 | test_list_service_offerings.py
ContextSuite context=TestListStoragePools>:setup | `Error` | 0.00 | test_list_storage_pools.py
ContextSuite context=TestListVolumes>:setup | `Error` | 0.00 | test_list_volumes.py
ContextSuite context=TestProjectSuspendActivate>:setup | `Error` | 1525.89 | test_projects.py
ContextSuite context=TestLoadBalance>:setup | `Error` | 0.00 | test_loadbalance.py
test_01_deployVMInSharedNetwork | `Failure` | 5.43 | test_network.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:teardown | `Error` | 6.53 | test_network.py
ContextSuite context=TestAdapterTypeForNic>:setup | `Error` | 0.00 | test_nic_adapter_type.py
test_01_non_strict_host_anti_affinity | `Failure` | 133.52 | test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity | `Error` | 87.44 | test_nonstrict_affinity_group.py
test_CRUD_operations_userdata | `Error` | 1519.90 | test_register_userdata.py
test_deploy_vm_with_registered_userdata | `Error` | 6.40 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow | `Error` | 6.29 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append | `Error` | 6.64 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_deny | `Error` | 6.30 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params | `Error` | 6.22 | test_register_userdata.py
test_link_and_unlink_userdata_to_template | `Error` | 6.27 | test_register_userdata.py
test_user_userdata_crud | `Error` | 6.39 | test_register_userdata.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
test_03_register_template | `Error` | 633.50 | test_resource_names.py
test_04_migrate_to_read_only_storage | `Failure` | 1.27 | test_secondary_storage.py
test_05_migrate_to_less_free_space | `Error` | 1.22 | test_secondary_storage.py
ContextSuite context=TestVPCNics>:setup | `Error` | 0.00 | test_vpc_router_nics.py
test_01_snapshot_root_disk | `Error` | 1.15 | test_snapshots.py
test_02_list_snapshots_with_removed_data_store | `Error` | 44.16 | test_snapshots.py
ContextSuite context=TestSnapshotStandaloneBackup>:setup | `Error` | 1579.17 | test_snapshots.py
ContextSuite context=TestVMWareStoragePolicies>:setup | `Error` | 0.00 | test_storage_policy.py
test_01_create_template | `Error` | 1.12 | test_templates.py
test_CreateTemplateWithDuplicateName | `Error` | 1.11 | test_templates.py
test_02_create_template_with_checksum_sha1 | `Error` | 65.58 | test_templates.py
test_03_create_template_with_checksum_sha256 | `Error` | 65.54 | test_templates.py
ContextSuite context=TestTemplates>:setup | `Error` | 254.82 | test_templates.py
test_01_ISO_usage | `Failure` | 1.11 | test_usage.py
test_01_snapshot_usage | `Error` | 35.87 | test_usage.py
test_01_template_usage | `Error` | 1.16 | test_usage.py
test_01_volume_usage | `Failure` | 787.73 | test_usage.py
test_10_attachAndDetach_iso | `Failure` | 1514.89 | test_vm_life_cycle.py
ContextSuite context=TestMigrateVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestRestoreVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestScaleVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestVMDeploymentPlannerStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
test_06_download_detached_volume | `Error` | 299.78 | test_volumes.py
test_13_migrate_volume_and_change_offering | `Error` | 124.03 | test_volumes.py
ContextSuite context=TestIpv6Vpc>:setup | `Error` | 0.00 | test_vpc_ipv6.py
test_disable_oobm_ha_state_ineligible | `Error` | 1515.19 | test_hostha_kvm.py
test_hostha_enable_ha_when_host_in_maintenance | `Error` | 305.90 | test_hostha_kvm.py
all_test_kubernetes_clusters | `Skipped` | --- | test_kubernetes_clusters.py
all_test_kubernetes_supported_versions | `Skipped` | --- | test_kubernetes_supported_versions.py
all_test_list_accounts | `Skipped` | --- | test_list_accounts.py
all_test_list_disk_offerings | `Skipped` | --- | test_list_disk_offerings.py
all_test_list_domains | `Skipped` | --- | test_list_domains.py
all_test_list_hosts | `Skipped` | --- | test_list_hosts.py
all_test_login | `Skipped` | --- | test_login.py
all_test_metrics_api | `Skipped` | --- | test_metrics_api.py
all_test_migration | `Skipped` | --- | test_migration.py
all_test_multipleips_per_nic | `Skipped` | --- | test_multipleips_per_nic.py
all_test_nested_virtualization | `Skipped` | --- | test_nested_virtualization.py
all_test_network_acl | `Skipped` | --- | test_network_acl.py
all_test_network_ipv6 | `Skipped` | --- | test_network_ipv6.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3752813807
@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3816182414
@kiranchavala a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5177085724
@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-6487406069
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3882755491
@Damans227 i see the zone creation is struck though the correct template got downloaded to s3 storage
logs
```
2026-02-11 07:09:07,191 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] (pool-11-thread-1:[ctx-4c665486]) (logid:ff3c1fc5) Executing command "DownloadProgressCommand" [{"jobId":"0d1856f7-08fa-4e78-a644-109350cedcf3","request":"GET_STATUS","hvm":false,"description":"SystemVM Template (KVM)","checksum":"0520ffc83b6b8aa412e55056509cc70b","maxDownloadSizeInBytes":53687091200,"id":3,"resourceType":"TEMPLATE","installPath":"template/tmpl/1/3/routing-3","_store":{"id":1,"uuid":"9b63bee0-04ab-4975-b592-78974f81fb3f","accessKey":"7G2194SICMBG834PBGMJ","secretKey":"6VzoXhXoz58ScOPzenXbugnaAGSkCgg8A4gz9mQC","endPoint":"http://10.0.33.100","bucketName":"cloudstack-storage3","httpsFlag":false,"created":"Feb 11, 2026, 7:08:34 AM","enableRRS":false,"maxSingleUploadSizeInBytes":5368709120},"followRedirects":false,"url":"https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2","format":"QCOW2","accountId":1,"name":"routing-3","contextMap":{},"wait":0,"bypassHostMai
ntenance":false}].
```
<img width="1202" height="838" alt="Screenshot 2026-02-11 at 12 40 41 PM" src="https://github.com/user-attachments/assets/ebb2e350-2f2e-40ab-9115-a95cab8413a3" />
<img width="1122" height="179" alt="Screenshot 2026-02-11 at 12 43 40 PM" src="https://github.com/user-attachments/assets/0aebfc0a-975c-43ab-b8c0-4180d997bc25" />
<img width="1140" height="223" alt="Screenshot 2026-02-11 at 12 43 33 PM" src="https://github.com/user-attachments/assets/fcab5541-240f-4311-ae31-9c596215a94d" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3897573102
<b>[SF] Trillian Build Failed (tid-15458)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3946865920
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16918
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3945698061
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3946155786
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16914
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3947620351
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3947881182
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16920
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3840680646
Also @Damans227 If possible can you try to improve the cloudstack UI so that bucket details are show in the secondary storage details
Currently there is no way to identify the bucket and s3 URL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5995622799
@kiranchavala can you check again? retested on 4.20.4 above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3897680247
<b>[SF] Trillian Build Failed (tid-15460)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3914283550
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3834637933
> @Damans227
>
> Please find the issues that i observed
>
> Tested with the pr packages and on oracle linux 8.6
>
> 1. Unable to use ceph s3 storage , got the following exception is logs
>
> ```
> 2026-01-30 07:52:30,770 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] (pool-11-thread-1:[ctx-a125d9e1]) (logid:7259f280) Executing command "DownloadCommand" [{"hvm":false,"description":"SystemVM Template (KVM)","checksum":"c059b0d051e0cd6fbe9d5d4fc40c7e5d","maxDownloadSizeInBytes":53687091200,"id":3,"resourceType":"TEMPLATE","installPath":"template/tmpl/1/3/routing-3","_store":{"id":1,"uuid":"fdb66906-6b57-4e32-a7df-cbb93a917fce","accessKey":"AFYT2BKNI1U8T6DY6435","secretKey":"kia5kyDAZuP7QjwxNmhVAEE5l5dzsSJWbxtSXCIA","endPoint":"https://10.0.33.100","bucketName":"testbucket","httpsFlag":true,"created":"Jan 30, 2026, 7:52:20 AM","enableRRS":false,"maxSingleUploadSizeInBytes":5368709120},"followRedirects":false,"url":"http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2","format":"QCOW2","accountId":1,"name":"routing-3","contextMap":{},"wait":0,"bypassHostMaintenance":false}].
> 2026-01-30 07:52:30,798 DEBUG [c.c.u.n.HTTPUtils] (pool-11-thread-1:[ctx-a125d9e1]) (logid:7259f280) Initializing new HttpMethodRetryHandler with retry count 5
> 2026-01-30 07:52:30,892 INFO [c.c.s.t.S3TemplateDownloader] (pool-10-thread-1:[ctx-2cad9dd2]) (logid:1b4a38df) Starting download from http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 to S3 bucket testbucket and size (304.60 MB) 319401369 bytes
> 2026-01-30 07:52:30,897 DEBUG [c.c.u.s.S.S3Utils] (pool-10-thread-1:[ctx-2cad9dd2]) (logid:1b4a38df) Sending stream as S3 object template/tmpl/1/3/routing-3/systemvm64template-4.6.0-kvm.qcow2.bz2 in bucket testbucket using PutObjectRequest
> 2026-01-30 07:52:31,217 DEBUG [c.c.u.s.S.S3Utils] (pool-10-thread-1:[ctx-2cad9dd2]) (logid:1b4a38df) Creating S3 client with configuration: [protocol: https, signer: null, connectionTimeOut: 10000, maxErrorRetry: -1, socketTimeout: 50000, useTCPKeepAlive: null, connectionTtl: null]
> 2026-01-30 07:52:31,468 DEBUG [c.c.u.s.S.S3Utils] (pool-10-thread-1:[ctx-2cad9dd2]) (logid:1b4a38df) Setting the end point for S3 client with access key AFYT2BKNI1U8T6DY6435 to https://10.0.33.100.
>
>
>
> 2026-01-30 07:52:33,803 INFO [o.a.c.s.i.BaseImageStoreDriverImpl] (pool-11-thread-1:[ctx-a125d9e1]) (logid:7259f280) Updating store ref entry for template Template {"format":"QCOW2","id":3,"name":"SystemVM Template (KVM)","uniqueName":"routing-3","uuid":"56911227-fd0c-11f0-9d05-1e00e00002fb"}
> 2026-01-30 07:52:33,817 WARN [c.c.a.AlertManagerImpl] (pool-11-thread-1:[ctx-a125d9e1]) (logid:7259f280) alertType=[28] dataCenterId=[1] podId=[null] clusterId=[null] message=[Failed to register template: 56911227-fd0c-11f0-9d05-1e00e00002fb with error: ].
> 2026-01-30 07:52:33,825 WARN [c.c.a.AlertManagerImpl] (pool-11-thread-1:[ctx-a125d9e1]) (logid:7259f280) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Failed to register template: 56911227-fd0c-11f0-9d05-1e00e00002fb with error: ] and content [Failed to register template: 56911227-fd0c-11f0-9d05-1e00e00002fb with error: ].
> 2026-01-30 07:52:33,825 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl] (pool-11-thread-1:[ctx-a125d9e1]) (logid:7259f280) Failed to register template: 56911227-fd0c-11f0-9d05-1e00e00002fb
> with error:
> ```
>
> 2. Used Minio s3 storage, the systemvm template got registered successfully
>
> But it was of "systemvm64template-4.6.0-kvm.qcow2.bz2" and the systemvm were struck in starting state
>
> <img alt="Image" width="1827" height="532" src="https://private-user-images.githubusercontent.com/1401014/542754813-f1361c5a-3e7e-4058-8cb3-1dac449dfd8a.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzAwMzI4MzgsIm5iZiI6MTc3MDAzMjUzOCwicGF0aCI6Ii8xNDAxMDE0LzU0Mjc1NDgxMy1mMTM2MWM1YS0zZTdlLTQwNTgtOGNiMy0xZGFjNDQ5ZGZkOGEucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDIwMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjAyMDJUMTE0MjE4WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YTIxYTEwODRmNTljYzRkOGUzMGJmMTc0M2Y5NDRmMjA3NDVhZGJhMWZiYTFlNGM2MWJmMmQ5ZmQ1ODEwZWE1ZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.imxjgRN1pwMqvTVWswcjnDLGRMwyB3JJWkp5BsMsXqI">
> 3. When the primary storage is of zone scope,
>
> logs
>
> ```
> 2026-01-30 09:32:23,838 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-7:[ctx-45ba2dab, job-40/job-47, ctx-3f578420]) (logid:17e47580) Found template Template {"format":"QCOW2","id":3,"name":"SystemVM Template (KVM)","uniqueName":"routing-3","uuid":"56911227-fd0c-11f0-9d05-1e00e00002fb"} in storage pool StoragePool {"id":1,"name":"pri","poolType":"NetworkFilesystem","uuid":"cfc7f591-fc1e-36dd-b2c5-dc6712acf57e"} with VMTemplateStoragePool: TmplPool[3-3-1-null]
> 2026-01-30 09:32:23,839 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-7:[ctx-45ba2dab, job-40/job-47, ctx-3f578420]) (logid:17e47580) Acquire lock on VMTemplateStoragePool 3 with timeout 3600 seconds
> ```
>
> 4. When the primary storage is of cluster scope,
>
> logs
>
> ```
>
> 2026-01-30 09:47:14,659 DEBUG [o.a.c.s.c.m.StorageCacheManagerImpl] (Work-Job-Executor-6:[ctx-fdef51b0, job-62/job-64, ctx-9c2c6347]) (logid:11a5fb7e) waiting cache copy completion type: template, id: 3, lock: 638897157
> 2026-01-30 09:47:24,659 DEBUG [o.a.c.s.c.m.StorageCacheManagerImpl] (Work-Job-Executor-6:[ctx-fdef51b0, job-62/job-64, ctx-9c2c6347]) (logid:11a5fb7e) waken up
> 2026-01-30 09:47:24,661 DEBUG [o.a.c.s.c.m.StorageCacheManagerImpl] (Work-Job-Executor-6:[ctx-fdef51b0, job-62/job-64, ctx-9c2c6347]) (logid:11a5fb7e) waiting cache copy completion type: template, id: 3, lock: 638897157
> ```
I see. Do you have the test env available that I can use for debugging?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3837182139
@kiranchavala Regarding the Ceph issue, your logs show HTTPS endpoint (`https://10.0.33.100`) but port 443 isn't responding when I curl on that host:
```
darora@dell16plus:~$ curl -k https://10.0.33.100/testbucket
curl: (7) Failed to connect to 10.0.33.100 port 443 after 193 ms: Couldn't connect to server
```
However, testing with with HTTP `(http://10.0.33.100)` curl works fine, and the template downloads successfully:
The `testbucket` in screenshot below is where SystemVM template was downloaded.
<img width="2400" height="1108" alt="image" src="https://github.com/user-attachments/assets/148e84a8-711d-495a-8562-feac27ce50a4" />
Could you try with HTTP instead? Is there a reason HTTPS was configured?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3752690803
@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3752687857
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3861927623
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3873813772
@kiranchavala I tested the fix. On adding S3 as secondary storage, the SystemVM template URL in `vm_template` was automatically updated to `4.20.2` (from the stale 4.6.0) and the template downloaded successfully to the S3 bucket. Fix works end-to-end. Please verify when you get a chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5621754983
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3916456251
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3891677644
<b>[SF] Trillian Build Failed (tid-15446)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3898006264
<b>[SF] Trillian Build Failed (tid-15463)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3914281029
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3753678949
<b>[SF] Trillian Build Failed (tid-15187)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3946510505
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3946503871
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3842626503
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5176102523
@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5623143558
@kiranchavala fixed. Retested on a fresh `4.20.4.0` env with the latest packages, KVM/ol8, MinIO as the S3 store.
I set the KVM template URL back to the old 4.6 one by hand to get your exact starting state, then added the S3 store. It corrected itself.
Before:
`http://download.cloudstack.org/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2`
After:
`https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2`
And that is what went into the bucket:
```
mysql> select url from vm_template where id=3;
https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2
# mc ls --recursive local/cloudstack-sec
[2026-09-10 17:35:36 UTC] 504MiB template/tmpl/1/3/routing-3/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2
```
`template_store_ref` shows `DOWNLOADED` / `Ready`.
I also redid the case from your Feb 19 comment. Deleted the SSVM first, then added another S3 store. The management server did the download itself and got the right file again:
```
17:36:54 DEBUG [o.a.c.s.r.NfsSecondaryStorageResource] (pool-17-thread-1) Executing command "DownloadCommand"
url: https://download.cloudstack.org/systemvm/4.20/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2
17:36:54 INFO [c.c.s.t.S3TemplateDownloader] Starting download from .../systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2 to S3 bucket cloudstack-sec2
[2026-09-10 17:37:30 UTC] 504MiB template/tmpl/1/3/routing-3/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2
```
No `There is no secondary storage VM` message this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3855783412
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16717
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3842503571
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3840702751
> 2026-01-30 07:52:33,825 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl] (pool-11-thread-1:[ctx-a125d9e1]) (logid:7259f280) Failed to register template: 56911227-fd0c-11f0-9d05-1e00e00002fb
> with error:
Yea, I’ll look into it.
Also noticed that when an exception happens during template registration, the error string is coming through empty. I’ll try to fix that as well.
```
2026-01-30 07:52:33,825 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl] (pool-11-thread-1:[ctx-a125d9e1]) (logid:7259f280) Failed to register template: 56911227-fd0c-11f0-9d05-1e00e00002fb
with error:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3842886833
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16691
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3861618042
> Thanks @Damans227 of the fixes
>
> is the scope set to Region by Default or can Zone be set ?
No, S3 [is hardcoded to REGION scope only ](https://github.com/apache/cloudstack/blob/main/plugins/storage/image/s3/src/main/java/org/apache/cloudstack/storage/datastore/provider/S3ImageStoreProviderImpl.java#L92-L96)and can't be set to ZONE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3861935269
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3862138991
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16730
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on code in PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#discussion_r2698595266
##########
utils/src/main/java/com/cloud/utils/storage/S3/S3Utils.java:
##########
@@ -114,6 +114,8 @@ public static TransferManager getTransferManager(final ClientOptions clientOptio
LOGGER.debug(format("Setting the end point for S3 client with access key %1$s to %2$s.", clientOptions.getAccessKey(), clientOptions.getEndPoint()));
client.setEndpoint(clientOptions.getEndPoint());
+ // Enable path-style access for S3-compatible storage
+ client.setS3ClientOptions(com.amazonaws.services.s3.S3ClientOptions.builder().setPathStyleAccess(true).build());
Review Comment:
So, when debugging the issue... I noticed that the connection to MinIO failed at the time of template upload, with an error that looked something like:
`UnknownHostException: cloudstack-secondary.10.0.34.157` i.e. the SDK was trying to connect to the `http://cloudstack-secondary.10.0.34.157:9000/...` which is the virtual-hosted style (refer: [virtual style vs path style syntax for s3](https://kb.netapp.com/hybrid/StorageGRID/Grid_Tenant_Manager/What_is_path_style_vs_virtual_style)).
Looking at other S3-compatible plugins in CloudStack, I found that both [CephObjectStoreDriverImpl](https://github.com/apache/cloudstack/blob/main/plugins/storage/object/ceph/src/main/java/org/apache/cloudstack/storage/datastore/driver/CephObjectStoreDriverImpl.java#L348) and [CloudianHyperStoreUtil](https://github.com/apache/cloudstack/blob/main/plugins/storage/object/cloudian/src/main/java/org/apache/cloudstack/storage/datastore/util/CloudianHyperStoreUtil.java#L135) use `enablePathStyleAccess()` to get path-style URLs `http://10.0.34.157:9000/cloudstack-secondary/...` i.e.
```
AmazonS3 client = AmazonS3ClientBuilder.standard()
.enablePathStyleAccess()
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(accessKey, secretKey)))
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(url, "auto"))
.build();
```
Applying the same fix here worked. The [AWS SDK documentation](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/S3ClientOptions.Builder.html) confirms that path-style access must be explicitly enabled for S3-compatible stores.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3892000165
Hmm, for me adding a new zone with S3 bucket as a secondary storage ended up in error. Looking into it rn.
<img width="1844" height="462" alt="image" src="https://github.com/user-attachments/assets/643c1db0-bc6d-4e6d-abef-b6b5dca7e633" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-3914605249
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16855
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5176096477
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix automatic SystemVM template download to S3 secondary storage [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #12426:
URL: https://github.com/apache/cloudstack/pull/12426#issuecomment-5177078361
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org