You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by AWeber - Benjamin Krein <be...@aweber.com> on 2012/08/02 19:50:36 UTC
Copying templates between availability zones
When I try to copy a template from one availability zone to another the webui says it succeeds, but the copy never shows up & I see this in the management log:
2012-08-02 17:44:00,935 DEBUG [agent.transport.Request] (Timer-6:null) Seq 12-1476463345: Sending { Cmd , MgmtId: 161334717904, via: 12, Ver: v1, Flags: 100011, [{"storage.DownloadProgressCommand":{"jobId":"7b468191-a7dc-41bf-8453-9e39b01b5fba","request":"GET_STATUS","hvm":true,"description":"Ubuntu10.04.4 64-bit (5GB)","checksum":"393d4aebf205603314affb8cd4d58d64","auth":{"userName":"cloud","password":"XXXXXXXXXXXXX"},"maxDownloadSizeInBytes":53687091200,"id":205,"url":"https://10-1-6-126.realhostip.com/copy/SecStorage/52659020-ee26-38cc-bca1-d037e65522a5/template/tmpl/2/205//50cfbdf4-1ae5-43ee-af9d-df9e73f56593.qcow2","format":"QCOW2","accountId":2,"name":"50cfbdf4-1ae5-43ee-af9d-df9e73f56593","secUrl":"nfs://myhost.internal.net/storage/cloud-sec2","wait":0}}] }
2012-08-02 17:44:00,977 DEBUG [agent.transport.Request] (AgentManager-Handler-1:null) Seq 12-1476463345: Processing: { Ans: , MgmtId: 161334717904, via: 12, Ver: v1, Flags: 10, [{"storage.DownloadAnswer":{"jobId":"7b468191-a7dc-41bf-8453-9e39b01b5fba","downloadPct":0,"errorString":" HTTP Server returned 403 (expected 200 OK) ","downloadStatus":"NOT_DOWNLOADED","downloadPath":"/mnt/SecStorage/16d441cf-f675-343b-884e-2ea5677773e8/template/tmpl/2/205/dnld4460230303792654544tmp_","templateSize":0,"templatePhySicalSize":0,"result":false,"wait":0}}] }
I do not see the 403 in the Apache logs of the storage VMs. I can select "Download Template" from the actions & successfully download the qcow through the webui.
--
Benjamin Krein
Re: Copying templates between availability zones
Posted by AWeber - Benjamin Krein <be...@aweber.com>.
I think I just found my issues. I actually did something very similar to what Nitin suggests below to track it down. The problem is that we're using basic networking & have the private network setup with the same gateway & subnet as the public network. When the storage VM comes up the public network gets setup first but then when the private network comes up on eth2 it clobbers the gateway & sets it to the eth2 interface. So when the copy is initiated between the storage VMs it happens across the private network but the /var/www/html/copy/.htaccess file only allows the public IP of the other SSVM, thus the 403 errors.
--
Benjamin Krein
On Aug 2, 2012, at 2:29 PM, Nitin Mehta wrote:
> Can you see the first log for this template initiation ? It should be logged with DownloadCommand and should have the url of the source ssvm's template. Then you can try going to the destination SSVM and try downloading that url.
> See what issues you get. I would also check the iptable rules to see if the destination ssvm is blocked from accessing the source ssvm and also if there is any .htaccess file in the apache directories forbidding the download of template
> Refer to http://wiki.cloudstack.org/pages/viewpage.action?pageId=9601278 for help as well.
>
> Thanks,
> -Nitin
> ________________________________________
> From: AWeber - Benjamin Krein [benk@aweber.com]
> Sent: Thursday, August 02, 2012 11:25 PM
> To: cloudstack-users@incubator.apache.org
> Subject: Re: Copying templates between availability zones
>
> On Aug 2, 2012, at 1:50 PM, AWeber - Benjamin Krein wrote:
>
>> When I try to copy a template from one availability zone to another the webui says it succeeds, but the copy never shows up & I see this in the management log:
>>
>> 2012-08-02 17:44:00,935 DEBUG [agent.transport.Request] (Timer-6:null) Seq 12-1476463345: Sending { Cmd , MgmtId: 161334717904, via: 12, Ver: v1, Flags: 100011, [{"storage.DownloadProgressCommand":{"jobId":"7b468191-a7dc-41bf-8453-9e39b01b5fba","request":"GET_STATUS","hvm":true,"description":"Ubuntu10.04.4 64-bit (5GB)","checksum":"393d4aebf205603314affb8cd4d58d64","auth":{"userName":"cloud","password":"XXXXXXXXXXXXX"},"maxDownloadSizeInBytes":53687091200,"id":205,"url":"https://10-1-6-126.realhostip.com/copy/SecStorage/52659020-ee26-38cc-bca1-d037e65522a5/template/tmpl/2/205//50cfbdf4-1ae5-43ee-af9d-df9e73f56593.qcow2","format":"QCOW2","accountId":2,"name":"50cfbdf4-1ae5-43ee-af9d-df9e73f56593","secUrl":"nfs://myhost.internal.net/storage/cloud-sec2","wait":0}}] }
>>
>> 2012-08-02 17:44:00,977 DEBUG [agent.transport.Request] (AgentManager-Handler-1:null) Seq 12-1476463345: Processing: { Ans: , MgmtId: 161334717904, via: 12, Ver: v1, Flags: 10, [{"storage.DownloadAnswer":{"jobId":"7b468191-a7dc-41bf-8453-9e39b01b5fba","downloadPct":0,"errorString":" HTTP Server returned 403 (expected 200 OK) ","downloadStatus":"NOT_DOWNLOADED","downloadPath":"/mnt/SecStorage/16d441cf-f675-343b-884e-2ea5677773e8/template/tmpl/2/205/dnld4460230303792654544tmp_","templateSize":0,"templatePhySicalSize":0,"result":false,"wait":0}}] }
>>
>> I do not see the 403 in the Apache logs of the storage VMs. I can select "Download Template" from the actions & successfully download the qcow through the webui.
>
>
> I should also note that I see the above log lines over and over again even after making the initial request. It's almost like the initial request is stuck & no other requests are actually doing anything. I've tried destroying/rebuilding the storage VMs & restarting the cloud-management service followed by re-running cloud-setup-management several times to no avail.
>
> I've also set the global 'secstorage.allowed.internal.sites' to match our internal subnet (10.1.0.0/16) which you can see that the storage VM request is in along with the secondary storage, agents, etc.
>
>
> --
> Benjamin Krein
RE: Copying templates between availability zones
Posted by Nitin Mehta <Ni...@citrix.com>.
Can you see the first log for this template initiation ? It should be logged with DownloadCommand and should have the url of the source ssvm's template. Then you can try going to the destination SSVM and try downloading that url.
See what issues you get. I would also check the iptable rules to see if the destination ssvm is blocked from accessing the source ssvm and also if there is any .htaccess file in the apache directories forbidding the download of template
Refer to http://wiki.cloudstack.org/pages/viewpage.action?pageId=9601278 for help as well.
Thanks,
-Nitin
________________________________________
From: AWeber - Benjamin Krein [benk@aweber.com]
Sent: Thursday, August 02, 2012 11:25 PM
To: cloudstack-users@incubator.apache.org
Subject: Re: Copying templates between availability zones
On Aug 2, 2012, at 1:50 PM, AWeber - Benjamin Krein wrote:
> When I try to copy a template from one availability zone to another the webui says it succeeds, but the copy never shows up & I see this in the management log:
>
> 2012-08-02 17:44:00,935 DEBUG [agent.transport.Request] (Timer-6:null) Seq 12-1476463345: Sending { Cmd , MgmtId: 161334717904, via: 12, Ver: v1, Flags: 100011, [{"storage.DownloadProgressCommand":{"jobId":"7b468191-a7dc-41bf-8453-9e39b01b5fba","request":"GET_STATUS","hvm":true,"description":"Ubuntu10.04.4 64-bit (5GB)","checksum":"393d4aebf205603314affb8cd4d58d64","auth":{"userName":"cloud","password":"XXXXXXXXXXXXX"},"maxDownloadSizeInBytes":53687091200,"id":205,"url":"https://10-1-6-126.realhostip.com/copy/SecStorage/52659020-ee26-38cc-bca1-d037e65522a5/template/tmpl/2/205//50cfbdf4-1ae5-43ee-af9d-df9e73f56593.qcow2","format":"QCOW2","accountId":2,"name":"50cfbdf4-1ae5-43ee-af9d-df9e73f56593","secUrl":"nfs://myhost.internal.net/storage/cloud-sec2","wait":0}}] }
>
> 2012-08-02 17:44:00,977 DEBUG [agent.transport.Request] (AgentManager-Handler-1:null) Seq 12-1476463345: Processing: { Ans: , MgmtId: 161334717904, via: 12, Ver: v1, Flags: 10, [{"storage.DownloadAnswer":{"jobId":"7b468191-a7dc-41bf-8453-9e39b01b5fba","downloadPct":0,"errorString":" HTTP Server returned 403 (expected 200 OK) ","downloadStatus":"NOT_DOWNLOADED","downloadPath":"/mnt/SecStorage/16d441cf-f675-343b-884e-2ea5677773e8/template/tmpl/2/205/dnld4460230303792654544tmp_","templateSize":0,"templatePhySicalSize":0,"result":false,"wait":0}}] }
>
> I do not see the 403 in the Apache logs of the storage VMs. I can select "Download Template" from the actions & successfully download the qcow through the webui.
I should also note that I see the above log lines over and over again even after making the initial request. It's almost like the initial request is stuck & no other requests are actually doing anything. I've tried destroying/rebuilding the storage VMs & restarting the cloud-management service followed by re-running cloud-setup-management several times to no avail.
I've also set the global 'secstorage.allowed.internal.sites' to match our internal subnet (10.1.0.0/16) which you can see that the storage VM request is in along with the secondary storage, agents, etc.
--
Benjamin Krein
Re: Copying templates between availability zones
Posted by AWeber - Benjamin Krein <be...@aweber.com>.
On Aug 2, 2012, at 1:50 PM, AWeber - Benjamin Krein wrote:
> When I try to copy a template from one availability zone to another the webui says it succeeds, but the copy never shows up & I see this in the management log:
>
> 2012-08-02 17:44:00,935 DEBUG [agent.transport.Request] (Timer-6:null) Seq 12-1476463345: Sending { Cmd , MgmtId: 161334717904, via: 12, Ver: v1, Flags: 100011, [{"storage.DownloadProgressCommand":{"jobId":"7b468191-a7dc-41bf-8453-9e39b01b5fba","request":"GET_STATUS","hvm":true,"description":"Ubuntu10.04.4 64-bit (5GB)","checksum":"393d4aebf205603314affb8cd4d58d64","auth":{"userName":"cloud","password":"XXXXXXXXXXXXX"},"maxDownloadSizeInBytes":53687091200,"id":205,"url":"https://10-1-6-126.realhostip.com/copy/SecStorage/52659020-ee26-38cc-bca1-d037e65522a5/template/tmpl/2/205//50cfbdf4-1ae5-43ee-af9d-df9e73f56593.qcow2","format":"QCOW2","accountId":2,"name":"50cfbdf4-1ae5-43ee-af9d-df9e73f56593","secUrl":"nfs://myhost.internal.net/storage/cloud-sec2","wait":0}}] }
>
> 2012-08-02 17:44:00,977 DEBUG [agent.transport.Request] (AgentManager-Handler-1:null) Seq 12-1476463345: Processing: { Ans: , MgmtId: 161334717904, via: 12, Ver: v1, Flags: 10, [{"storage.DownloadAnswer":{"jobId":"7b468191-a7dc-41bf-8453-9e39b01b5fba","downloadPct":0,"errorString":" HTTP Server returned 403 (expected 200 OK) ","downloadStatus":"NOT_DOWNLOADED","downloadPath":"/mnt/SecStorage/16d441cf-f675-343b-884e-2ea5677773e8/template/tmpl/2/205/dnld4460230303792654544tmp_","templateSize":0,"templatePhySicalSize":0,"result":false,"wait":0}}] }
>
> I do not see the 403 in the Apache logs of the storage VMs. I can select "Download Template" from the actions & successfully download the qcow through the webui.
I should also note that I see the above log lines over and over again even after making the initial request. It's almost like the initial request is stuck & no other requests are actually doing anything. I've tried destroying/rebuilding the storage VMs & restarting the cloud-management service followed by re-running cloud-setup-management several times to no avail.
I've also set the global 'secstorage.allowed.internal.sites' to match our internal subnet (10.1.0.0/16) which you can see that the storage VM request is in along with the secondary storage, agents, etc.
--
Benjamin Krein