You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Taylor <ts...@live.com> on 2017/07/19 04:29:56 UTC
Cloudstack 4.9 CentOS Template failure - Connection Reset
Hello,
I am experiencing an issue while trying to download the CentOS template.
It seems the connection is timing out and then failing.
To debug I logged into the SSVM and tried running a wget from the nfs directory mounted on that vm. This also failed due to connection reset.
Wget will eventually succeed if i use retry logic as follows:
wget http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2 --read-timeout=10
Using this setting the download will complete in pieces after multiple timeouts (logs below).
Is there a work around to add retry logic? Can I manually download and add the template to the database and restart the service? How can I file a bug report?
Thanks,
Taylor
=============================================================================
LOGS:
=============================================================================
root@s-103-VM:/mnt/SecStorage/5d8e791e-01cc-3d7c-84d8-f469944056e0/template/tmpl/1/5# wget http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2 --read-timeout=10
--2017-07-19 03:25:10-- http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Resolving download.cloud.com (download.cloud.com)... 54.231.81.40
Connecting to download.cloud.com (download.cloud.com)|54.231.81.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 374730926 (357M) [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
19% [============================> ] 74,734,204 --.-K/s in 97s
2017-07-19 03:26:55 (756 KB/s) - Read error at byte 74734204/374730926 (Connection timed out). Retrying.
--2017-07-19 03:26:56-- (try: 2) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|54.231.81.40|:80... failed: Connection timed out.
Resolving download.cloud.com (download.cloud.com)... 52.216.81.16
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 299996722 (286M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
24% [+++++++++++++++++++++++++++++======> ] 93,094,824 --.-K/s in 24s
2017-07-19 03:28:23 (740 KB/s) - Read error at byte 93094824/374730926 (Connection timed out). Retrying.
--2017-07-19 03:28:25-- (try: 3) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 281636102 (269M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
29% [++++++++++++++++++++++++++++++++++++======> ] 112,111,876 --.-K/s in 26s
2017-07-19 03:29:23 (702 KB/s) - Read error at byte 112111876/374730926 (Connection timed out). Retrying.
--2017-07-19 03:29:26-- (try: 4) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 262619050 (250M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
34% [+++++++++++++++++++++++++++++++++++++++++++=======> ] 130,790,615 --.-K/s in 26s
2017-07-19 03:30:23 (712 KB/s) - Read error at byte 130790615/374730926 (Connection timed out). Retrying.
--2017-07-19 03:30:27-- (try: 5) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 243940311 (233M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
49% [+++++++++++++++++++++++++++++++++++++++++++++++++++====================> ] 185,802,362 --.-K/s in 49s
2017-07-19 03:31:23 (1.08 MB/s) - Read error at byte 185802362/374730926 (Connection timed out). Retrying.
--2017-07-19 03:31:28-- (try: 6) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... failed: Connection timed out.
Resolving download.cloud.com (download.cloud.com)... 52.216.225.48
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 188928564 (180M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
60% [++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++================> ] 226,947,877 --.-K/s in 32s
2017-07-19 03:33:11 (1.21 MB/s) - Read error at byte 226947877/374730926 (Connection timed out). Retrying.
--2017-07-19 03:33:17-- (try: 7) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 147783049 (141M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
65% [+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++======> ] 247,259,880 --.-K/s in 28s
2017-07-19 03:33:46 (697 KB/s) - Read error at byte 247259880/374730926 (Connection timed out). Retrying.
--2017-07-19 03:33:53-- (try: 8) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 127471046 (122M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
100%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++==================================================>] 374,730,926 1010K/s in 2m 10s
Re: Cloudstack 4.9 CentOS Template failure - Connection Reset
Posted by Dag Sonstebo <Da...@shapeblue.com>.
Hi Taylor,
This is most likely an issue with your environment rather than a bug. Take a look at your public network and how that is connected to the internet. You have to let CloudStack pull down the template, it’s difficult to manually populate this.
A couple of other things to try:
- recreate the SSVM – simply delete it and CloudStack will generate a new one.
- from internally in the SSVM you can also run the SSVM check script, which will do some basic health checks for you:
root@s-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh
================================================
First DNS server is 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 48 data bytes
56 bytes from 8.8.8.8: icmp_seq=0 ttl=53 time=24.146 ms
56 bytes from 8.8.8.8: icmp_seq=1 ttl=53 time=22.320 ms
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 22.320/23.233/24.146/0.913 ms
Good: Can ping DNS server
================================================
Good: DNS resolves download.cloud.com
================================================
nfs is currently mounted
Mount point is /mnt/SecStorage/a833f5f1-1c6d-3e54-9a55-1fc9b7875c54
Good: Can write to mount point
================================================
Management server is 10.10.45.2. Checking connectivity.
Good: Can connect to management server port 8250
================================================
Good: Java process is running
================================================
Tests Complete. Look for ERROR or WARNING above.
Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue
On 19/07/2017, 05:29, "Taylor" <ts...@live.com> wrote:
Hello,
I am experiencing an issue while trying to download the CentOS template.
It seems the connection is timing out and then failing.
To debug I logged into the SSVM and tried running a wget from the nfs directory mounted on that vm. This also failed due to connection reset.
Wget will eventually succeed if i use retry logic as follows:
wget http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2 --read-timeout=10
Using this setting the download will complete in pieces after multiple timeouts (logs below).
Is there a work around to add retry logic? Can I manually download and add the template to the database and restart the service? How can I file a bug report?
Thanks,
Taylor
=============================================================================
LOGS:
=============================================================================
root@s-103-VM:/mnt/SecStorage/5d8e791e-01cc-3d7c-84d8-f469944056e0/template/tmpl/1/5# wget http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2 --read-timeout=10
--2017-07-19 03:25:10-- http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Resolving download.cloud.com (download.cloud.com)... 54.231.81.40
Connecting to download.cloud.com (download.cloud.com)|54.231.81.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 374730926 (357M) [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
19% [============================> ] 74,734,204 --.-K/s in 97s
2017-07-19 03:26:55 (756 KB/s) - Read error at byte 74734204/374730926 (Connection timed out). Retrying.
--2017-07-19 03:26:56-- (try: 2) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|54.231.81.40|:80... failed: Connection timed out.
Resolving download.cloud.com (download.cloud.com)... 52.216.81.16
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 299996722 (286M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
24% [+++++++++++++++++++++++++++++======> ] 93,094,824 --.-K/s in 24s
2017-07-19 03:28:23 (740 KB/s) - Read error at byte 93094824/374730926 (Connection timed out). Retrying.
--2017-07-19 03:28:25-- (try: 3) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 281636102 (269M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
29% [++++++++++++++++++++++++++++++++++++======> ] 112,111,876 --.-K/s in 26s
2017-07-19 03:29:23 (702 KB/s) - Read error at byte 112111876/374730926 (Connection timed out). Retrying.
--2017-07-19 03:29:26-- (try: 4) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 262619050 (250M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
34% [+++++++++++++++++++++++++++++++++++++++++++=======> ] 130,790,615 --.-K/s in 26s
2017-07-19 03:30:23 (712 KB/s) - Read error at byte 130790615/374730926 (Connection timed out). Retrying.
--2017-07-19 03:30:27-- (try: 5) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 243940311 (233M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
49% [+++++++++++++++++++++++++++++++++++++++++++++++++++====================> ] 185,802,362 --.-K/s in 49s
2017-07-19 03:31:23 (1.08 MB/s) - Read error at byte 185802362/374730926 (Connection timed out). Retrying.
--2017-07-19 03:31:28-- (try: 6) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80... failed: Connection timed out.
Resolving download.cloud.com (download.cloud.com)... 52.216.225.48
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 188928564 (180M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
60% [++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++================> ] 226,947,877 --.-K/s in 32s
2017-07-19 03:33:11 (1.21 MB/s) - Read error at byte 226947877/374730926 (Connection timed out). Retrying.
--2017-07-19 03:33:17-- (try: 7) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 147783049 (141M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
65% [+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++======> ] 247,259,880 --.-K/s in 28s
2017-07-19 03:33:46 (697 KB/s) - Read error at byte 247259880/374730926 (Connection timed out). Retrying.
--2017-07-19 03:33:53-- (try: 8) http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 127471046 (122M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
100%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++==================================================>] 374,730,926 1010K/s in 2m 10s
Dag.Sonstebo@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue