You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Karel Vervaeke (Created) (JIRA)" <ji...@apache.org> on 2012/02/20 09:11:37 UTC
[jira] [Created] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Add a retry loop around apt-get and yum commands to overcome transient errors
-----------------------------------------------------------------------------
Key: WHIRR-517
URL: https://issues.apache.org/jira/browse/WHIRR-517
Project: Whirr
Issue Type: Improvement
Reporter: Karel Vervaeke
Often, installation on one or more nodes fails because of transient errors
(mostly failed connection attempts).
Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211782#comment-13211782 ]
Andrei Savu commented on WHIRR-517:
-----------------------------------
I will finish this one because I know Karel is busy this week.
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214785#comment-13214785 ]
Andrei Savu commented on WHIRR-517:
-----------------------------------
I'm starting to run tests for all services with branch-0.7 + this patch. If everything works fine I will commit and cut an RC (after also fixing WHIRR-526 - should not affect the code in any way).
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.7.1, 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch, WHIRR-517-for-0.7.1.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-517:
------------------------------
Attachment: WHIRR-517-for-0.7.1.patch
I am attaching an updated version of the patch. All integration tests are passing on aws-ec2 (see the update test matrix for more details). I will commit as soon as I do a bit more testing on cloudservers-us.
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.7.1, 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch, WHIRR-517-for-0.7.1.patch, WHIRR-517-for-0.7.1.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-517:
------------------------------
Fix Version/s: 0.7.1
Also adding this on the roadmap for 0.7.1
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.7.1, 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214794#comment-13214794 ]
Andrei Savu commented on WHIRR-517:
-----------------------------------
Here is the test matrix for this release: https://docs.google.com/spreadsheet/ccc?key=0AvGPW01Ku6xTdGtBUVU1VWFXRk9FWm5IcGlSSkc0bFE
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.7.1, 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch, WHIRR-517-for-0.7.1.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-517:
------------------------------
Attachment: WHIRR-517-for-0.7.1.patch
I am attaching a version of this patch that updates all the services for inclusion in 0.7.1.
For trunk we need to refactor things to avoid duplication (find a way to make a available to all service a set of core functions with a proper namespace).
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.7.1, 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch, WHIRR-517-for-0.7.1.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-517:
------------------------------
Component/s: core
Fix Version/s: 0.8.0
Assignee: Karel Vervaeke
+1 looks good to me. Good to commit as soon as we replace all apt-get / yum calls with this.
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Karel Vervaeke
> Fix For: 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214893#comment-13214893 ]
Andrei Savu commented on WHIRR-517:
-----------------------------------
Ok, I'm done with testing on cloudservers for today (the UK deployment is returning a large number of internal failures, the US deployment is extremely slow). I will just assume that if the core tests pass and everything works on aws-ec2 everything should also work on cloudservers. I will look into WHIRR-526 and prepare RC0 for 0.7.1. We can do more testing while voting.
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.7.1, 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch, WHIRR-517-for-0.7.1.patch, WHIRR-517-for-0.7.1.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu reassigned WHIRR-517:
---------------------------------
Assignee: Andrei Savu (was: Karel Vervaeke)
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.8.0
>
> Attachments: WHIRR-517-evaluation.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Karel Vervaeke (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karel Vervaeke updated WHIRR-517:
---------------------------------
Attachment: WHIRR-517-evaluation.patch
Attached a patch with proposed implementation.
It adds 3 functions: a generic 'retry' function, 'retry_apt_get' and 'retry_yum'.
For now, I've only changed the zookeeper-cdh implementation for evaluation. If no-one forsees major problems i'll start replacing all apt-get and yum calls.
I had to change the <filtering> in the pom because it would mess up expressions like $((tries-1)). (This also explains why JAVA_HOME wasn't being set properly).
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Reporter: Karel Vervaeke
> Attachments: WHIRR-517-evaluation.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (WHIRR-517) Add a retry loop around apt-get and
yum commands to overcome transient errors
Posted by "Andrei Savu (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu resolved WHIRR-517.
-------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.8.0)
Committed to branch 0.7. I will mark this as fixed for 0.7.1 and create a new issue for 0.8.0 (just to keep the release notes clean). Thanks Karel!
> Add a retry loop around apt-get and yum commands to overcome transient errors
> -----------------------------------------------------------------------------
>
> Key: WHIRR-517
> URL: https://issues.apache.org/jira/browse/WHIRR-517
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Reporter: Karel Vervaeke
> Assignee: Andrei Savu
> Fix For: 0.7.1
>
> Attachments: WHIRR-517-evaluation.patch, WHIRR-517-for-0.7.1.patch, WHIRR-517-for-0.7.1.patch
>
>
> Often, installation on one or more nodes fails because of transient errors
> (mostly failed connection attempts).
> Therefore we should make use of command's built-in retry options and/or add our own retry loops.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira