You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jeremy Hanna (JIRA)" <ji...@apache.org> on 2011/01/25 18:30:43 UTC
[jira] Created: (CASSANDRA-2052) Add configurable retry count to
Hadoop code
Add configurable retry count to Hadoop code
-------------------------------------------
Key: CASSANDRA-2052
URL: https://issues.apache.org/jira/browse/CASSANDRA-2052
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Affects Versions: 0.7.0, 0.6.9
Reporter: Jeremy Hanna
The Hadoop integration code doesn't do a retry if it times out. Often people have to tune the batch size and the rpc timeout in order to get it to work. We should probably have a configurable timeout in there (credit to Jairam - http://www.mail-archive.com/user@cassandra.apache.org/msg08938.html) so that it doesn't just fail the job with a timeout exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-2052) Add configurable retry count to
Hadoop code
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986540#action_12986540 ]
Jonathan Ellis commented on CASSANDRA-2052:
-------------------------------------------
As discussed in CASSANDRA-919 and CASSANDRA-959, I think this is the Wrong Fix. Timeouts indicate an overload scenario, so retrying might get you your results eventually, but it's more likely to perpetuate the over-capacity problem, causing more timeouts, causing more retries, ...
> Add configurable retry count to Hadoop code
> -------------------------------------------
>
> Key: CASSANDRA-2052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2052
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
> Affects Versions: 0.6.9, 0.7.0
> Reporter: Jeremy Hanna
>
> The Hadoop integration code doesn't do a retry if it times out. Often people have to tune the batch size and the rpc timeout in order to get it to work. We should probably have a configurable timeout in there (credit to Jairam - http://www.mail-archive.com/user@cassandra.apache.org/msg08938.html) so that it doesn't just fail the job with a timeout exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-2052) Add configurable retry count to
Hadoop code
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Hanna resolved CASSANDRA-2052.
-------------------------------------
Resolution: Won't Fix
like Jonathan mentioned, the problems with timing out should be handled with the existing configuration options.
> Add configurable retry count to Hadoop code
> -------------------------------------------
>
> Key: CASSANDRA-2052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2052
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
> Affects Versions: 0.6.9, 0.7.0
> Reporter: Jeremy Hanna
>
> The Hadoop integration code doesn't do a retry if it times out. Often people have to tune the batch size and the rpc timeout in order to get it to work. We should probably have a configurable timeout in there (credit to Jairam - http://www.mail-archive.com/user@cassandra.apache.org/msg08938.html) so that it doesn't just fail the job with a timeout exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-2052) Add configurable retry count to
Hadoop code
Posted by "Jeremy Hanna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986574#action_12986574 ]
Jeremy Hanna commented on CASSANDRA-2052:
-----------------------------------------
Based on discussion in IRC, it sounds like we just need to communicate good settings to users for when they're configuring and trying out their cluster. So a combination of batch size and rpc timeout tuning along with potentially separating out the nodes into a virtual datacenter with the network topology strategy so that you don't hit your nodes with batch processing jobs that really need to be responsive for oltp.
I can update the docs on the wiki and we should probably update the readme in the various hadoop contrib modules to point to the hadoop support wiki page as well.
> Add configurable retry count to Hadoop code
> -------------------------------------------
>
> Key: CASSANDRA-2052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2052
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
> Affects Versions: 0.6.9, 0.7.0
> Reporter: Jeremy Hanna
>
> The Hadoop integration code doesn't do a retry if it times out. Often people have to tune the batch size and the rpc timeout in order to get it to work. We should probably have a configurable timeout in there (credit to Jairam - http://www.mail-archive.com/user@cassandra.apache.org/msg08938.html) so that it doesn't just fail the job with a timeout exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.