You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Manish Zope (Created) (JIRA)" <ji...@apache.org> on 2012/04/13 14:11:16 UTC
[jira] [Created] (CASSANDRA-4146) Delete sstables on abnormal
termination of sstableloader session
Delete sstables on abnormal termination of sstableloader session
----------------------------------------------------------------
Key: CASSANDRA-4146
URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
Project: Cassandra
Issue Type: Improvement
Components: Tools
Affects Versions: 1.0.9
Reporter: Manish Zope
Fix For: 1.1.0
There are three cases where we have observed the abnormal termination
1) In case of exception while loading.
2) User terminates the loading process.
3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) Delete sstables on abnormal
termination of sstableloader session
Posted by "Manish Zope (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Zope updated CASSANDRA-4146:
-----------------------------------
Remaining Estimate: 168h (was: 504h)
Original Estimate: 168h (was: 504h)
> Delete sstables on abnormal termination of sstableloader session
> ----------------------------------------------------------------
>
> Key: CASSANDRA-4146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Affects Versions: 1.0.9
> Reporter: Manish Zope
> Labels: sstableloader, tools
> Fix For: 1.1.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> There are three cases where we have observed the abnormal termination
> 1) In case of exception while loading.
> 2) User terminates the loading process.
> 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
> In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
> sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
> Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
> So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) Delete sstables on abnormal
termination of sstableloader session
Posted by "Manish Zope (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Zope updated CASSANDRA-4146:
-----------------------------------
Remaining Estimate: 48h (was: 168h)
Original Estimate: 48h (was: 168h)
> Delete sstables on abnormal termination of sstableloader session
> ----------------------------------------------------------------
>
> Key: CASSANDRA-4146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Affects Versions: 1.0.9
> Reporter: Manish Zope
> Labels: sstableloader, tools
> Fix For: 1.1.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> There are three cases where we have observed the abnormal termination
> 1) In case of exception while loading.
> 2) User terminates the loading process.
> 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
> In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
> sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
> Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
> So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) sstableloader should detect and
report failures
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4146:
--------------------------------------
Reviewer: yukim
> sstableloader should detect and report failures
> -----------------------------------------------
>
> Key: CASSANDRA-4146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Affects Versions: 1.0.9
> Reporter: Manish Zope
> Assignee: Brandon Williams
> Priority: Minor
> Labels: sstableloader, tools
> Fix For: 1.1.1
>
> Attachments: 4146.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> There are three cases where we have observed the abnormal termination
> 1) In case of exception while loading.
> 2) User terminates the loading process.
> 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
> In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
> sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
> Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
> So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) sstableloader should detect and
report failures
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4146:
----------------------------------------
Summary: sstableloader should detect and report failures (was: Delete sstables on abnormal termination of sstableloader session)
> sstableloader should detect and report failures
> -----------------------------------------------
>
> Key: CASSANDRA-4146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Affects Versions: 1.0.9
> Reporter: Manish Zope
> Priority: Minor
> Labels: sstableloader, tools
> Fix For: 1.1.1
>
> Attachments: 4146.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> There are three cases where we have observed the abnormal termination
> 1) In case of exception while loading.
> 2) User terminates the loading process.
> 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
> In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
> sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
> Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
> So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) Delete sstables on abnormal
termination of sstableloader session
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4146:
----------------------------------------
Reviewer: (was: jbellis)
Priority: Minor (was: Major)
Fix Version/s: (was: 1.1.0)
1.1.1
I don't think deleting the sstables is the right thing to do, since the loader didn't create those files, and they are potentially very expensive to recreate. However, I do understand that the loader not failing, nor being able to give any indication of failure is a problem, and now that we have CASSANDRA-4051 we can address that.
> Delete sstables on abnormal termination of sstableloader session
> ----------------------------------------------------------------
>
> Key: CASSANDRA-4146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Affects Versions: 1.0.9
> Reporter: Manish Zope
> Priority: Minor
> Labels: sstableloader, tools
> Fix For: 1.1.1
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> There are three cases where we have observed the abnormal termination
> 1) In case of exception while loading.
> 2) User terminates the loading process.
> 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
> In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
> sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
> Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
> So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4146) Delete sstables on abnormal
termination of sstableloader session
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4146:
----------------------------------------
Attachment: 4146.txt
Patch to detect and report failures, as well as exit with a non-zero return code if this happens. Also cuts the FST attempts to 4, which causes about 28s of delay, instead of 8 which results in a ridiculous 8+ minutes.
> Delete sstables on abnormal termination of sstableloader session
> ----------------------------------------------------------------
>
> Key: CASSANDRA-4146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Affects Versions: 1.0.9
> Reporter: Manish Zope
> Priority: Minor
> Labels: sstableloader, tools
> Fix For: 1.1.1
>
> Attachments: 4146.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> There are three cases where we have observed the abnormal termination
> 1) In case of exception while loading.
> 2) User terminates the loading process.
> 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
> In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
> sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
> Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
> So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4146) sstableloader should detect and
report failures
Posted by "Brandon Williams (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams reassigned CASSANDRA-4146:
-------------------------------------------
Assignee: Brandon Williams
> sstableloader should detect and report failures
> -----------------------------------------------
>
> Key: CASSANDRA-4146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Affects Versions: 1.0.9
> Reporter: Manish Zope
> Assignee: Brandon Williams
> Priority: Minor
> Labels: sstableloader, tools
> Fix For: 1.1.1
>
> Attachments: 4146.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> There are three cases where we have observed the abnormal termination
> 1) In case of exception while loading.
> 2) User terminates the loading process.
> 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
> In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
> sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
> Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
> So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4146) sstableloader should detect and
report failures
Posted by "Yuki Morishita (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254799#comment-13254799 ]
Yuki Morishita commented on CASSANDRA-4146:
-------------------------------------------
+1
> sstableloader should detect and report failures
> -----------------------------------------------
>
> Key: CASSANDRA-4146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4146
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Affects Versions: 1.0.9
> Reporter: Manish Zope
> Assignee: Brandon Williams
> Priority: Minor
> Labels: sstableloader, tools
> Fix For: 1.1.1
>
> Attachments: 4146.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> There are three cases where we have observed the abnormal termination
> 1) In case of exception while loading.
> 2) User terminates the loading process.
> 3) If some node is down OR un-reachable then sstableloader get stucked.In this case user have to terminate the process in between.
> In case of abnormal termination, sstables (which are added in this session) remains as it is on the cluster.In case user starts the process all over again by fixing the exception, it results in duplication of the data till Major compaction is triggered.
> sstableloader can maintain the session while loading the sstables in cluster.So in case of abnormal termination sstableloader triggers the event that will delete the sstables loaded in that session.
> Also It would be great to have timeout in case of sstableloader.That can be kept configurable.
> So if sstableloader process got stucked for period longer than timeout, it can terminate itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira