You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jay Zhuang (JIRA)" <ji...@apache.org> on 2017/11/20 03:10:00 UTC

[jira] [Updated] (CASSANDRA-14060) Separate CorruptSSTableException and FSError handling policies

     [ https://issues.apache.org/jira/browse/CASSANDRA-14060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jay Zhuang updated CASSANDRA-14060:
-----------------------------------
    Description: 
Currently, if [{{disk_failure_policy}}|https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L230] is set to {{stop}} (default), StorageService will shutdown for {{FSError}}, but not {{CorruptSSTableException}} [DefaultFSErrorHandler.java:40|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DefaultFSErrorHandler.java#L40].

But when we use policy: {{die}}, it has different behave, JVM will be killed for both {{FSError}} and {{CorruptSSTableException}} [JVMStabilityInspector.java:63|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java#L63]:

||{{disk_failure_policy}}|| hit {{FSError}} Exception || hit {{CorruptSSTableException}} ||
|{{stop}}| (/) stop | (x) not stop |
|{{die}}| (/) die | (/) die |

We saw {{CorruptSSTableException}} from time to time in our production, but mostly it's *not* because of a disk issue. So I would suggest having a separate policy for CorruptSSTable.

  was:
Currently, if [{{disk_failure_policy}}|https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L230] is set to {{stop}} (default), StorageService will shutdown for {{FSError}}, but not {{CorruptSSTableException}} [DefaultFSErrorHandler.java:40|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DefaultFSErrorHandler.java#L40].

But when we use policy: {{die}}, it has different behave, JVM will be killed for both {{FSError}} and {{CorruptSSTableException}} [JVMStabilityInspector.java:63|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java#L63]:

||{{dis_failure_policy}}|| hit {{FSError}} Exception || hit {{CorruptSSTableException}} ||
|{{stop}}| (/) stop | (x) not stop |
|{{die}}| (/) die | (/) die |

We saw {{CorruptSSTableException}} from time to time in our production, but mostly it's *not* because of a disk issue. So I would suggest having a separate policy for CorruptSSTable.


> Separate CorruptSSTableException and FSError handling policies
> --------------------------------------------------------------
>
>                 Key: CASSANDRA-14060
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14060
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Configuration
>            Reporter: Jay Zhuang
>            Assignee: Jay Zhuang
>            Priority: Minor
>
> Currently, if [{{disk_failure_policy}}|https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L230] is set to {{stop}} (default), StorageService will shutdown for {{FSError}}, but not {{CorruptSSTableException}} [DefaultFSErrorHandler.java:40|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/DefaultFSErrorHandler.java#L40].
> But when we use policy: {{die}}, it has different behave, JVM will be killed for both {{FSError}} and {{CorruptSSTableException}} [JVMStabilityInspector.java:63|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java#L63]:
> ||{{disk_failure_policy}}|| hit {{FSError}} Exception || hit {{CorruptSSTableException}} ||
> |{{stop}}| (/) stop | (x) not stop |
> |{{die}}| (/) die | (/) die |
> We saw {{CorruptSSTableException}} from time to time in our production, but mostly it's *not* because of a disk issue. So I would suggest having a separate policy for CorruptSSTable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org