You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Yaqian Zhang (Jira)" <ji...@apache.org> on 2021/11/04 06:39:00 UTC
[jira] [Updated] (KYLIN-4964) Receiver consumer thread should be
stoped while encounting unrecoverable error
[ https://issues.apache.org/jira/browse/KYLIN-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yaqian Zhang updated KYLIN-4964:
--------------------------------
Fix Version/s: v3.1.3
> Receiver consumer thread should be stoped while encounting unrecoverable error
> ------------------------------------------------------------------------------
>
> Key: KYLIN-4964
> URL: https://issues.apache.org/jira/browse/KYLIN-4964
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
> Affects Versions: v3.1.1
> Reporter: Xingxing Di
> Assignee: Xingxing Di
> Priority: Major
> Fix For: v3.1.3
>
>
> org.apache.kylin.stream.core.storage.columnar.ColumnarMemoryStorePersister#persist will catch exception and just log it, this behavior will cause critical problem, like our case: there is no space left on device, `persist` method failed for many times, we lost several hours of data.
> Here is our solution which already tested:
> # Throw IllegalStorageException while we cannot do the persist
> # Then stop the consumer thread
> # Add `consumer_thread_alive` in ConsumerStats for monitoring
> # Also fix an another issue which will cause wrong result after restore from checkpoint
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)