You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/05/14 02:53:00 UTC
[jira] [Commented] (KYLIN-4964) Receiver consumer thread should be
stoped while encounting unrecoverable error
[ https://issues.apache.org/jira/browse/KYLIN-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344264#comment-17344264 ]
ASF GitHub Bot commented on KYLIN-4964:
---------------------------------------
hit-lacus commented on pull request #1622:
URL: https://github.com/apache/kylin/pull/1622#issuecomment-840970658
LGTM, thx @dixingxing0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Receiver consumer thread should be stoped while encounting unrecoverable error
> ------------------------------------------------------------------------------
>
> Key: KYLIN-4964
> URL: https://issues.apache.org/jira/browse/KYLIN-4964
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
> Affects Versions: v3.1.1
> Reporter: Xingxing Di
> Assignee: Xingxing Di
> Priority: Major
>
> org.apache.kylin.stream.core.storage.columnar.ColumnarMemoryStorePersister#persist will catch exception and just log it, this behavior will cause critical problem, like our case: there is no space left on device, `persist` method failed for many times, we lost several hours of data.
> Here is our solution which already tested:
> # Throw IllegalStorageException while we cannot do the persist
> # Then stop the consumer thread
> # Add `consumer_thread_alive` in ConsumerStats for monitoring
> # Also fix an another issue which will cause wrong result after restore from checkpoint
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)