You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Madhan Neethiraj (JIRA)" <ji...@apache.org> on 2018/05/17 05:01:00 UTC

[jira] [Updated] (ATLAS-2047) NotificationHookConsumer: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling

     [ https://issues.apache.org/jira/browse/ATLAS-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Madhan Neethiraj updated ATLAS-2047:
------------------------------------
    Affects Version/s:     (was: trunk)
        Fix Version/s:     (was: trunk)
                       0.8.2
                       1.0.0

Committed to following branches:
 - master: http://git-wip-us.apache.org/repos/asf/atlas/commit/b837c0ee
 - branch-0.8: http://git-wip-us.apache.org/repos/asf/atlas/commit/cb151bdc

> NotificationHookConsumer: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling 
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: ATLAS-2047
>                 URL: https://issues.apache.org/jira/browse/ATLAS-2047
>             Project: Atlas
>          Issue Type: Bug
>          Components:  atlas-core
>    Affects Versions: 0.8-incubating
>            Reporter: Ashutosh Mestry
>            Assignee: Ashutosh Mestry
>            Priority: Major
>             Fix For: 0.8.2, 1.0.0
>
>         Attachments: ATLAS-2047-HookConsumer-ExceptionHandling.patch
>
>
> *Background*
> _KafkaConsumer_ is abstracted by _AtlasKafkaConsumer_. This is run using _HookConsumer_ which is derived from _kafka.utils.ShutdownableThread_.
> The _ShutdownableThread_ manages the thread. It handles exceptions thrown in the _doWork_ method and logs them.
> *Problem*
> Exception reported in the bug is thrown by _KafkaConsumer_ is handed by _HookConsumer_. Exception is logged but the thread keeps running. In cases where Kafa is in irrecoverable state, this behavior ends up filling up logs and providing no value in return.
> *Solution*
> Let _ShutdownableThread_ hand exceptions thrown by _KafkaConsumer_. Stop the thread if _KafkaConsumer_ suffers from irrecoverable error. This will avoid the situation described in the problem section.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)