You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Hemanth Yamijala (JIRA)" <ji...@apache.org> on 2016/04/04 13:59:25 UTC
[jira] [Commented] (ATLAS-629) Kafka messages in ATLAS_HOOK might
be lost in HA mode at the instant of failover.
[ https://issues.apache.org/jira/browse/ATLAS-629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223987#comment-15223987 ]
Hemanth Yamijala commented on ATLAS-629:
----------------------------------------
On a related note, the number of messages written to ATLAS_ENTITIES also differed from a case with switchovers to one without. Both of these issues are related to our auto-acknowledging Kafka messages before they are committed to the system, without a replay capability.
> Kafka messages in ATLAS_HOOK might be lost in HA mode at the instant of failover.
> ---------------------------------------------------------------------------------
>
> Key: ATLAS-629
> URL: https://issues.apache.org/jira/browse/ATLAS-629
> Project: Atlas
> Issue Type: Bug
> Reporter: Hemanth Yamijala
> Priority: Critical
> Fix For: 0.7-incubating
>
>
> Write data to Kafka continuously from Hive hook - can do this by writing a script that constantly creates tables. Bring down the Active instance with kill -9. Ensure writes continue after passive becomes active. The expectation is the number of tables created and the number of tables in Atlas match.
> In one test, wrote 180 tables and switched over 6 times from one instance to another. Found that 1 table was lost of the lot. i.e. 179 tables were created, and 1 did not get in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)