[jira] [Created] (ATLAS-1077) Kakfa exception during load testing in Atlas

["Sharmadha Sainath (JIRA)" <ji...@apache.org>]

Sharmadha Sainath created ATLAS-1077:
----------------------------------------

             Summary: Kakfa exception during load testing in Atlas
                 Key: ATLAS-1077
                 URL: https://issues.apache.org/jira/browse/ATLAS-1077
             Project: Atlas
          Issue Type: Bug
         Environment: Cluster set up :
Machine 1 : Atlas , Solr,
Machine 2 : Kafka , HBase
Machine 3 : Hive, client
            Reporter: Sharmadha Sainath


5,00,000 Hive tables are created via hiveserver2 and ingested to Atlas. Only 4,18,655 tables are created in Atlas. 5,00,000 tables are created in Hive and no exception is found in Hive. Zookeeper throws the following exception :


ERROR - [atlas_cn058-10.l42scl.hortonworks.com-1469728635962-8a80738e_watcher_executor:] ~ [atlas_cn058-10.l42scl.hortonworks.com-1469728635962-8a80738e], error during syncedRebalance (Logging$class:103)
kafka.common.ConsumerRebalanceFailedException: atlas_cn058-10.l42scl.hortonworks.com-1469728635962-8a80738e can't rebalance after 4 retries
	at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:670)
	at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$2.run(ZookeeperConsumerConnector.scala:589)


the test is run with default settings :
atlas.kafka.zookeeper.session.timeout.ms=400
atlas.kafka.zookeeper.connection.timeout.ms=200
atlas.kafka.zookeeper.sync.time.ms=20
atlas.audit.zookeeper.session.timeout.ms=1000



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)