Review Request 74100: ATLAS-4620: Refactor EmbeddedKafkaServer in Atlas notification module to remove core kafka dependency

[Patrik Márton via Review Board <no...@reviews.apache.org>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74100/
-----------------------------------------------------------

Review request for atlas.


Bugs: ATLAS-4620
    https://issues.apache.org/jira/browse/ATLAS-4620


Repository: atlas


Description
-------

Goal is to break the strong coupling between Atlas components and Kafka. These dependencies include using server side libraries of Kafka (this couples the Scala version and other non-public interfaces of Kafka). Any code using server side libraries of Kafka should be refactored.

The notification module uses an EmbeddedKafkaServer which brings in the core kafka dependency.The main reason for using this dependency is the kafka.server.KafkaServer. 
With this commit, the kafka.server.KafkaServer is changed to net.mguenther.kafka.junit.EmbeddedKafkaCluster. This is a kafka-junit library for testing - github: https://github.com/mguenther/kafka-junit / docs: https://mguenther.github.io/kafka-junit/ .

Notable changes:
- Removed the kafka.server.KafkaServer and replaced with net.mguenther.kafka.junit.EmbeddedKafkaCluster
- Since the net.mguenther.kafka.junit.EmbeddedKafkaCluster contains and embedded zookeeper as well, the embedded zookeeper in org.apache.atlas.kafka.EmbeddedKafkaServer was removed.
- The new kafka is using the same configuration properties as the old
- Removed the kafka_${kafka.scala.binary.version} artifact from the addons
- kafka_${kafka.scala.binary.version} is completly removed from the project


Diffs
-----

  addons/falcon-bridge/pom.xml 1e2ce7c81 
  addons/hbase-bridge/pom.xml a6ed51421 
  addons/hive-bridge/pom.xml 8c3636e5d 
  addons/impala-bridge/pom.xml 186251abe 
  addons/sqoop-bridge/pom.xml 021e93f56 
  addons/storm-bridge/pom.xml e8106afce 
  notification/pom.xml aaf11c7a4 
  notification/src/main/java/org/apache/atlas/kafka/EmbeddedKafkaServer.java 0a1f02add 
  pom.xml a2cbb8144 
  webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java 716f592f5 


Diff: https://reviews.apache.org/r/74100/diff/1/


Testing
-------

- Build is successful
- Unit / IT tests are green
- Atlas is healthy in manually provisioned cluster


Thanks,

Patrik Márton