You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Deep Singh <de...@gmail.com> on 2020/12/01 16:16:02 UTC
Re: Review Request 73049: ATLAS-4043: Atlas import-kafka.sh does not
import "__consumer_offsets" topic
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73049/
-----------------------------------------------------------
(Updated Dec. 1, 2020, 4:16 p.m.)
Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, and Sarath Subramanian.
Summary (updated)
-----------------
ATLAS-4043: Atlas import-kafka.sh does not import "__consumer_offsets" topic
Bugs: ATLAS-4043
https://issues.apache.org/jira/browse/ATLAS-4043
Repository: atlas
Description
-------
Recently changes were made to remove dependency on zookeeper. This bug got introduced as regression. Kafka AdminClient is now used to get the list of existing topics. The API removes the topic which are internal to Kafka.
With this fix I have introduced an API option to include the internal topic as well.
Diffs
-----
common/src/main/java/org/apache/atlas/utils/KafkaUtils.java 7a397b19b
Diff: https://reviews.apache.org/r/73049/diff/1/
Testing
-------
Manual testing done.
Thanks,
Deep Singh
Re: Review Request 73049: ATLAS-4043: Atlas import-kafka.sh does not
import "__consumer_offsets" topic
Posted by Deep Singh <de...@gmail.com>.
> On Dec. 1, 2020, 7:20 p.m., Sarath Subramanian wrote:
> > common/src/main/java/org/apache/atlas/utils/KafkaUtils.java
> > Lines 61 (patched)
> > <https://reviews.apache.org/r/73049/diff/2/?file=2242526#file2242526line61>
> >
> > consider renaming to:
> >
> > "atlas.hook.kafka.import.internal.topics" => "atlas.kafka.bridge.enable.internal.topics.import"
I some how missed this comment. This patch is already submitted. I will open another ticket to fix this.
- Deep
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73049/#review222259
-----------------------------------------------------------
On Dec. 1, 2020, 6:02 p.m., Deep Singh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73049/
> -----------------------------------------------------------
>
> (Updated Dec. 1, 2020, 6:02 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4043
> https://issues.apache.org/jira/browse/ATLAS-4043
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Recently changes were made to remove dependency on zookeeper. This bug got introduced as regression. Kafka AdminClient is now used to get the list of existing topics. The API removes the topic which are internal to Kafka.
> With this fix I have introduced an API option to include the internal topic as well.
> The option is configuration driven.
> One need to set atlas.hook.kafka.import.internal.topics=true in the atlas application properties(/etc/atlas/conf/atlas-application.properties)
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/atlas/utils/KafkaUtils.java 7a397b19b
>
>
> Diff: https://reviews.apache.org/r/73049/diff/2/
>
>
> Testing
> -------
>
> Manual testing done.
>
>
> Thanks,
>
> Deep Singh
>
>
Re: Review Request 73049: ATLAS-4043: Atlas import-kafka.sh does not
import "__consumer_offsets" topic
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73049/#review222259
-----------------------------------------------------------
Fix it, then Ship it!
common/src/main/java/org/apache/atlas/utils/KafkaUtils.java
Lines 61 (patched)
<https://reviews.apache.org/r/73049/#comment311326>
consider renaming to:
"atlas.hook.kafka.import.internal.topics" => "atlas.kafka.bridge.enable.internal.topics.import"
- Sarath Subramanian
On Dec. 1, 2020, 10:02 a.m., Deep Singh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73049/
> -----------------------------------------------------------
>
> (Updated Dec. 1, 2020, 10:02 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4043
> https://issues.apache.org/jira/browse/ATLAS-4043
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Recently changes were made to remove dependency on zookeeper. This bug got introduced as regression. Kafka AdminClient is now used to get the list of existing topics. The API removes the topic which are internal to Kafka.
> With this fix I have introduced an API option to include the internal topic as well.
> The option is configuration driven.
> One need to set atlas.hook.kafka.import.internal.topics=true in the atlas application properties(/etc/atlas/conf/atlas-application.properties)
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/atlas/utils/KafkaUtils.java 7a397b19b
>
>
> Diff: https://reviews.apache.org/r/73049/diff/2/
>
>
> Testing
> -------
>
> Manual testing done.
>
>
> Thanks,
>
> Deep Singh
>
>
Re: Review Request 73049: ATLAS-4043: Atlas import-kafka.sh does not
import "__consumer_offsets" topic
Posted by Deep Singh <de...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73049/
-----------------------------------------------------------
(Updated Dec. 1, 2020, 6:02 p.m.)
Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, and Sarath Subramanian.
Bugs: ATLAS-4043
https://issues.apache.org/jira/browse/ATLAS-4043
Repository: atlas
Description (updated)
-------
Recently changes were made to remove dependency on zookeeper. This bug got introduced as regression. Kafka AdminClient is now used to get the list of existing topics. The API removes the topic which are internal to Kafka.
With this fix I have introduced an API option to include the internal topic as well.
The option is configuration driven.
One need to set atlas.hook.kafka.import.internal.topics=true in the atlas application properties(/etc/atlas/conf/atlas-application.properties)
Diffs
-----
common/src/main/java/org/apache/atlas/utils/KafkaUtils.java 7a397b19b
Diff: https://reviews.apache.org/r/73049/diff/2/
Testing
-------
Manual testing done.
Thanks,
Deep Singh
Re: Review Request 73049: ATLAS-4043: Atlas import-kafka.sh does not
import "__consumer_offsets" topic
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73049/#review222258
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Dec. 1, 2020, 5:51 p.m., Deep Singh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73049/
> -----------------------------------------------------------
>
> (Updated Dec. 1, 2020, 5:51 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4043
> https://issues.apache.org/jira/browse/ATLAS-4043
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Recently changes were made to remove dependency on zookeeper. This bug got introduced as regression. Kafka AdminClient is now used to get the list of existing topics. The API removes the topic which are internal to Kafka.
> With this fix I have introduced an API option to include the internal topic as well.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/atlas/utils/KafkaUtils.java 7a397b19b
>
>
> Diff: https://reviews.apache.org/r/73049/diff/2/
>
>
> Testing
> -------
>
> Manual testing done.
>
>
> Thanks,
>
> Deep Singh
>
>
Re: Review Request 73049: ATLAS-4043: Atlas import-kafka.sh does not
import "__consumer_offsets" topic
Posted by Deep Singh <de...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73049/
-----------------------------------------------------------
(Updated Dec. 1, 2020, 5:51 p.m.)
Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, and Sarath Subramanian.
Bugs: ATLAS-4043
https://issues.apache.org/jira/browse/ATLAS-4043
Repository: atlas
Description
-------
Recently changes were made to remove dependency on zookeeper. This bug got introduced as regression. Kafka AdminClient is now used to get the list of existing topics. The API removes the topic which are internal to Kafka.
With this fix I have introduced an API option to include the internal topic as well.
Diffs (updated)
-----
common/src/main/java/org/apache/atlas/utils/KafkaUtils.java 7a397b19b
Diff: https://reviews.apache.org/r/73049/diff/2/
Changes: https://reviews.apache.org/r/73049/diff/1-2/
Testing
-------
Manual testing done.
Thanks,
Deep Singh