[jira] [Updated] (ATLAS-4043) Atlas import-kafka.sh does not import "__consumer_offsets" topic

["Sarath Subramanian (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ATLAS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sarath Subramanian updated ATLAS-4043:
--------------------------------------
    Fix Version/s: 2.2.0
                   3.0.0

> Atlas import-kafka.sh does not import "__consumer_offsets" topic
> ----------------------------------------------------------------
>
>                 Key: ATLAS-4043
>                 URL: https://issues.apache.org/jira/browse/ATLAS-4043
>             Project: Atlas
>          Issue Type: Bug
>          Components:  atlas-core
>    Affects Versions: 2.1.0
>            Reporter: Umesh Padashetty
>            Assignee: Deep Singh
>            Priority: Major
>             Fix For: 3.0.0, 2.2.0
>
>         Attachments: Fixed import __consumer_offsets.png, Screenshot 2020-11-20 at 11.23.47 PM.png
>
>
> Even though kafka-topics.sh --list, lists the __consumer_offsets topic as below, this topic alone is not imported into Atlas when import-kafka.sh script is run
> {code:java}
> ATLAS_ENTITIES 
> ATLAS_HOOK 
> ATLAS_SPARK_HOOK 
> __consumer_offsets 
> connect-configs 
> connect-offsets 
> connect-status  {code}
> I did not find anything unusual in atlas/application.log. This was working fine before, seems like a regression.
> As per official Kafka documentation, __consumer_offsets is used to store information about committed offsets for each topic:partition per group of consumers (groupID). It is compacted topic, so data will be periodically compressed and only latest offsets information available.
> Since __consumer_offsets has __ in the beginning, I have created a topic starting with __ and ran the import-kafka script. And it works fine. So that leaves us with having issue with __consumer_offsets alone.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)