You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Tomas Satka <sa...@gmail.com> on 2021/08/06 12:40:21 UTC

apache beam to apache atlas connector

Hi Team

Is there any data connector that can send metadata from beam to apache
atlas?
As of today we intend to use apache spark and send metadata to apache atlas
by hortonworks connector (
https://github.com/hortonworks-spark/spark-atlas-connector).

Is there an alternative once we switch spark to beam to the above mentioned
connector?

Thanks & Regards
Tomas

Re: apache beam to apache atlas connector

Posted by Robert Bradshaw <ro...@google.com>.

Also, if you can find a Hadoop connector, you could use that from
Beam: https://beam.apache.org/documentation/io/built-in/hadoop/

On Fri, Aug 6, 2021 at 7:58 AM Chamikara Jayalath <ch...@google.com> wrote:
>
> Please see here for currently available built-in Beam connectors: https://beam.apache.org/documentation/io/built-in/
>
> Thanks,
> Cham
>
> On Fri, Aug 6, 2021 at 5:40 AM Tomas Satka <sa...@gmail.com> wrote:
>>
>> Hi Team
>>
>> Is there any data connector that can send metadata from beam to apache atlas?
>> As of today we intend to use apache spark and send metadata to apache atlas by hortonworks connector (https://github.com/hortonworks-spark/spark-atlas-connector).
>>
>> Is there an alternative once we switch spark to beam to the above mentioned connector?
>>
>> Thanks & Regards
>> Tomas

Re: apache beam to apache atlas connector

Posted by Robert Bradshaw <ro...@google.com>.

Also, if you can find a Hadoop connector, you could use that from
Beam: https://beam.apache.org/documentation/io/built-in/hadoop/

On Fri, Aug 6, 2021 at 7:58 AM Chamikara Jayalath <ch...@google.com> wrote:
>
> Please see here for currently available built-in Beam connectors: https://beam.apache.org/documentation/io/built-in/
>
> Thanks,
> Cham
>
> On Fri, Aug 6, 2021 at 5:40 AM Tomas Satka <sa...@gmail.com> wrote:
>>
>> Hi Team
>>
>> Is there any data connector that can send metadata from beam to apache atlas?
>> As of today we intend to use apache spark and send metadata to apache atlas by hortonworks connector (https://github.com/hortonworks-spark/spark-atlas-connector).
>>
>> Is there an alternative once we switch spark to beam to the above mentioned connector?
>>
>> Thanks & Regards
>> Tomas

Re: apache beam to apache atlas connector

Posted by Chamikara Jayalath <ch...@google.com>.

Please see here for currently available built-in Beam connectors:
https://beam.apache.org/documentation/io/built-in/

Thanks,
Cham

On Fri, Aug 6, 2021 at 5:40 AM Tomas Satka <sa...@gmail.com> wrote:

> Hi Team
>
> Is there any data connector that can send metadata from beam to apache
> atlas?
> As of today we intend to use apache spark and send metadata to apache
> atlas by hortonworks connector (
> https://github.com/hortonworks-spark/spark-atlas-connector).
>
> Is there an alternative once we switch spark to beam to the above
> mentioned connector?
>
> Thanks & Regards
> Tomas
>

Re: apache beam to apache atlas connector

Posted by Chamikara Jayalath <ch...@google.com>.

Please see here for currently available built-in Beam connectors:
https://beam.apache.org/documentation/io/built-in/

Thanks,
Cham

On Fri, Aug 6, 2021 at 5:40 AM Tomas Satka <sa...@gmail.com> wrote:

> Hi Team
>
> Is there any data connector that can send metadata from beam to apache
> atlas?
> As of today we intend to use apache spark and send metadata to apache
> atlas by hortonworks connector (
> https://github.com/hortonworks-spark/spark-atlas-connector).
>
> Is there an alternative once we switch spark to beam to the above
> mentioned connector?
>
> Thanks & Regards
> Tomas
>