You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Marco Heyden <he...@gmail.com> on 2020/11/26 12:31:34 UTC
Kafka Authentication PR
Hey all,
I added username/password authentication to the Kafka connect adapter and the Kafka sink. While the unauthenticated version is still available, we now have the additional option to publish and consume from a username/password secured broker. See also https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_plain.html for additional information.
You find the PRs here:
• https://github.com/apache/incubator-streampipes-extensions/pull/32
• https://github.com/apache/incubator-streampipes/pull/28
Currently, however, username and password are sent in plaintext, thus I will add TLS encryption in the near future.
Best
Marco
Re: Kafka Authentication PR
Posted by Marco Heyden <he...@gmail.com>.
Hi Philipp,
that is great.
I will check that everything is working properly as soon as the build is completed.
Best
Marco
Am 01.12.20, 08:05 schrieb "Philipp Zehnder" <ze...@apache.org>:
Hi Marco,
thanks for your changes.
I reviewed the pull request and merged it.
Once the build is complete we can check if everything is working properly.
Philipp
> On 30. Nov 2020, at 12:36, Marco Heyden <he...@gmail.com> wrote:
>
> Hi Philipp,
>
> Thank you for your feedback.
> I updated the PR. Kindly let me know If there are any further issues.
>
> Best
> Marco
> Am 27. Nov. 2020, 19:16 +0100 schrieb Philipp Zehnder <ze...@apache.org>:
>> Hi Marco,
>>
>> thanks a lot for your PRs, the feature is very cool and the code looks good.
>>
>> There are two minor issues we have to fix before we can merge it.
>> It seems that there was a merge problem and somehow the file “streampipes-app-file-export/pom.xml <https://github.com/apache/incubator-streampipes/pull/28/files#diff-1dc29a04e5d3709e738a854d95de8a3f26517ed00e4ab3a8725caebd5fb7e857>” ended up in the code.
>> It looks like an old file that we removed from the code base. Can you please remove it.
>>
>> Several files do not have an Apache header. Can you please add those.
>> During the build it is checked that all files have such a header, otherwise the build fails.
>> You can test this locally by running “mvn clean package”, this should list all the files where a header is missing, if there are any.
>> You can also configure your IDE to automatically add a header for new files.
>>
>> Just write if you have a problem.
>>
>> I really like the idea of also integrating the TLS encryption. Do you already have an idea how this could be done?
>> Is this possible with our current concepts or do we need an additional feature to support it properly?
>>
>> Philipp
>>
>>
>>> On 26. Nov 2020, at 13:31, Marco Heyden <he...@gmail.com> wrote:
>>>
>>> Hey all,
>>>
>>> I added username/password authentication to the Kafka connect adapter and the Kafka sink. While the unauthenticated version is still available, we now have the additional option to publish and consume from a username/password secured broker. See also https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_plain.html for additional information.
>>>
>>> You find the PRs here:
>>>
>>> • https://github.com/apache/incubator-streampipes-extensions/pull/32
>>> • https://github.com/apache/incubator-streampipes/pull/28
>>>
>>> Currently, however, username and password are sent in plaintext, thus I will add TLS encryption in the near future.
>>>
>>> Best
>>> Marco
>>
Re: Kafka Authentication PR
Posted by Philipp Zehnder <ze...@apache.org>.
Hi Marco,
thanks for your changes.
I reviewed the pull request and merged it.
Once the build is complete we can check if everything is working properly.
Philipp
> On 30. Nov 2020, at 12:36, Marco Heyden <he...@gmail.com> wrote:
>
> Hi Philipp,
>
> Thank you for your feedback.
> I updated the PR. Kindly let me know If there are any further issues.
>
> Best
> Marco
> Am 27. Nov. 2020, 19:16 +0100 schrieb Philipp Zehnder <ze...@apache.org>:
>> Hi Marco,
>>
>> thanks a lot for your PRs, the feature is very cool and the code looks good.
>>
>> There are two minor issues we have to fix before we can merge it.
>> It seems that there was a merge problem and somehow the file “streampipes-app-file-export/pom.xml <https://github.com/apache/incubator-streampipes/pull/28/files#diff-1dc29a04e5d3709e738a854d95de8a3f26517ed00e4ab3a8725caebd5fb7e857>” ended up in the code.
>> It looks like an old file that we removed from the code base. Can you please remove it.
>>
>> Several files do not have an Apache header. Can you please add those.
>> During the build it is checked that all files have such a header, otherwise the build fails.
>> You can test this locally by running “mvn clean package”, this should list all the files where a header is missing, if there are any.
>> You can also configure your IDE to automatically add a header for new files.
>>
>> Just write if you have a problem.
>>
>> I really like the idea of also integrating the TLS encryption. Do you already have an idea how this could be done?
>> Is this possible with our current concepts or do we need an additional feature to support it properly?
>>
>> Philipp
>>
>>
>>> On 26. Nov 2020, at 13:31, Marco Heyden <he...@gmail.com> wrote:
>>>
>>> Hey all,
>>>
>>> I added username/password authentication to the Kafka connect adapter and the Kafka sink. While the unauthenticated version is still available, we now have the additional option to publish and consume from a username/password secured broker. See also https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_plain.html for additional information.
>>>
>>> You find the PRs here:
>>>
>>> • https://github.com/apache/incubator-streampipes-extensions/pull/32
>>> • https://github.com/apache/incubator-streampipes/pull/28
>>>
>>> Currently, however, username and password are sent in plaintext, thus I will add TLS encryption in the near future.
>>>
>>> Best
>>> Marco
>>
Re: Kafka Authentication PR
Posted by Marco Heyden <he...@gmail.com>.
Hi Philipp,
Thank you for your feedback.
I updated the PR. Kindly let me know If there are any further issues.
Best
Marco
Am 27. Nov. 2020, 19:16 +0100 schrieb Philipp Zehnder <ze...@apache.org>:
> Hi Marco,
>
> thanks a lot for your PRs, the feature is very cool and the code looks good.
>
> There are two minor issues we have to fix before we can merge it.
> It seems that there was a merge problem and somehow the file “streampipes-app-file-export/pom.xml <https://github.com/apache/incubator-streampipes/pull/28/files#diff-1dc29a04e5d3709e738a854d95de8a3f26517ed00e4ab3a8725caebd5fb7e857>” ended up in the code.
> It looks like an old file that we removed from the code base. Can you please remove it.
>
> Several files do not have an Apache header. Can you please add those.
> During the build it is checked that all files have such a header, otherwise the build fails.
> You can test this locally by running “mvn clean package”, this should list all the files where a header is missing, if there are any.
> You can also configure your IDE to automatically add a header for new files.
>
> Just write if you have a problem.
>
> I really like the idea of also integrating the TLS encryption. Do you already have an idea how this could be done?
> Is this possible with our current concepts or do we need an additional feature to support it properly?
>
> Philipp
>
>
> > On 26. Nov 2020, at 13:31, Marco Heyden <he...@gmail.com> wrote:
> >
> > Hey all,
> >
> > I added username/password authentication to the Kafka connect adapter and the Kafka sink. While the unauthenticated version is still available, we now have the additional option to publish and consume from a username/password secured broker. See also https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_plain.html for additional information.
> >
> > You find the PRs here:
> >
> > • https://github.com/apache/incubator-streampipes-extensions/pull/32
> > • https://github.com/apache/incubator-streampipes/pull/28
> >
> > Currently, however, username and password are sent in plaintext, thus I will add TLS encryption in the near future.
> >
> > Best
> > Marco
>
Re: Kafka Authentication PR
Posted by Philipp Zehnder <ze...@apache.org>.
Hi Marco,
thanks a lot for your PRs, the feature is very cool and the code looks good.
There are two minor issues we have to fix before we can merge it.
It seems that there was a merge problem and somehow the file “streampipes-app-file-export/pom.xml <https://github.com/apache/incubator-streampipes/pull/28/files#diff-1dc29a04e5d3709e738a854d95de8a3f26517ed00e4ab3a8725caebd5fb7e857>” ended up in the code.
It looks like an old file that we removed from the code base. Can you please remove it.
Several files do not have an Apache header. Can you please add those.
During the build it is checked that all files have such a header, otherwise the build fails.
You can test this locally by running “mvn clean package”, this should list all the files where a header is missing, if there are any.
You can also configure your IDE to automatically add a header for new files.
Just write if you have a problem.
I really like the idea of also integrating the TLS encryption. Do you already have an idea how this could be done?
Is this possible with our current concepts or do we need an additional feature to support it properly?
Philipp
> On 26. Nov 2020, at 13:31, Marco Heyden <he...@gmail.com> wrote:
>
> Hey all,
>
> I added username/password authentication to the Kafka connect adapter and the Kafka sink. While the unauthenticated version is still available, we now have the additional option to publish and consume from a username/password secured broker. See also https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_plain.html for additional information.
>
> You find the PRs here:
>
> • https://github.com/apache/incubator-streampipes-extensions/pull/32
> • https://github.com/apache/incubator-streampipes/pull/28
>
> Currently, however, username and password are sent in plaintext, thus I will add TLS encryption in the near future.
>
> Best
> Marco