You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2020/05/27 09:04:45 UTC
[GitHub] [nifi] gadLinux opened a new pull request #4300: NIfI-7431 kafka exitcompression type
gadLinux opened a new pull request #4300:
URL: https://github.com/apache/nifi/pull/4300
#### Description of PR
Enables de client 2.1 of kafka that supports new compression types and features.
The original issue is here.
https://issues.apache.org/jira/browse/NIFI-7431
Adds new components supporting this kafka 2.1 client. Tested in lab.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] github-actions[bot] closed pull request #4300: NIFI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #4300:
URL: https://github.com/apache/nifi/pull/4300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #4300: NIFI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-826170803
We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] gadLinux commented on pull request #4300: NIFI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
gadLinux commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-691492193
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4300: NIfI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
pvillard31 commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-634610385
Thanks for the contribution @gadLinux!
IIRC we introduced separate bundles to make sure we don't face any compatibility issue against Kafka between different versions. However, if there is no backward incompatibility with the Kafka clients (meaning that any 2.x client can talk to any 2.x broker - we would need to double check), I'd recommend to just keep using the existing ``nifi-kafka-2-0-nar`` and update the client version and add any property that would make sense. We could even bump the version to 2.5.0 if there is no backward compatibility issue.
@markap14 - what are your thoughts on this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4300: NIfI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
pvillard31 commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-636308035
Yeah I totally understand. I believe that any client 2.x would work with lower versions on the broker side as long as the user does not use properties that are new and not interpreted by the broker. I don't know what is best between bumping the version and having a dedicated bundle. If we go with the bundle, I think it'd be best to use the latest version available at this time. I defer to @markap14 on this one as he worked a lot on the Kafka processors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #4300: NIFI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-826170803
We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] gadLinux commented on pull request #4300: NIfI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
gadLinux commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-636301954
Hi,
But client 2.0 doesn't have proper support. And it doesn't work with current installation. So I had to bump the client. I can bump the client over 2.0 if you want. This solution will be easy but I don't know if it can break anything because transitive deps.
We can try.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] gadLinux edited a comment on pull request #4300: NIfI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
gadLinux edited a comment on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-636301954
Hi,
But client 2.0 doesn't have proper support. And it doesn't work with current installation. So I had to bump the client. I can bump the kafka client to 2.1 over 2.0 bundle if you want. But this will confuse people I think. Since 2.0 bundle will have 2.1 client. If this is okay we can give it a try.
This solution will be easy but I don't know if it can break anything because transitive deps.
We can try.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] gadLinux commented on pull request #4300: NIFI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
gadLinux commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-691492193
I rebased this branch, maybe we should decide what to do...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] gadLinux commented on pull request #4300: NIFI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
gadLinux commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-691492193
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #4300: NIFI-7431 kafka exitcompression type
Posted by GitBox <gi...@apache.org>.
markap14 commented on pull request #4300:
URL: https://github.com/apache/nifi/pull/4300#issuecomment-667180566
I do think it's probably better to have a separate bundle. Just because it clearly conveys the intent. It's also the case that while a newer server may be compatible with an older client, it doesn't mean that it works *well*. We've seen this in the past with Kafka where using a 1.0 client with a 1.1 server (it's entirely possible that these are not the correct versions, just using this as an example) worked but the performance was about an order of magnitude slower than using 1.0 client with 1.0 server or 1.1 client with 1.1 server. It had to do with upgrading/downgrading the protocol and was super expensive. So best to stick with matching versions. Also of note, there is a PR up already (https://github.com/apache/nifi/pull/4383) for Kafka 2.5 processors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org