You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/03/09 18:00:58 UTC
[GitHub] [kafka] dongjoon-hyun opened a new pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
dongjoon-hyun opened a new pull request #10285:
URL: https://github.com/apache/kafka/pull/10285
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjoon-hyun edited a comment on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-796460374
Since the new features are not used, you may right. However, I believe the benefits are three-fold.
- There exists ZSTD side bug-fixes at ZSTD 1.4.9.
- There exists ZSTD JNI side memory optimization improvements at ZSTD JNI 1.4.8-5 ~ 1.4.8-7. (This includes some incompatible changes and recovery).
- I hope this will reduce the chance of future potential version conflict issues across Apache projects. It's important when some downstream project starts to use new feature.
- Apache Spark 3.2.0 will use ZSTD 1.4.9. (https://github.com/apache/spark/commit/ba7e525a114cba01735ba94dc6d3a55dc372a2fc)
- Apache Parquet 1.12.0 will use ZSTD 1.4.9 (https://github.com/apache/parquet-mr/commit/66ac28ce232c586626f53b72110276ea45ec5fa2)
- Apache Avro 1.10.3 will use ZSTD 1.4.9 (https://github.com/apache/avro/commit/806667cb2b18e45d16fff25514c17a0272db2b2d)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 merged pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
chia7712 merged pull request #10285:
URL: https://github.com/apache/kafka/pull/10285
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjoon-hyun commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-797250027
Thank you for review and merging, @chia7712 and all!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjoon-hyun commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-796479622
Thank you, @ijuma .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjoon-hyun commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-796392392
Hi, @ijuma . Could you review this PR please?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjoon-hyun commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-796460374
Since the new feature is not used. You may right. However, I believe the benefits are three-fold.
- There exists ZSTD side bug-fixes at ZSTD 1.4.9.
- There exists ZSTD JNI side memory optimization improvements at ZSTD JNI 1.4.8-5 ~ 1.4.8-7. (This includes some incompatible changes and recovery).
- I hope this will reduce the chance of future potential version conflict issues across Apache projects.
- Apache Spark 3.2.0 will use ZSTD 1.4.9. (https://github.com/apache/spark/commit/ba7e525a114cba01735ba94dc6d3a55dc372a2fc)
- Apache Parquet 1.12.0 will use ZSTD 1.4.9 (https://github.com/apache/parquet-mr/commit/66ac28ce232c586626f53b72110276ea45ec5fa2)
- Apache Avro 1.10.3 will use ZSTD 1.4.9 (https://github.com/apache/avro/commit/806667cb2b18e45d16fff25514c17a0272db2b2d)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjoon-hyun commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-795683313
Thank you, @chia7712 and @dongjinleekr .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-796471625
Sounds good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-797576186
@chia7712 worth cherry-picking to `2.8` if the aim is to align the versions with other projects that are releasing soon.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjinleekr commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjinleekr commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-795247219
@dongjoon-hyun Hi Hyun, it looks good to me. I love this update. :smile:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjoon-hyun edited a comment on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-796460374
Since the new features are not used, you may right. However, I believe the benefits are three-fold.
- There exists ZSTD side bug-fixes at ZSTD 1.4.9.
- There exists ZSTD JNI side memory optimization improvements at ZSTD JNI 1.4.8-5 ~ 1.4.8-7. (This includes some incompatible changes and recovery).
- I hope this will reduce the chance of future potential version conflict issues across Apache projects.
- Apache Spark 3.2.0 will use ZSTD 1.4.9. (https://github.com/apache/spark/commit/ba7e525a114cba01735ba94dc6d3a55dc372a2fc)
- Apache Parquet 1.12.0 will use ZSTD 1.4.9 (https://github.com/apache/parquet-mr/commit/66ac28ce232c586626f53b72110276ea45ec5fa2)
- Apache Avro 1.10.3 will use ZSTD 1.4.9 (https://github.com/apache/avro/commit/806667cb2b18e45d16fff25514c17a0272db2b2d)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
chia7712 commented on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-798047080
> worth cherry-picking to 2.8 if the aim is to align the versions with other projects that are releasing soon.
done!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] dongjoon-hyun edited a comment on pull request #10285: KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #10285:
URL: https://github.com/apache/kafka/pull/10285#issuecomment-796460374
Since the new features are not used, you may right. However, I believe the benefits are three-fold.
- There exists ZSTD side bug-fixes at ZSTD 1.4.9.
- There exists ZSTD JNI side memory optimization improvements at ZSTD JNI 1.4.8-5 ~ 1.4.8-7. (This includes some incompatible changes and recovery. So, 1.4.9 is more human-readable stable version number.).
- I hope this will reduce the chance of future potential version conflict issues across Apache projects. It's important when some downstream project starts to use new feature.
- Apache Spark 3.2.0 will use ZSTD 1.4.9. (https://github.com/apache/spark/commit/ba7e525a114cba01735ba94dc6d3a55dc372a2fc)
- Apache Parquet 1.12.0 will use ZSTD 1.4.9 (https://github.com/apache/parquet-mr/commit/66ac28ce232c586626f53b72110276ea45ec5fa2)
- Apache Avro 1.10.3 will use ZSTD 1.4.9 (https://github.com/apache/avro/commit/806667cb2b18e45d16fff25514c17a0272db2b2d)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org