You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "BryanCutler (via GitHub)" <gi...@apache.org> on 2023/06/21 18:47:05 UTC
[GitHub] [arrow] BryanCutler opened a new pull request, #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
BryanCutler opened a new pull request, #36211:
URL: https://github.com/apache/arrow/pull/36211
### Rationale for this change
Upgrading Netty dependency due to CVE https://github.com/advisories/GHSA-6mjq-h674-j845
This also requires a patch to arrow-memory
### What changes are included in this PR?
Upgrading Netty, gRPC and Protobuf dependencies
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
### Are these changes tested?
Existing tests
### Are there any user-facing changes?
No
<!--
Please uncomment the line below (and provide explanation) if the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld). We use this to highlight fixes to issues that may affect users without their knowledge. For this reason, fixing bugs that cause errors don't count, since those are usually obvious.
-->
**This PR contains a "Critical Fix".**
netty-handler SniHandler 16MB allocation
The SniHandler can allocate up to 16MB of heap for each channel during the TLS handshake. When the handler or the channel does not have an idle timeout, it can be used to make a TCP server using the SniHandler to allocate 16MB of heap.
https://github.com/advisories/GHSA-6mjq-h674-j845
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1601407065
I think 12.0.2 is unlikely, given the 13.0.0 code freeze is a few weeks away, but if you raise it on the ML we can see.
I think this can affect arrow flight, given gRPC uses Netty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1624132752
The code freeze is proposed to be July 10th (https://lists.apache.org/thread/f9r0dsd65ohdtcvc7fnnlfs23n3z0n7f). It would then be one to several weeks to chase down release blockers, prepare binaries, and vote on the release, depending on if any last-minute issues are found.
Generally releases are conducted quarterly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1626004854
I filed https://github.com/apache/arrow/issues/36562 to investigate this in the future.
(Though you gave me an idea; maybe we could make InnerAllocator here an interface and dispatch between a naive version and an internals-using version at runtime; presumably the JIT could devirtualize the call over time.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1603944469
Conbench analyzed the 6 benchmark runs on commit `ea4f03ac`.
There were 3 benchmark results indicating a performance regression:
- Commit Run on `ursa-thinkcentre-m75q` at [2023-06-22 21:47:35Z](http://conbench.ursa.dev/compare/runs/90a85de46b9b44928e2d68d05da1068f...7362d26f07184e31b8cdffbb09539c70/)
- [params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark](http://conbench.ursa.dev/compare/benchmarks/064949c371117dec8000128a87737dbf...06494c1bce3d7b4c8000d5464c925a0d)
- Commit Run on `arm64-m6g-linux-compute` at [2023-06-22 00:22:05Z](http://conbench.ursa.dev/compare/runs/4a6dbf76e5d6415ba39c0d6e8db76059...80aaaadba10f41179ee5e5b763849a2a/)
- [params=131072/2, source=cpp-micro, suite=arrow-bit-util-benchmark](http://conbench.ursa.dev/compare/benchmarks/0649384128007be08000596e01959584...06493946c4c27d81800042be0bf56db2)
- and 1 more (see the report linked below)
The [full Conbench report](https://github.com/apache/arrow/runs/14496511100) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] idelpivnitskiy commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "idelpivnitskiy (via GitHub)" <gi...@apache.org>.
idelpivnitskiy commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1625888920
Thanks for response @lidavidm,
I understand your general release cadence, but would like to highlight that this incompatibility does not allow upgrading Netty version for anything else in the same classpath. It will be great for the project if Arrow can find a way to either backport it to older versions (maybe `MethodHandler`s with runtime Netty version check can help), or find a way to stop using **internal** Netty API completely (preferred approach).
In the meanwhile, are there any known downsides or issues for users switching from `arrow-memory-netty` to `arrow-memory-unsafe`? I couldn't find much information about side-effects. Assuming that they are interchangeable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm merged pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm merged PR #36211:
URL: https://github.com/apache/arrow/pull/36211
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1625906803
We would love to be more agile with releases. However, it takes quite a bit of maintainer effort. Help is welcome.
@danepitkin was exploring if reflection could help. If you have ideas on how exactly that could work, that may be useful. I'm not familiar with why the code is structured like this, other than performance (but I'm not aware of benchmarks for this).
`arrow-memory-unsafe` has not received as much usage, in my estimation, because `netty` was for a long time the only implementation and is still the 'default' in effect. Also, if your application also uses Netty, the `unsafe` implementation is not aware of memory allocated by Netty and vice versa (may have to tweak the JVM native memory limit).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] BryanCutler commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "BryanCutler (via GitHub)" <gi...@apache.org>.
BryanCutler commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1601402020
@lidavidm this is a bit of a problem because Arrow memory needed to be patched to use the fixed version. Is there any discussion of doing a 12.0.2 release that could include this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] BryanCutler commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "BryanCutler (via GitHub)" <gi...@apache.org>.
BryanCutler commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1601589654
Thanks @lidavidm !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] idelpivnitskiy commented on pull request #36211: GH-36209: [Java] Upgrade Netty due to security vulnerability
Posted by "idelpivnitskiy (via GitHub)" <gi...@apache.org>.
idelpivnitskiy commented on PR #36211:
URL: https://github.com/apache/arrow/pull/36211#issuecomment-1624034800
What is ETA for 13.0.0 release?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org