You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/12/23 18:20:16 UTC
[GitHub] [spark] tobiasstadler opened a new pull request, #39196: [SPARK-41695][BUILD] Upgrade netty to 4.1.86.Final
tobiasstadler opened a new pull request, #39196:
URL: https://github.com/apache/spark/pull/39196
### What changes were proposed in this pull request?
Upgrade netty to 4.1.86.Final;
I cherry-picked
* 56086cb
* 6d3c1c1
* 7154fb9
* ad81ba8
* c95ed82
* c26d99e
* 24adac3
* ff68d0e
### Why are the changes needed?
CVEs
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bjornjorgensen commented on pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
bjornjorgensen commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1364733333
@tobiasstadler
Hi, I have enabled [SNYK](https://snyk.io) and Github Dependabot on my repo.
This way SNYK [are opening PR's](https://github.com/bjornjorgensen/spark/pulls) and [Dependabot are opening issues](https://github.com/bjornjorgensen/spark/security/dependabot).
Right now, most of these PR's are related to Hadoop 2.7 which we will remove in spark 3.5 or 3.6.
I also build spark master and scan code with sonar
[This one is for python and javascript](https://sonarcloud.io/project/issues?resolved=false&id=spark-python)
[and this is for](https://sonarcloud.io/project/issues?resolved=false&id=sparklocal) java, scala and xml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tobiasstadler commented on pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
tobiasstadler commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1364724834
Ok, I didn‘t read that. I guess it is better to close the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bjornjorgensen commented on pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
bjornjorgensen commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1364734128
Hmm.. you can't see the Dependabot page
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bjornjorgensen commented on pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
bjornjorgensen commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1365190148
My intentions are to explain to a new contributor how I do it and what tools I use. Sonar is built on best practice rules, the problem is that not everything hits equally well. Here are the ones that I have found using Sonar that have entered the code base.
https://github.com/apache/spark/pull/36465
https://github.com/apache/spark/pull/36383
https://github.com/apache/spark/pull/36364
https://github.com/apache/spark/pull/35775
https://github.com/apache/spark/pull/35755
https://github.com/apache/spark/pull/35741
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bjornjorgensen commented on pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
bjornjorgensen commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1364710814
https://github.com/apache/spark/commit/56086cb6c6df241e811be96cdf8bc6bc713b649d
### Does this PR introduce _any_ user-facing change?
Yes, all changes are caused by the change of Netty.
If the user wants to maintain the same behavior as the previous version, the user needs to make the following configuration:
- add `-Dio.netty.allocator.useCacheForAllThreads=true` to enable `useCacheForAllThreads `
- add `-Dio.netty.allocator.maxOrder=11` to keep the chunk size of `PooledByteBufAllocator` to 16m
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bjornjorgensen commented on pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
bjornjorgensen commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1364713687
And this one https://github.com/apache/spark/pull/38981
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1364946474
> @tobiasstadler Hi, I have enabled [SNYK](https://snyk.io) and Github Dependabot on my repo. This way SNYK [are opening PR's](https://github.com/bjornjorgensen/spark/pulls) and [Dependabot are opening issues](https://github.com/bjornjorgensen/spark/security/dependabot).
>
> Right now, most of these PR's are related to Hadoop 2.7 which we will remove in spark 3.5 or 3.6.
>
> I also build spark master and scan code with sonar
>
> [This one is for python and javascript](https://sonarcloud.io/project/issues?resolved=false&id=spark-python)
>
> [and this is for](https://sonarcloud.io/project/issues?resolved=false&id=sparklocal) java, scala and xml
>
> CC @panbingkun and @LuciferYang
Is the problem detected by sonar worth fixing @bjornjorgensen ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1364560339
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tobiasstadler closed pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
tobiasstadler closed pull request #39196: [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
URL: https://github.com/apache/spark/pull/39196
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] bjornjorgensen commented on pull request #39196: [SPARK-41695][BUILD] Upgrade netty to 4.1.86.Final
Posted by GitBox <gi...@apache.org>.
bjornjorgensen commented on PR #39196:
URL: https://github.com/apache/spark/pull/39196#issuecomment-1364510344
Change [SPARK-41695][BUILD] Upgrade netty to 4.1.86.Final to [SPARK-41695][BUILD][3.3] Upgrade netty to 4.1.86.Final
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org