You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Adam Roberts (JIRA)" <ji...@apache.org> on 2016/09/02 14:18:21 UTC
[jira] [Comment Edited] (SPARK-17379) Upgrade netty-all to
4.1.5.Final
[ https://issues.apache.org/jira/browse/SPARK-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458656#comment-15458656 ]
Adam Roberts edited comment on SPARK-17379 at 9/2/16 2:18 PM:
--------------------------------------------------------------
Good point about the 1.6 stream, this change isn't as trivial as other version upgrades: I kicked off a build with our own testing farm together with the snappy-java version upgrade, noticed the below compile errors.
Without the netty version change we're back to building fine, so looks like the API has changed between 4.0.29 and 4.1.5.
Snippet of the console log below, deprecated methods, MessageWithHeader and EncryptedMessage abstract class errors to handle:
```
[INFO] Compiling 1 Scala source and 5 Java sources to /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/unsafe/target/scala-2.11/test-classes...
[ERROR] warning: [options] bootstrap class path not set in conjunction with -source 1.7
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java:143: warning: [deprecation] group() in AbstractBootstrap has been deprecated
[ERROR] if (bootstrap != null && bootstrap.group() != null) {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java:144: warning: [deprecation] group() in AbstractBootstrap has been deprecated
[ERROR] bootstrap.group().shutdownGracefully();
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java:146: warning: [deprecation] childGroup() in ServerBootstrap has been deprecated
[ERROR] if (bootstrap != null && bootstrap.childGroup() != null) {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java:147: warning: [deprecation] childGroup() in ServerBootstrap has been deprecated
[ERROR] bootstrap.childGroup().shutdownGracefully();
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/protocol/MessageWithHeader.java:38: error: MessageWithHeader is not abstract and does not override abstract method touch(Object) in ReferenceCounted
[ERROR] class MessageWithHeader extends AbstractReferenceCounted implements FileRegion {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/protocol/MessageWithHeader.java:94: warning: [deprecation] transfered() in FileRegion has been deprecated
[ERROR] public long transfered() {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/sasl/SaslEncryption.java:132: error: EncryptedMessage is not abstract and does not override abstract method touch(Object) in ReferenceCounted
[ERROR] static class EncryptedMessage extends AbstractReferenceCounted implements FileRegion {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/sasl/SaslEncryption.java:186: warning: [deprecation] transfered() in FileRegion has been deprecated
[ERROR] public long transfered() {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/sasl/SaslEncryption.java:265: warning: [deprecation] transfered() in FileRegion has been deprecated
[ERROR] region.transferTo(byteChannel, region.transfered());
[ERROR] ^
[ERROR] 2 errors
[ERROR] 8 warnings
```
was (Author: aroberts):
Good point about the 1.6 stream, this change isn't as trivial as other version upgrades: I kicked off a build with our own testing farm together with the snappy-java version upgrade, noticed the below compile errors.
Without the netty version change we're back to building fine, so looks like the API has changed between 4.0.29 and 4.1.5.
Snippet of the console log below, deprecated methods, MessageWithHeader and EncryptedMessage abstract class errors to handle:
```
[INFO] Compiling 1 Scala source and 5 Java sources to /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/unsafe/target/scala-2.11/test-classes...
[ERROR] warning: [options] bootstrap class path not set in conjunction with -source 1.7
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java:143: warning: [deprecation] group() in AbstractBootstrap has been deprecated
[ERROR] if (bootstrap != null && bootstrap.group() != null) {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java:144: warning: [deprecation] group() in AbstractBootstrap has been deprecated
[ERROR] bootstrap.group().shutdownGracefully();
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java:146: warning: [deprecation] childGroup() in ServerBootstrap has been deprecated
[ERROR] if (bootstrap != null && bootstrap.childGroup() != null) {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/server/TransportServer.java:147: warning: [deprecation] childGroup() in ServerBootstrap has been deprecated
[ERROR] bootstrap.childGroup().shutdownGracefully();
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/protocol/MessageWithHeader.java:38: error: MessageWithHeader is not abstract and does not override abstract method touch(Object) in ReferenceCounted
[ERROR] class MessageWithHeader extends AbstractReferenceCounted implements FileRegion {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/protocol/MessageWithHeader.java:94: warning: [deprecation] transfered() in FileRegion has been deprecated
[ERROR] public long transfered() {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/sasl/SaslEncryption.java:132: error: EncryptedMessage is not abstract and does not override abstract method touch(Object) in ReferenceCounted
[ERROR] static class EncryptedMessage extends AbstractReferenceCounted implements FileRegion {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/sasl/SaslEncryption.java:186: warning: [deprecation] transfered() in FileRegion has been deprecated
[ERROR] public long transfered() {
[ERROR] ^
[ERROR] /home/jenkins/workspace/Spark/GIT_BRANCH/user-defined/label/PPC64LE/common/network-common/src/main/java/org/apache/spark/network/sasl/SaslEncryption.java:265: warning: [deprecation] transfered() in FileRegion has been deprecated
[ERROR] region.transferTo(byteChannel, region.transfered());
[ERROR] ^
[ERROR] 2 errors
[ERROR] 8 warnings
```
> Upgrade netty-all to 4.1.5.Final
> --------------------------------
>
> Key: SPARK-17379
> URL: https://issues.apache.org/jira/browse/SPARK-17379
> Project: Spark
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Adam Roberts
> Priority: Minor
>
> We should use the newest version of netty based on info here: http://netty.io/news/2016/08/29/4-0-41-Final-4-1-5-Final.html, especially interested in the static initialiser deadlock fix: https://github.com/netty/netty/pull/5730
> Lots more fixes mentioned so will create the pull request - again a case of updating the pom and then the dependency files
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org