You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Yiannis Gkoufas <jo...@gmail.com> on 2016/12/10 11:08:24 UTC
Nosuchmethod Error for Netty
Hi there,
I have been trying to do a bulk load in HBase 2.0.0-SNAPSHOT using Spark
1.6.3.
I followed the example in BulkLoadSuite, while the test works ok, I cannot
make it work in my cluster.
The error I have been getting has to do with a netty dependency conflict I
guess because I get:
no such method bytebuf retainedDuplicate
Would really appreciate any hints.
Thanks!
Re: Nosuchmethod Error for Netty
Posted by Ted Yu <yu...@gmail.com>.
For 2.0.0-SNAPSHOT branch, there is more work w.r.t. the version of
netty-all dependency.
The workaround you outlined is fine in terms of loading data.
But how do you plan to query your data (via DataFrame, e.g.) ?
On Wed, Dec 14, 2016 at 9:24 AM, yiannis.gkoufas <jo...@gmail.com>
wrote:
> Hi Ted,
>
> sorry if it's a stupid question, but the patch attached to that issue is
> applicable only on branch-1 right?
> Since I am working on the 2.0.0-SNAPSHOT branch, it cannot be applied
> right?
> In any case, the workaround that I did is to use
> HFileOutputFormat2.configureIncrementalLoadMap and then call
> hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles from the
> shell.
>
> Thanks!
>
>
>
> --
> View this message in context: http://apache-hbase.679495.n3.
> nabble.com/Nosuchmethod-Error-for-Netty-tp4085028p4085132.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>
Re: Nosuchmethod Error for Netty
Posted by "yiannis.gkoufas" <jo...@gmail.com>.
Hi Ted,
sorry if it's a stupid question, but the patch attached to that issue is
applicable only on branch-1 right?
Since I am working on the 2.0.0-SNAPSHOT branch, it cannot be applied right?
In any case, the workaround that I did is to use
HFileOutputFormat2.configureIncrementalLoadMap and then call
hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles from the
shell.
Thanks!
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Nosuchmethod-Error-for-Netty-tp4085028p4085132.html
Sent from the HBase Developer mailing list archive at Nabble.com.
Re: Nosuchmethod Error for Netty
Posted by Ted Yu <yu...@gmail.com>.
Just attached the backport to HBASE-14160
FYI
On Sun, Dec 11, 2016 at 4:17 PM, yiannis.gkoufas <jo...@gmail.com>
wrote:
> Thanks Ted! That would be really helpful!
>
>
>
> --
> View this message in context: http://apache-hbase.679495.n3.
> nabble.com/Nosuchmethod-Error-for-Netty-tp4085028p4085052.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>
Re: Nosuchmethod Error for Netty
Posted by "yiannis.gkoufas" <jo...@gmail.com>.
Thanks Ted! That would be really helpful!
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Nosuchmethod-Error-for-Netty-tp4085028p4085052.html
Sent from the HBase Developer mailing list archive at Nabble.com.
Re: Nosuchmethod Error for Netty
Posted by Ted Yu <yu...@gmail.com>.
I happened to have generated a backport of hbase-spark module to hbase
branch-1 the past Friday.
Let me see if I can upload it somewhere.
On Sun, Dec 11, 2016 at 1:18 PM, yiannis.gkoufas <jo...@gmail.com>
wrote:
> Hi Ted,
>
> thanks for the reply.
> I found out as well about the 4.0.x dependency, but updating it to 4.1.x it
> makes the compilation fail.
> Do you think there is no way to work around that?
> Even Spark 2.2.0-Snapshot is on 4.0.42.Final.
>
> Thanks!
>
>
>
> --
> View this message in context: http://apache-hbase.679495.n3.
> nabble.com/Nosuchmethod-Error-for-Netty-tp4085028p4085050.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>
Re: Nosuchmethod Error for Netty
Posted by "yiannis.gkoufas" <jo...@gmail.com>.
Hi Ted,
thanks for the reply.
I found out as well about the 4.0.x dependency, but updating it to 4.1.x it
makes the compilation fail.
Do you think there is no way to work around that?
Even Spark 2.2.0-Snapshot is on 4.0.42.Final.
Thanks!
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Nosuchmethod-Error-for-Netty-tp4085028p4085050.html
Sent from the HBase Developer mailing list archive at Nabble.com.
Re: Nosuchmethod Error for Netty
Posted by Ted Yu <yu...@gmail.com>.
From your description, it seems the error was similar to the one in the
description of HADOOP-13866 where netty-all-4.0.2?.Final.jar (from hdfs or
Spark) is ahead of 4.1.1.Final jar (used by hbase) in the classpath.
In pom.xml of Spark 1.6, you can see:
<artifactId>netty-all</artifactId>
<version>4.0.29.Final</version>
Please check the classpath and confirm.
Cheers
On Sat, Dec 10, 2016 at 3:08 AM, Yiannis Gkoufas <jo...@gmail.com>
wrote:
> Hi there,
>
> I have been trying to do a bulk load in HBase 2.0.0-SNAPSHOT using Spark
> 1.6.3.
> I followed the example in BulkLoadSuite, while the test works ok, I cannot
> make it work in my cluster.
> The error I have been getting has to do with a netty dependency conflict I
> guess because I get:
>
> no such method bytebuf retainedDuplicate
>
> Would really appreciate any hints.
>
> Thanks!
>