java.lang.reflect.InvocationTargetException when doing batch with large number of increment

[Keith Lui <Ke...@gmail.com>]

Hi,

Tried to do a 10K increment with batch. When using
public void batch(List<? extends Row> actions, Object[] results)
got
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.AssertionError: results.length
at
org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.<init>(AsyncProcess.java:763)
at
org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
at
org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)

Interestingly no exception is thrown when using the deprecated method
public Object[] batch(List<? extends Row> actions)

This is a sample code in Scala:

val table = connection.getTable(TableName.valueOf("test_table"))
val increments = for (i <- 0 until 10000) yield

{ val increment = new Increment(Random.nextDouble().toString.getBytes)
increment.addColumn(family, qualifier, Random.nextLong()) increment }

table.batch(increments, Array.empty[Object])

Same issue happens when running in Apache Spark Streaming which tries to
consume data from Kafka and writes to HBase.

java.lang.AssertionError: results.length
    at
org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.<init>(AsyncProcess.java:763)
    at
org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
    at
org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
    at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)

This error will not show up and run smoothly when using the deprecated
batch method.

Re: java.lang.reflect.InvocationTargetException when doing batch with large number of increment

[Ted Yu <yu...@gmail.com>]

Here is related code from AsyncProcess:

        if (results.length != actions.size()) {

          throw new AssertionError("results.length");

        }

It means that the length of results (0 in your case) is not the same as
number of Action<Row>'s

Please create results array with proper length.


Cheers

On Wed, Dec 16, 2015 at 12:01 PM, Keith Lui <Ke...@gmail.com> wrote:

> Hi,
>
> Tried to do a 10K increment with batch. When using
> public void batch(List<? extends Row> actions, Object[] results)
> got
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.simontuffs.onejar.Boot.run(Boot.java:340)
> at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.AssertionError: results.length
> at
>
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.<init>(AsyncProcess.java:763)
> at
>
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
> at
>
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
>
> Interestingly no exception is thrown when using the deprecated method
> public Object[] batch(List<? extends Row> actions)
>
> This is a sample code in Scala:
>
> val table = connection.getTable(TableName.valueOf("test_table"))
> val increments = for (i <- 0 until 10000) yield
>
> { val increment = new Increment(Random.nextDouble().toString.getBytes)
> increment.addColumn(family, qualifier, Random.nextLong()) increment }
>
> table.batch(increments, Array.empty[Object])
>
> Same issue happens when running in Apache Spark Streaming which tries to
> consume data from Kafka and writes to HBase.
>
> java.lang.AssertionError: results.length
>     at
>
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.<init>(AsyncProcess.java:763)
>     at
>
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
>     at
>
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
>     at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
>
> This error will not show up and run smoothly when using the deprecated
> batch method.
>