You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Lydia Ickler <ic...@googlemail.com> on 2015/09/24 14:20:31 UTC
Connection closes prematurely
Hi all,
I am trying to get the HBaseReadExample from Apache Flink to run. I have filled a table with the HBaseWriteExample (that works great) and purposely split it over 3 regions.
Now when I try to read from it the first split seems to be scanned (170 rows) fine and after that the Connections of Zookeeper and RCP are suddenly closed down.
Does anyone has an idea why this is happening?
Best regards,
Lydia
22:28:10,178 DEBUG org.apache.flink.runtime.operators.DataSourceTask - Opening input split Locatable Split (2) at [grips5:60020]: DataSource (at createInput(ExecutionEnvironment.java:502) (org.apache.flink.HBaseReadExample$1)) (1/1)
22:28:10,178 INFO org.apache.flink.addons.hbase.TableInputFormat - opening split [2|[grips5:60020]|aaaaaaaa|-]
22:28:10,189 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null finished:false header:: 3,4 replyHeader:: 3,51539607639,0 request:: '/hbase/meta-region-server,F response:: #ffffffff0001a726567696f6e7365727665723a363030$
22:28:10,202 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null finished:false header:: 4,4 replyHeader:: 4,51539607639,0 request:: '/hbase/meta-region-server,F response:: #ffffffff0001a726567696f6e7365727665723a363030$
22:28:10,211 DEBUG LocalActorRefProvider(akka://flink) - resolve of path sequence [/temp/$b] failed
22:28:10,233 DEBUG org.apache.hadoop.hbase.util.ByteStringer - Failed to classload HBaseZeroCopyByteString: java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString
22:28:10,358 DEBUG org.apache.hadoop.ipc.RpcClient - Use SIMPLE authentication for service ClientService, sasl=false
22:28:10,370 DEBUG org.apache.hadoop.ipc.RpcClient - Connecting to grips1/130.73.20.14:60020
22:28:10,380 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips1/130.73.20.14:60020 from hduser: starting, connections 1
22:28:10,394 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips1/130.73.20.14:60020 from hduser: got response header call_id: 0, totalSize: 469 bytes
22:28:10,397 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips1/130.73.20.14:60020 from hduser: wrote request header call_id: 0 method_name: "Get" request_param: true
22:28:10,413 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null finished:false header:: 5,4 replyHeader:: 5,51539607639,0 request:: '/hbase/meta-region-server,F response:: #ffffffff0001a726567696f6e7365727665723a363030$
22:28:10,424 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips1/130.73.20.14:60020 from hduser: wrote request header call_id: 1 method_name: "Scan" request_param: true priority: 100
22:28:10,426 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips1/130.73.20.14:60020 from hduser: got response header call_id: 1 cell_block_meta { length: 480 }, totalSize: 497 bytes
22:28:10,432 DEBUG org.apache.hadoop.hbase.client.ClientSmallScanner - Finished with small scan at {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
22:28:10,434 DEBUG org.apache.hadoop.ipc.RpcClient - Use SIMPLE authentication for service ClientService, sasl=false
22:28:10,434 DEBUG org.apache.hadoop.ipc.RpcClient - Connecting to grips5/130.73.20.16:60020
22:28:10,435 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote request header call_id: 2 method_name: "Scan" request_param: true
22:28:10,436 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: starting, connections 2
22:28:10,437 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got response header call_id: 2, totalSize: 12 bytes
22:28:10,438 DEBUG org.apache.flink.runtime.operators.DataSourceTask - Starting to read input from split Locatable Split (2) at [grips5:60020]: DataSource (at createInput(ExecutionEnvironment.java:502) (org.apache.flink.HBaseReadExample$1)) (1/1)
22:28:10,438 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote request header call_id: 3 method_name: "Scan" request_param: true
22:28:10,457 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got response header call_id: 3 cell_block_meta { length: 4679 }, totalSize: 4899 bytes
22:28:10,476 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote request header call_id: 4 method_name: "Scan" request_param: true
22:28:10,480 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got response header call_id: 4 cell_block_meta { length: 3306 }, totalSize: 3466 bytes
22:28:10,482 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got response header call_id: 5, totalSize: 8 bytes
22:28:10,482 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote request header call_id: 5 method_name: "Scan" request_param: true
22:28:10,487 DEBUG org.apache.flink.runtime.operators.DataSourceTask - Closing input split Locatable Split (2) at [grips5:60020]: DataSource (at createInput(ExecutionEnvironment.java:502) (org.apache.flink.HBaseReadExample$1)) (1/1)
22:28:10,489 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Closing zookeeper sessionid=0x24ff6a96ecd000a
22:28:10,489 DEBUG org.apache.zookeeper.ZooKeeper - Closing session: 0x24ff6a96ecd000a
22:28:10,489 DEBUG org.apache.zookeeper.ClientCnxn - Closing client for session: 0x24ff6a96ecd000a
22:28:10,499 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null finished:false header:: 6,-11 replyHeader:: 6,51539607640,0 request:: null response:: null
22:28:10,499 DEBUG org.apache.zookeeper.ClientCnxn - Disconnecting client for session: 0x24ff6a96ecd000a
22:28:10,499 INFO org.apache.zookeeper.ClientCnxn - EventThread shut down
22:28:10,499 INFO org.apache.zookeeper.ZooKeeper - Session: 0x24ff6a96ecd000a closed
22:28:10,499 DEBUG org.apache.hadoop.ipc.RpcClient - Stopping rpc client
22:28:10,501 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips1/130.73.20.14:60020 from hduser: closed
22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips1/130.73.20.14:60020 from hduser: stopped, connections 0
22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: closed
22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client (2145423150) connection to grips5/130.73.20.16:60020 from hduser: stopped, connections 0
22:28:10,502 INFO org.apache.flink.addons.hbase.TableInputFormat - Closing split (scanned 170 rows)
22:28:10,508 DEBUG org.apache.flink.runtime.operators.DataSourceTask - Opening input split Locatable Split (1) at [grips4:60020]: DataSource (at createInput(ExecutionEnvironment.java:502) (org.apache.flink.HBaseReadExample$1)) (1/1)
22:28:10,509 INFO org.apache.flink.addons.hbase.TableInputFormat - opening split [1|[grips4:60020]|55555555|aaaaaaaa]
22:28:11,380 DEBUG org.apache.flink.runtime.taskmanager.TaskManager - Received message SendHeartbeat at akka://flink/user/taskmanager from Actor[akka://flink/deadLetters].
22:28:11,380 DEBUG org.apache.flink.runtime.taskmanager.TaskManager - Sending heartbeat to JobManager
Re: Connection closes prematurely
Posted by Ted Yu <yu...@gmail.com>.
See if this helps:
http://pastebin.com/s9e86JHW
On Thu, Sep 24, 2015 at 11:46 PM, Lydia Ickler <ic...@googlemail.com>
wrote:
>
>
> Hi Ted,
>
> I was running against hadoop-2.
> Yeah, the Hbase-protocol seems to ne missing?
> How would I add it?
>
> Best regards,
> Lydia
>
>
>
> > Am 24.09.2015 um 16:56 schrieb Ted Yu <yu...@gmail.com>:
> >
> > 22:28:10,233 DEBUG org.apache.hadoop.hbase.util.ByteStringer
> > - Failed to classload HBaseZeroCopyByteString:
> > java.lang.IllegalAccessError: class
> com.google.protobuf.HBaseZeroCopyByteString
> > cannot access its superclass com.google.protobuf.LiteralByteString
> >
> > Can you check the classpath ?
> > HBaseZeroCopyByteString is in hbase-protocol module.
> >
> > Were you running against hadoop-1 or hadoop-2 ?
> >
> > Cheers
> >
> > On Thu, Sep 24, 2015 at 5:20 AM, Lydia Ickler <ic...@googlemail.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> I am trying to get the HBaseReadExample from Apache Flink to run. I have
> >> filled a table with the HBaseWriteExample (that works great) and
> purposely
> >> split it over 3 regions.
> >> Now when I try to read from it the first split seems to be scanned (170
> >> rows) fine and after that the Connections of Zookeeper and RCP are
> suddenly
> >> closed down.
> >>
> >> Does anyone has an idea why this is happening?
> >>
> >> Best regards,
> >> Lydia
> >>
> >>
> >> 22:28:10,178 DEBUG org.apache.flink.runtime.operators.DataSourceTask
> >> - Opening input split Locatable Split (2) at [grips5:60020]:
> >> DataSource (at createInput(ExecutionEnvironment.java:502)
> >> (org.apache.flink.HBaseReadExample$1)) (1/1)
> >> 22:28:10,178 INFO org.apache.flink.addons.hbase.TableInputFormat
> >> - opening split [2|[grips5:60020]|aaaaaaaa|-]
> >> 22:28:10,189 DEBUG org.apache.zookeeper.ClientCnxn - Reading
> reply
> >> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
> >> finished:false header:: 3,4 replyHeader:: 3,51539607639,0 request::
> >> '/hbase/meta-region-server,F response::
> >> #ffffffff0001a726567696f6e7365727665723a363030$
> >> 22:28:10,202 DEBUG org.apache.zookeeper.ClientCnxn - Reading
> reply
> >> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
> >> finished:false header:: 4,4 replyHeader:: 4,51539607639,0 request::
> >> '/hbase/meta-region-server,F response::
> >> #ffffffff0001a726567696f6e7365727665723a363030$
> >> 22:28:10,211 DEBUG LocalActorRefProvider(akka://flink) - resolve of
> >> path sequence [/temp/$b] failed
> >> 22:28:10,233 DEBUG org.apache.hadoop.hbase.util.ByteStringer
> >> - Failed to classload HBaseZeroCopyByteString:
> >> java.lang.IllegalAccessError: class
> >> com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass
> >> com.google.protobuf.LiteralByteString
> >> 22:28:10,358 DEBUG org.apache.hadoop.ipc.RpcClient - Use SIMPLE
> >> authentication for service ClientService, sasl=false
> >> 22:28:10,370 DEBUG org.apache.hadoop.ipc.RpcClient - Connecting
> to
> >> grips1/130.73.20.14:60020
> >> 22:28:10,380 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips1/130.73.20.14:60020 from hduser:
> >> starting, connections 1
> >> 22:28:10,394 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: got
> >> response header call_id: 0, totalSize: 469 bytes
> >> 22:28:10,397 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: wrote
> >> request header call_id: 0 method_name: "Get" request_param: true
> >> 22:28:10,413 DEBUG org.apache.zookeeper.ClientCnxn - Reading
> reply
> >> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
> >> finished:false header:: 5,4 replyHeader:: 5,51539607639,0 request::
> >> '/hbase/meta-region-server,F response::
> >> #ffffffff0001a726567696f6e7365727665723a363030$
> >> 22:28:10,424 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: wrote
> >> request header call_id: 1 method_name: "Scan" request_param: true
> priority:
> >> 100
> >> 22:28:10,426 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: got
> >> response header call_id: 1 cell_block_meta { length: 480 }, totalSize:
> 497
> >> bytes
> >> 22:28:10,432 DEBUG org.apache.hadoop.hbase.client.ClientSmallScanner
> >> - Finished with small scan at {ENCODED => 1588230740, NAME =>
> >> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
> >> 22:28:10,434 DEBUG org.apache.hadoop.ipc.RpcClient - Use SIMPLE
> >> authentication for service ClientService, sasl=false
> >> 22:28:10,434 DEBUG org.apache.hadoop.ipc.RpcClient - Connecting
> to
> >> grips5/130.73.20.16:60020
> >> 22:28:10,435 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
> >> request header call_id: 2 method_name: "Scan" request_param: true
> >> 22:28:10,436 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser:
> >> starting, connections 2
> >> 22:28:10,437 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
> >> response header call_id: 2, totalSize: 12 bytes
> >> 22:28:10,438 DEBUG org.apache.flink.runtime.operators.DataSourceTask
> >> - Starting to read input from split Locatable Split (2) at
> >> [grips5:60020]: DataSource (at
> createInput(ExecutionEnvironment.java:502)
> >> (org.apache.flink.HBaseReadExample$1)) (1/1)
> >> 22:28:10,438 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
> >> request header call_id: 3 method_name: "Scan" request_param: true
> >> 22:28:10,457 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
> >> response header call_id: 3 cell_block_meta { length: 4679 }, totalSize:
> >> 4899 bytes
> >> 22:28:10,476 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
> >> request header call_id: 4 method_name: "Scan" request_param: true
> >> 22:28:10,480 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
> >> response header call_id: 4 cell_block_meta { length: 3306 }, totalSize:
> >> 3466 bytes
> >> 22:28:10,482 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
> >> response header call_id: 5, totalSize: 8 bytes
> >> 22:28:10,482 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
> >> request header call_id: 5 method_name: "Scan" request_param: true
> >> 22:28:10,487 DEBUG org.apache.flink.runtime.operators.DataSourceTask
> >> - Closing input split Locatable Split (2) at [grips5:60020]:
> >> DataSource (at createInput(ExecutionEnvironment.java:502)
> >> (org.apache.flink.HBaseReadExample$1)) (1/1)
> >> 22:28:10,489 INFO
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation
> >> - Closing zookeeper sessionid=0x24ff6a96ecd000a
> >> 22:28:10,489 DEBUG org.apache.zookeeper.ZooKeeper - Closing
> >> session: 0x24ff6a96ecd000a
> >> 22:28:10,489 DEBUG org.apache.zookeeper.ClientCnxn - Closing
> >> client for session: 0x24ff6a96ecd000a
> >> 22:28:10,499 DEBUG org.apache.zookeeper.ClientCnxn - Reading
> reply
> >> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
> >> finished:false header:: 6,-11 replyHeader:: 6,51539607640,0 request::
> >> null response:: null
> >> 22:28:10,499 DEBUG org.apache.zookeeper.ClientCnxn -
> Disconnecting
> >> client for session: 0x24ff6a96ecd000a
> >> 22:28:10,499 INFO org.apache.zookeeper.ClientCnxn - EventThread
> >> shut down
> >> 22:28:10,499 INFO org.apache.zookeeper.ZooKeeper - Session:
> >> 0x24ff6a96ecd000a closed
> >> 22:28:10,499 DEBUG org.apache.hadoop.ipc.RpcClient - Stopping
> rpc
> >> client
> >> 22:28:10,501 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips1/130.73.20.14:60020 from hduser:
> closed
> >> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips1/130.73.20.14:60020 from hduser:
> >> stopped, connections 0
> >> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser:
> closed
> >> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> >> (2145423150) connection to grips5/130.73.20.16:60020 from hduser:
> >> stopped, connections 0
> >> 22:28:10,502 INFO org.apache.flink.addons.hbase.TableInputFormat
> >> - Closing split (scanned 170 rows)
> >> 22:28:10,508 DEBUG org.apache.flink.runtime.operators.DataSourceTask
> >> - Opening input split Locatable Split (1) at [grips4:60020]:
> >> DataSource (at createInput(ExecutionEnvironment.java:502)
> >> (org.apache.flink.HBaseReadExample$1)) (1/1)
> >> 22:28:10,509 INFO org.apache.flink.addons.hbase.TableInputFormat
> >> - opening split [1|[grips4:60020]|55555555|aaaaaaaa]
> >> 22:28:11,380 DEBUG org.apache.flink.runtime.taskmanager.TaskManager
> >> - Received message SendHeartbeat at akka://flink/user/taskmanager
> >> from Actor[akka://flink/deadLetters].
> >> 22:28:11,380 DEBUG org.apache.flink.runtime.taskmanager.TaskManager
> >> - Sending heartbeat to JobManager
> >>
> >>
>
Re: Connection closes prematurely
Posted by Lydia Ickler <ic...@googlemail.com>.
Hi Ted,
I was running against hadoop-2.
Yeah, the Hbase-protocol seems to ne missing?
How would I add it?
Best regards,
Lydia
> Am 24.09.2015 um 16:56 schrieb Ted Yu <yu...@gmail.com>:
>
> 22:28:10,233 DEBUG org.apache.hadoop.hbase.util.ByteStringer
> - Failed to classload HBaseZeroCopyByteString:
> java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString
> cannot access its superclass com.google.protobuf.LiteralByteString
>
> Can you check the classpath ?
> HBaseZeroCopyByteString is in hbase-protocol module.
>
> Were you running against hadoop-1 or hadoop-2 ?
>
> Cheers
>
> On Thu, Sep 24, 2015 at 5:20 AM, Lydia Ickler <ic...@googlemail.com>
> wrote:
>
>> Hi all,
>>
>> I am trying to get the HBaseReadExample from Apache Flink to run. I have
>> filled a table with the HBaseWriteExample (that works great) and purposely
>> split it over 3 regions.
>> Now when I try to read from it the first split seems to be scanned (170
>> rows) fine and after that the Connections of Zookeeper and RCP are suddenly
>> closed down.
>>
>> Does anyone has an idea why this is happening?
>>
>> Best regards,
>> Lydia
>>
>>
>> 22:28:10,178 DEBUG org.apache.flink.runtime.operators.DataSourceTask
>> - Opening input split Locatable Split (2) at [grips5:60020]:
>> DataSource (at createInput(ExecutionEnvironment.java:502)
>> (org.apache.flink.HBaseReadExample$1)) (1/1)
>> 22:28:10,178 INFO org.apache.flink.addons.hbase.TableInputFormat
>> - opening split [2|[grips5:60020]|aaaaaaaa|-]
>> 22:28:10,189 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
>> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
>> finished:false header:: 3,4 replyHeader:: 3,51539607639,0 request::
>> '/hbase/meta-region-server,F response::
>> #ffffffff0001a726567696f6e7365727665723a363030$
>> 22:28:10,202 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
>> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
>> finished:false header:: 4,4 replyHeader:: 4,51539607639,0 request::
>> '/hbase/meta-region-server,F response::
>> #ffffffff0001a726567696f6e7365727665723a363030$
>> 22:28:10,211 DEBUG LocalActorRefProvider(akka://flink) - resolve of
>> path sequence [/temp/$b] failed
>> 22:28:10,233 DEBUG org.apache.hadoop.hbase.util.ByteStringer
>> - Failed to classload HBaseZeroCopyByteString:
>> java.lang.IllegalAccessError: class
>> com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass
>> com.google.protobuf.LiteralByteString
>> 22:28:10,358 DEBUG org.apache.hadoop.ipc.RpcClient - Use SIMPLE
>> authentication for service ClientService, sasl=false
>> 22:28:10,370 DEBUG org.apache.hadoop.ipc.RpcClient - Connecting to
>> grips1/130.73.20.14:60020
>> 22:28:10,380 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips1/130.73.20.14:60020 from hduser:
>> starting, connections 1
>> 22:28:10,394 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: got
>> response header call_id: 0, totalSize: 469 bytes
>> 22:28:10,397 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: wrote
>> request header call_id: 0 method_name: "Get" request_param: true
>> 22:28:10,413 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
>> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
>> finished:false header:: 5,4 replyHeader:: 5,51539607639,0 request::
>> '/hbase/meta-region-server,F response::
>> #ffffffff0001a726567696f6e7365727665723a363030$
>> 22:28:10,424 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: wrote
>> request header call_id: 1 method_name: "Scan" request_param: true priority:
>> 100
>> 22:28:10,426 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: got
>> response header call_id: 1 cell_block_meta { length: 480 }, totalSize: 497
>> bytes
>> 22:28:10,432 DEBUG org.apache.hadoop.hbase.client.ClientSmallScanner
>> - Finished with small scan at {ENCODED => 1588230740, NAME =>
>> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
>> 22:28:10,434 DEBUG org.apache.hadoop.ipc.RpcClient - Use SIMPLE
>> authentication for service ClientService, sasl=false
>> 22:28:10,434 DEBUG org.apache.hadoop.ipc.RpcClient - Connecting to
>> grips5/130.73.20.16:60020
>> 22:28:10,435 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
>> request header call_id: 2 method_name: "Scan" request_param: true
>> 22:28:10,436 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser:
>> starting, connections 2
>> 22:28:10,437 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
>> response header call_id: 2, totalSize: 12 bytes
>> 22:28:10,438 DEBUG org.apache.flink.runtime.operators.DataSourceTask
>> - Starting to read input from split Locatable Split (2) at
>> [grips5:60020]: DataSource (at createInput(ExecutionEnvironment.java:502)
>> (org.apache.flink.HBaseReadExample$1)) (1/1)
>> 22:28:10,438 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
>> request header call_id: 3 method_name: "Scan" request_param: true
>> 22:28:10,457 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
>> response header call_id: 3 cell_block_meta { length: 4679 }, totalSize:
>> 4899 bytes
>> 22:28:10,476 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
>> request header call_id: 4 method_name: "Scan" request_param: true
>> 22:28:10,480 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
>> response header call_id: 4 cell_block_meta { length: 3306 }, totalSize:
>> 3466 bytes
>> 22:28:10,482 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
>> response header call_id: 5, totalSize: 8 bytes
>> 22:28:10,482 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
>> request header call_id: 5 method_name: "Scan" request_param: true
>> 22:28:10,487 DEBUG org.apache.flink.runtime.operators.DataSourceTask
>> - Closing input split Locatable Split (2) at [grips5:60020]:
>> DataSource (at createInput(ExecutionEnvironment.java:502)
>> (org.apache.flink.HBaseReadExample$1)) (1/1)
>> 22:28:10,489 INFO
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation
>> - Closing zookeeper sessionid=0x24ff6a96ecd000a
>> 22:28:10,489 DEBUG org.apache.zookeeper.ZooKeeper - Closing
>> session: 0x24ff6a96ecd000a
>> 22:28:10,489 DEBUG org.apache.zookeeper.ClientCnxn - Closing
>> client for session: 0x24ff6a96ecd000a
>> 22:28:10,499 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
>> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
>> finished:false header:: 6,-11 replyHeader:: 6,51539607640,0 request::
>> null response:: null
>> 22:28:10,499 DEBUG org.apache.zookeeper.ClientCnxn - Disconnecting
>> client for session: 0x24ff6a96ecd000a
>> 22:28:10,499 INFO org.apache.zookeeper.ClientCnxn - EventThread
>> shut down
>> 22:28:10,499 INFO org.apache.zookeeper.ZooKeeper - Session:
>> 0x24ff6a96ecd000a closed
>> 22:28:10,499 DEBUG org.apache.hadoop.ipc.RpcClient - Stopping rpc
>> client
>> 22:28:10,501 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: closed
>> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips1/130.73.20.14:60020 from hduser:
>> stopped, connections 0
>> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: closed
>> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
>> (2145423150) connection to grips5/130.73.20.16:60020 from hduser:
>> stopped, connections 0
>> 22:28:10,502 INFO org.apache.flink.addons.hbase.TableInputFormat
>> - Closing split (scanned 170 rows)
>> 22:28:10,508 DEBUG org.apache.flink.runtime.operators.DataSourceTask
>> - Opening input split Locatable Split (1) at [grips4:60020]:
>> DataSource (at createInput(ExecutionEnvironment.java:502)
>> (org.apache.flink.HBaseReadExample$1)) (1/1)
>> 22:28:10,509 INFO org.apache.flink.addons.hbase.TableInputFormat
>> - opening split [1|[grips4:60020]|55555555|aaaaaaaa]
>> 22:28:11,380 DEBUG org.apache.flink.runtime.taskmanager.TaskManager
>> - Received message SendHeartbeat at akka://flink/user/taskmanager
>> from Actor[akka://flink/deadLetters].
>> 22:28:11,380 DEBUG org.apache.flink.runtime.taskmanager.TaskManager
>> - Sending heartbeat to JobManager
>>
>>
Re: Connection closes prematurely
Posted by Ted Yu <yu...@gmail.com>.
22:28:10,233 DEBUG org.apache.hadoop.hbase.util.ByteStringer
- Failed to classload HBaseZeroCopyByteString:
java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString
cannot access its superclass com.google.protobuf.LiteralByteString
Can you check the classpath ?
HBaseZeroCopyByteString is in hbase-protocol module.
Were you running against hadoop-1 or hadoop-2 ?
Cheers
On Thu, Sep 24, 2015 at 5:20 AM, Lydia Ickler <ic...@googlemail.com>
wrote:
> Hi all,
>
> I am trying to get the HBaseReadExample from Apache Flink to run. I have
> filled a table with the HBaseWriteExample (that works great) and purposely
> split it over 3 regions.
> Now when I try to read from it the first split seems to be scanned (170
> rows) fine and after that the Connections of Zookeeper and RCP are suddenly
> closed down.
>
> Does anyone has an idea why this is happening?
>
> Best regards,
> Lydia
>
>
> 22:28:10,178 DEBUG org.apache.flink.runtime.operators.DataSourceTask
> - Opening input split Locatable Split (2) at [grips5:60020]:
> DataSource (at createInput(ExecutionEnvironment.java:502)
> (org.apache.flink.HBaseReadExample$1)) (1/1)
> 22:28:10,178 INFO org.apache.flink.addons.hbase.TableInputFormat
> - opening split [2|[grips5:60020]|aaaaaaaa|-]
> 22:28:10,189 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
> finished:false header:: 3,4 replyHeader:: 3,51539607639,0 request::
> '/hbase/meta-region-server,F response::
> #ffffffff0001a726567696f6e7365727665723a363030$
> 22:28:10,202 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
> finished:false header:: 4,4 replyHeader:: 4,51539607639,0 request::
> '/hbase/meta-region-server,F response::
> #ffffffff0001a726567696f6e7365727665723a363030$
> 22:28:10,211 DEBUG LocalActorRefProvider(akka://flink) - resolve of
> path sequence [/temp/$b] failed
> 22:28:10,233 DEBUG org.apache.hadoop.hbase.util.ByteStringer
> - Failed to classload HBaseZeroCopyByteString:
> java.lang.IllegalAccessError: class
> com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass
> com.google.protobuf.LiteralByteString
> 22:28:10,358 DEBUG org.apache.hadoop.ipc.RpcClient - Use SIMPLE
> authentication for service ClientService, sasl=false
> 22:28:10,370 DEBUG org.apache.hadoop.ipc.RpcClient - Connecting to
> grips1/130.73.20.14:60020
> 22:28:10,380 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips1/130.73.20.14:60020 from hduser:
> starting, connections 1
> 22:28:10,394 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: got
> response header call_id: 0, totalSize: 469 bytes
> 22:28:10,397 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: wrote
> request header call_id: 0 method_name: "Get" request_param: true
> 22:28:10,413 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
> finished:false header:: 5,4 replyHeader:: 5,51539607639,0 request::
> '/hbase/meta-region-server,F response::
> #ffffffff0001a726567696f6e7365727665723a363030$
> 22:28:10,424 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: wrote
> request header call_id: 1 method_name: "Scan" request_param: true priority:
> 100
> 22:28:10,426 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: got
> response header call_id: 1 cell_block_meta { length: 480 }, totalSize: 497
> bytes
> 22:28:10,432 DEBUG org.apache.hadoop.hbase.client.ClientSmallScanner
> - Finished with small scan at {ENCODED => 1588230740, NAME =>
> 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
> 22:28:10,434 DEBUG org.apache.hadoop.ipc.RpcClient - Use SIMPLE
> authentication for service ClientService, sasl=false
> 22:28:10,434 DEBUG org.apache.hadoop.ipc.RpcClient - Connecting to
> grips5/130.73.20.16:60020
> 22:28:10,435 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
> request header call_id: 2 method_name: "Scan" request_param: true
> 22:28:10,436 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser:
> starting, connections 2
> 22:28:10,437 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
> response header call_id: 2, totalSize: 12 bytes
> 22:28:10,438 DEBUG org.apache.flink.runtime.operators.DataSourceTask
> - Starting to read input from split Locatable Split (2) at
> [grips5:60020]: DataSource (at createInput(ExecutionEnvironment.java:502)
> (org.apache.flink.HBaseReadExample$1)) (1/1)
> 22:28:10,438 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
> request header call_id: 3 method_name: "Scan" request_param: true
> 22:28:10,457 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
> response header call_id: 3 cell_block_meta { length: 4679 }, totalSize:
> 4899 bytes
> 22:28:10,476 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
> request header call_id: 4 method_name: "Scan" request_param: true
> 22:28:10,480 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
> response header call_id: 4 cell_block_meta { length: 3306 }, totalSize:
> 3466 bytes
> 22:28:10,482 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: got
> response header call_id: 5, totalSize: 8 bytes
> 22:28:10,482 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: wrote
> request header call_id: 5 method_name: "Scan" request_param: true
> 22:28:10,487 DEBUG org.apache.flink.runtime.operators.DataSourceTask
> - Closing input split Locatable Split (2) at [grips5:60020]:
> DataSource (at createInput(ExecutionEnvironment.java:502)
> (org.apache.flink.HBaseReadExample$1)) (1/1)
> 22:28:10,489 INFO
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation
> - Closing zookeeper sessionid=0x24ff6a96ecd000a
> 22:28:10,489 DEBUG org.apache.zookeeper.ZooKeeper - Closing
> session: 0x24ff6a96ecd000a
> 22:28:10,489 DEBUG org.apache.zookeeper.ClientCnxn - Closing
> client for session: 0x24ff6a96ecd000a
> 22:28:10,499 DEBUG org.apache.zookeeper.ClientCnxn - Reading reply
> sessionid:0x24ff6a96ecd000a, packet:: clientPath:null serverPath:null
> finished:false header:: 6,-11 replyHeader:: 6,51539607640,0 request::
> null response:: null
> 22:28:10,499 DEBUG org.apache.zookeeper.ClientCnxn - Disconnecting
> client for session: 0x24ff6a96ecd000a
> 22:28:10,499 INFO org.apache.zookeeper.ClientCnxn - EventThread
> shut down
> 22:28:10,499 INFO org.apache.zookeeper.ZooKeeper - Session:
> 0x24ff6a96ecd000a closed
> 22:28:10,499 DEBUG org.apache.hadoop.ipc.RpcClient - Stopping rpc
> client
> 22:28:10,501 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips1/130.73.20.14:60020 from hduser: closed
> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips1/130.73.20.14:60020 from hduser:
> stopped, connections 0
> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser: closed
> 22:28:10,502 DEBUG org.apache.hadoop.ipc.RpcClient - IPC Client
> (2145423150) connection to grips5/130.73.20.16:60020 from hduser:
> stopped, connections 0
> 22:28:10,502 INFO org.apache.flink.addons.hbase.TableInputFormat
> - Closing split (scanned 170 rows)
> 22:28:10,508 DEBUG org.apache.flink.runtime.operators.DataSourceTask
> - Opening input split Locatable Split (1) at [grips4:60020]:
> DataSource (at createInput(ExecutionEnvironment.java:502)
> (org.apache.flink.HBaseReadExample$1)) (1/1)
> 22:28:10,509 INFO org.apache.flink.addons.hbase.TableInputFormat
> - opening split [1|[grips4:60020]|55555555|aaaaaaaa]
> 22:28:11,380 DEBUG org.apache.flink.runtime.taskmanager.TaskManager
> - Received message SendHeartbeat at akka://flink/user/taskmanager
> from Actor[akka://flink/deadLetters].
> 22:28:11,380 DEBUG org.apache.flink.runtime.taskmanager.TaskManager
> - Sending heartbeat to JobManager
>
>