You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Anty <an...@gmail.com> on 2010/03/10 10:01:51 UTC
DynamicSerDe/TBinaryProtocol
Hi: ALL
I encounter a problem, any suggestion will be appreciated!
MY hive version is 0.30.0
I create a table in CLI.
CREATE TABLE table2 (boo int,bar string)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
WITH SERDEPROPERTIES (
'serialization.format'=org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol')
STORED AS TEXTFILE;
Then a load some data to table2.
INSERT OVERWRITE TABLE table2 SELECT foo,bar from pokes.
Everything is OK. Also , i can issue queries against table2.
But, when i change the protocol to TBinaryProtocol,
CREATE TABLE table1 (boo int,bar string)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
WITH SERDEPROPERTIES (
'serialization.format'='org.apache.thrift.protocol.TBinaryProtocol')
STORED AS TEXTFILE;
then load some data to table1 ,there is some error ,the loading process
can't be completed.
java.lang.RuntimeException: org.apache.hadoop.hive.serde2.SerDeException:
org.apache.thrift.transport.TTransportException: Cannot read. Remote side
has closed. Tried to read 1 bytes, but only got 0 bytes. at
org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:182) at
org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at
org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.serde2.SerDeException:
org.apache.thrift.transport.TTransportException: Cannot read. Remote side
has closed. Tried to read 1 bytes, but only got 0 bytes. at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:328) at
org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:165) ... 4
more Caused by: org.apache.hadoop.hive.serde2.SerDeException:
org.apache.thrift.transport.TTransportException: Cannot read. Remote side
has closed. Tried to read 1 bytes, but only got 0 bytes. at
org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.deserialize(DynamicSerDe.java:135)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:319)
... 5 more Caused by: org.apache.thrift.transport.TTransportException:
Cannot read. Remote side has closed. Tried to read 1 bytes, but only got 0
bytes. at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:314)
at
org.apache.thrift.protocol.TBinaryProtocol.readByte(TBinaryProtocol.java:247)
at
org.apache.thrift.protocol.TBinaryProtocol.readFieldBegin(TBinaryProtocol.java:216)
at
org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeFieldList.deserialize(DynamicSerDeFieldList.java:163)
at
org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeStructBase.deserialize(DynamicSerDeStructBase.java:59)
at
org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.deserialize(DynamicSerDe.java:131)
... 6 more
If there is something wrong with TBinaryProtocol?
--
Best Regards
Anty Rao
Re: DynamicSerDe/TBinaryProtocol
Posted by Zheng Shao <zs...@gmail.com>.
What is the format of your data?
TBinaryProtocol does not work with TextFile format, as you can imagine.
On 3/10/10, Anty <an...@gmail.com> wrote:
> Hi: ALL
>
> I encounter a problem, any suggestion will be appreciated!
> MY hive version is 0.30.0
> I create a table in CLI.
> CREATE TABLE table2 (boo int,bar string)
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
> WITH SERDEPROPERTIES (
> 'serialization.format'=org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol')
> STORED AS TEXTFILE;
> Then a load some data to table2.
> INSERT OVERWRITE TABLE table2 SELECT foo,bar from pokes.
> Everything is OK. Also , i can issue queries against table2.
>
> But, when i change the protocol to TBinaryProtocol,
>
>
> CREATE TABLE table1 (boo int,bar string)
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
> WITH SERDEPROPERTIES (
> 'serialization.format'='org.apache.thrift.protocol.TBinaryProtocol')
> STORED AS TEXTFILE;
>
> then load some data to table1 ,there is some error ,the loading process
> can't be completed.
>
> java.lang.RuntimeException: org.apache.hadoop.hive.serde2.SerDeException:
> org.apache.thrift.transport.TTransportException: Cannot read. Remote side
> has closed. Tried to read 1 bytes, but only got 0 bytes. at
> org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:182) at
> org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at
> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at
> org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.serde2.SerDeException:
> org.apache.thrift.transport.TTransportException: Cannot read. Remote side
> has closed. Tried to read 1 bytes, but only got 0 bytes. at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:328) at
> org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:165) ... 4
> more Caused by: org.apache.hadoop.hive.serde2.SerDeException:
> org.apache.thrift.transport.TTransportException: Cannot read. Remote side
> has closed. Tried to read 1 bytes, but only got 0 bytes. at
> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.deserialize(DynamicSerDe.java:135)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:319)
> ... 5 more Caused by: org.apache.thrift.transport.TTransportException:
> Cannot read. Remote side has closed. Tried to read 1 bytes, but only got 0
> bytes. at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:314)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readByte(TBinaryProtocol.java:247)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readFieldBegin(TBinaryProtocol.java:216)
> at
> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeFieldList.deserialize(DynamicSerDeFieldList.java:163)
> at
> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeStructBase.deserialize(DynamicSerDeStructBase.java:59)
> at
> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.deserialize(DynamicSerDe.java:131)
> ... 6 more
>
>
> If there is something wrong with TBinaryProtocol?
>
>
>
> --
> Best Regards
> Anty Rao
>
--
Sent from my mobile device
Yours,
Zheng