You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Xiaowei Wang (JIRA)" <ji...@apache.org> on 2015/11/02 12:19:27 UTC
[jira] [Commented] (HIVE-12303) HCatRecordSerDe throw a
IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/HIVE-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985077#comment-14985077 ]
Xiaowei Wang commented on HIVE-12303:
-------------------------------------
The schema is
{noformat}
# col_name data_type comment
ip string from deserializer
manualtime string from deserializer
timezone string from deserializer
pbparams map<string,string> from deserializer
pageurl string from deserializer
useragent string from deserializer
yyid string from deserializer
suv string from deserializer
line string from deserializer
applogs array<struct<productid:string,version:string,brand:string,model:string,device:string,product:string,sdk:string,os:string,deviceid:string,net:string,channel:string,w:string,h:string,imsi:string,source_channel:string,reserved_field0:string,reserved_field1:string,reserved_field2:string,reserved_field3:string,reserved_field4:string,type:string,parent:string,content:string,ts:string,extra:string>> from deserializer
# Partition Information
# col_name data_type comment
logdate string
# Detailed Table Information
Database: default
Owner: hive
CreateTime: Fri Nov 08 11:38:00 CST 2013
LastAccessTime: UNKNOWN
Protect Mode: None
Retention: 0
Location: viewfs://nsX/user/hive/warehouse/default.db/web/uigs/web_uigs_wapsearch
Table Type: EXTERNAL_TABLE
Table Parameters:
EXTERNAL TRUE
last_modified_by slave
last_modified_time 1414463853
transient_lastDdlTime 1414463853
# Storage Information
SerDe Library: com.custom.datacat.hive.DataCatSerde
InputFormat: com.custom.datadir.plugin.SymlinkLzoTextInputFormat
OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Compressed: No
Num Buckets: -1
Bucket Columns: []
Sort Columns: []
Storage Desc Params:
datacat.fieldInspector applogs:com.custom.datacat.hive.DataCatListObjectInspector:\t&pbparams:com.custom.datacat.hive.DataCatMapObjectInspector
datacat.lineInspector com.custom.datacat.wapapp.WapAppSearchInspector:
serialization.format 1
{noformat}
> HCatRecordSerDe throw a IndexOutOfBoundsException
> ----------------------------------------------------
>
> Key: HIVE-12303
> URL: https://issues.apache.org/jira/browse/HIVE-12303
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
> Affects Versions: 0.14.0, 1.2.1
> Reporter: Xiaowei Wang
> Assignee: Sushanth Sowmyan
> Fix For: 1.2.1
>
> Attachments: HIVE-12303.0.patch
>
>
> When access hive table using hcatlog in Pig,sometime it throws a exception !
> Exception
> {noformat}
> 2015-10-30 06:44:35,219 WARN [Thread-4] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error converting read value to tuple
> at org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
> at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:59)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:204)
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.IndexOutOfBoundsException: Index: 24, Size: 24
> at java.util.ArrayList.rangeCheck(ArrayList.java:635)
> at java.util.ArrayList.get(ArrayList.java:411)
> at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeStruct(HCatRecordSerDe.java:175)
> at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeList(HCatRecordSerDe.java:244)
> at org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:196)
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
> at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
> at org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:63)
> ... 13 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)