TYPE_NAME col in mysql issue.

[satyajit vegesna <sa...@gmail.com>]

Hi All,

I am trying to create a table with a single column as below in hive-2.1.0,

CREATE TABLE sample( data struct<`array`:array<struct<
hitNumber:bigint,time:bigint,hour:bigint,minute:bigint,isSecure:boolean,
isInteraction:boolean,isEntrance:boolean,isExit:boolean,referer:binary,page:
struct<pagePath:binary,hostname:binary,pageTitle:
binary,searchKeyword:binary,searchCategory:binary>,transaction:struct<
transactionId:binary,transactionRevenue:bigint,transactionTax:bigint,
transactionShipping:bigint,affiliation:binary,currencyCode:binary,
localTransactionRevenue:bigint,localTransactionTax:bigint,
localTransactionShipping:bigint,transactionCoupon:binary>,item:struct<
transactionId:binary,productName:binary,productCategory:binary,
productSku:binary,itemQuantity:bigint,itemRevenue:bigint,
currencyCode:binary,localItemRevenue:bigint>,contentInfo:struct<
contentDescription:binary>,appInfo:struct<name:binary,
version:binary,id:binary,installerId:binary,appInstallerId:binary,appName:
binary,appVersion:binary,appId:binary,screenName:binary,landingScreenName:
binary,exitScreenName:binary,screenDepth:binary>,exceptionInfo:struct<
description:binary,isFatal:boolean>,eventInfo:struct<eventCategory:binary,
eventAction:binary,eventLabel:binary,eventValue:bigint>,
product:struct<`array`:array<struct<productSKU:binary,v2ProductName:binary,
v2ProductCategory:binary,productVariant:binary,productBrand:binary,
productRevenue:bigint,localProductRevenue:bigint,productPrice:bigint,
localProductPrice:bigint,productQuantity:bigint,productRefundAmount:bigint,
localProductRefundAmount:bigint,isImpression:boolean,
customDimensions:struct<`array`:array<struct<index:bigint,value:binary>>>,
customMetrics:struct<`array`:array<struct<index:bigint,value:bigint>>>,
productListName:binary,productListPosition:bigint>>>,
promotion:struct<`array`:array<struct<promoId:binary,promoName:binary,
promoCreative:binary,promoPosition:binary>>>,promotionActionInfo:struct<
promoIsView:boolean,promoIsClick:boolean>,refund:struct<refundAmount:bigint,
localRefundAmount:bigint>,eCommerceAction:struct<action_
type:binary,step:bigint,option:binary>,experiment:
struct<`array`:array<struct<experimentId:binary,experimentVariant:binary>>>,
publisher:struct<dfpClicks:bigint,dfpImpressions:bigint,
dfpMatchedQueries:bigint,dfpMeasurableImpressions:bigint,dfpQueries:bigint,
dfpRevenueCpm:bigint,dfpRevenueCpc:bigint,dfpViewableImpressions:bigint,
dfpPagesViewed:bigint,adsenseBackfillDfpClicks:bigint,
adsenseBackfillDfpImpressions:bigint,adsenseBackfillDfpMatchedQueri
es:bigint,adsenseBackfillDfpMeasurableImpressions:bigint,
adsenseBackfillDfpQueries:bigint,adsenseBackfillDfpRevenueCpm:bigint,
adsenseBackfillDfpRevenueCpc:bigint,adsenseBackfillDfpViewableImpr
essions:bigint,adsenseBackfillDfpPagesViewed:bigint,adxBackfillDfpClicks:
bigint,adxBackfillDfpImpressions:bigint,adxBackfillDfpMatchedQueries:bigint,
adxBackfillDfpMeasurableImpressions:bigint,adxBackfillDfpQueries:bigint,
adxBackfillDfpRevenueCpm:bigint,adxBackfillDfpRevenueCpc:bigint,
adxBackfillDfpViewableImpressions:bigint,adxBackfillDfpPagesViewed:
bigint,adxClicks:bigint,adxImpressions:bigint,adxMatchedQueries:bigint,
adxMeasurableImpressions:bigint,adxQueries:bigint,adxRevenue:bigint,
adxViewableImpressions:bigint,adxPagesViewed:bigint,adsViewed:bigint,
adsUnitsViewed:bigint,adsUnitsMatched:bigint,viewableAdsViewed:bigint,
measurableAdsViewed:bigint,adsPagesViewed:bigint,
adsClicked:bigint,adsRevenue:bigint,dfpAdGroup:binary,dfpAdUnits:binary,
dfpNetworkId:binary>,customVariables:struct<`array`
:array<struct<index:bigint,customVarName:binary,customVarValue:binary>>>,
customDimensions:struct<`array`:array<struct<index:bigint,value:binary>>>,
customMetrics:struct<`array`:array<struct<index:bigint,
value:bigint>>>,type:binary,social:struct<socialInteractionNetwork:binary,
socialInteractionAction:binary>>>>)
STORED AS parquet;

But hive throws, FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. InvalidObjectException(message:Invalid
column type name is too long: struct<.......

I understand the issue, as i have fixed the similar problem in hive 1.2.0
by increasing the TYPE_NAME col size in COLUMNS_V2 table, but the same
doesnt work in hive-2.1.0.
Additionally i have also inserted the same column into the COLUMNS_V2 mysql
metastore table and found out the insert was successful.

Would like to know if there is any workaround, any help is highly
appreciated.

Regards,
Satyajit.