[jira] [Assigned] (HIVE-13203) STRUCT type not allowing more than 266 fields. Table get created with more fields but throws error while querying against the table. SHOW CREATE TABLE skips having some fields in STRUCT.

["VISHNU S NAIR (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

VISHNU S NAIR reassigned HIVE-13203:
------------------------------------

    Assignee: VISHNU S NAIR

> STRUCT type not allowing more than 266 fields. Table get created with more fields but throws error while querying against the table. SHOW CREATE TABLE skips having some fields in STRUCT.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-13203
>                 URL: https://issues.apache.org/jira/browse/HIVE-13203
>             Project: Hive
>          Issue Type: Bug
>          Components: Database/Schema
>    Affects Versions: 1.2.0
>            Reporter: Satyajit varma
>            Assignee: VISHNU S NAIR
>
> I am trying to create a table with multiple STRUCT column types, one of which having more than 300 fields.
> I get the table created with no problem, but when i load i face few errors in loading up the data.when i check SHOW CREATE TABLE TABLENAME, i get fields only till 266th column and rest of them do not appear in the table DDL.(check the issue at StateOrProvinc, search for StateOrProvinc in the second DDL).
> Would like to know, if there is any restriction on the number of fields to be using in STRUCT datatype.
> EXAMPLE:
> BELOW IS THE DDL i run in hive and the second below DLL is the TABLE DDL i get from SHOW CREATE TABLE.
> CREATE TABLE property_feature.property_feature_new(
> `_id` STRUCT<`mls_sysid`:String,`type`:String>,
> all_attrs STRUCT<`1`: String ,`10`: String ,`100`: String ,`105`: String ,`113`: String ,`118`: String ,`126`: String ,`1299`: String ,`1301`: String ,`1305`: String ,`1307`: String ,`1319`: String ,`1325`: String ,`1327`: String ,`1329`: String ,`1339`: String ,`1341`: String ,`1345`: String ,`1351`: String ,`1353`: String ,`1359`: String ,`1363`: String ,`1365`: String ,`1367`: String ,`1373`: String ,`1377`: String ,`1397`: String ,`1401`: String ,`1403`: String ,`1405`: String ,`1409`: String ,`1411`: String ,`1413`: String ,`1417`: String ,`1419`: String ,`1421`: String ,`1423`: String ,`1431`: String ,`1437`: String ,`1439`: String ,`144`: String ,`1441`: String ,`145`: String ,`1451`: String ,`1453`: String ,`146`: String ,`1467`: String ,`1469`: String ,`1483`: String ,`1485`: String ,`1491`: String ,`1493`: String ,`15`: String ,`1505`: String ,`1519`: String ,`1521`: String ,`1523`: String ,`155`: String ,`156`: String ,`158`: String ,`1586`: String ,`1678`: String ,`169`: String ,`17`: String ,`177`: String ,`1783`: String ,`1785`: String ,`1787`: String ,`1789`: String ,`1793`: String ,`1795`: String ,`1797`: String ,`1799`: String ,`1801`: String ,`1803`: String ,`1805`: String ,`1807`: String ,`1809`: String ,`1811`: String ,`1815`: String ,`1818`: String ,`1826`: String ,`1829`: String ,`1874`: String ,`1876`: String ,`1878`: String ,`1880`: String ,`1882`: String ,`207`: String ,`215`: String ,`226`: String ,`2293`: String ,`232`: String ,`239`: String ,`241`: String ,`245`: String ,`2456`: String ,`248`: String ,`2482`: String ,`2490`: String ,`2491`: String ,`2492`: String ,`2493`: String ,`2495`: String ,`25`: String ,`2502`: String ,`2503`: String ,`2517`: String ,`2522`: String ,`255`: String ,`256`: String ,`2573`: String ,`2574`: String ,`260`: String ,`263`: String ,`265`: String ,`276`: String ,`278`: String ,`286`: String ,`293`: String ,`296`: String ,`299`: String ,`3`: String ,`30`: String ,`307`: String ,`312`: String ,`320`: String ,`328`: String ,`340`: String ,`348`: String ,`354`: String ,`363`: String ,`364`: String ,`369`: String ,`375`: String ,`376`: String ,`383`: String ,`401`: String ,`405`: String ,`409`: String ,`412`: String ,`42`: String ,`423`: String ,`444`: String ,`445`: String ,`446`: String ,`447`: String ,`450`: String ,`451`: String ,`452`: String ,`453`: String ,`455`: String ,`458`: String ,`461`: String ,`463`: String ,`464`: String ,`51`: String ,`543472980`: String ,`57`: String ,`75`: String ,`90`: String ,`92`: String ,`AVMYN`: String ,`AccessibilityAmenities`: String ,`AddressPublic`: String ,`Amenities`: String ,`AmountofWaterFront`: String ,`AppliancesIncluded`: String ,`AppraiserInitials`: String ,`AppraiserName`: String ,`Basement`: String ,`BathsFull`: String ,`BathsHalf`: String ,`BathsTotal`: String ,`BedsTotal`: String ,`Block`: String ,`Book`: String ,`BuildingNumber`: String ,`City`: String ,`CloseDate`: String ,`ClosePrice`: String ,
> `CoListAgentDirectWorkPhone`: String ,`CoListAgentEmail`: String ,`CoListAgentFullName`: String ,`CoListAgentMLSID`: String ,`CoListAgent_MUI`: String ,`CoListOfficeMLSID`: String ,`CoListOfficeName`: String ,`CoListOfficePhone`: String ,`CoListOffice_MUI`: String ,`Color`: String ,`CommentsYN`: String ,`Construction`: String ,`Cooling`: String ,`CountyOrParish`: String ,`DOM`: String ,`Deed`: String ,`Directions`: String ,`Driveway`: String ,`Electric`: String ,`Equipment`: String ,`Exterior`: String ,`Fireplace`: String ,`FloorNumber`: String ,`Floors`: String ,`FoundationMaterials`: String ,`FullTaxAmount`: String ,`Gas`: String ,`HeatFuel`: String ,`HeatSystem`: String ,`Internet`: String ,`ListAgentDirectWorkPhone`: String ,`ListAgentEmail`: String ,`ListAgentFullName`: String ,`ListAgentMLSID`: String ,`ListAgent_MUI`: String ,`ListOfficeMLSID`: String ,`ListOfficeName`: String ,`ListOfficePhone`: String ,`ListOffice_MUI`: String ,`ListPrice`: String ,`ListingContractDate`: String ,`Location`: String ,`Lot`: String ,`LotSizeArea`: String ,`MLSNumber`: String ,`Make`: String ,`MapNumber`: String ,`MatrixModifiedDT`: String ,`MobileHomeIncludes`: String ,`Model`: String ,`MonthlyAssociationFee`: String ,`NeighborhoodAssoc`: String ,`NumberOfRooms`: String ,`OneBedroomUnits`: String ,`OriginalListPrice`: String ,`Page`: String ,`Parking`: String ,`PhotoCount`: String ,`PhotoModificationTimestamp`: String ,`PostalCode`: String ,`PropertySubType`: String ,`PropertyType`: String ,`PublicAddress`: String ,`RecreationalWater`: String ,`Remarks`: String ,`Restrictions`: String ,`RoadFrontage`: String ,`Roads`: String ,`Roof`: String ,`SchoolDistrict`: String ,`Seasonal`: String ,`SellingAgentDirectWorkPhone`: String ,`SellingAgentEmail`: String ,`SellingAgentFullName`: String ,`SellingAgentMLSID`: String ,`SellingAgent_MUI`: String ,`SellingOfficeMLSID`: String ,`SellingOfficeName`: String ,`SellingOfficePhone`: String ,`SellingOffice_MUI`: String ,`SerialNumber`: String ,`Site`: String ,`SoldTerms`: String ,`SourceofSquareFootage`: String ,`SqFtFinishedTotal`: String ,`StateOrProvince`: String ,`Status`: String ,`Style`: String ,`SupplementPublicCount`: String ,`Surveyed`: String ,`TaxYear`: String ,`ThreeBedroomUnits`: String ,`Transportation`: String ,`TwoBedroomUnits`: String ,`UnitType`: String ,`UnitsInBuilding`: String ,`VehicleStorage`: String ,`View`: String ,`VirtualTour`: String ,`WasteWaterDisposal`: String ,`Water`: String ,`WaterBodyName`: String ,`WaterBodyType`: String ,`WaterFrontage`: String ,`WaterFrontageOwned`: String ,`WaterFrontageShared`: String ,`WaterHeater`: String ,`YearBuilt`: String ,`Zoning`: String ,`matrix_unique_id`: String ,`removed_created_at`: String ,`sysid`: String>
> basement` string
> ) STORED AS parquet;
> CREATE TABLE `property_feature`(
>   `_id` struct<mls_sysid:string,type:string>,
>   `all_attrs` struct<1:string,10:string,100:string,105:string,113:string,118:string,126:string,1299:string,
> 1301:string,1305:string,1307:string,1319:string,1325:string,1327:string,1329:string,
> 1339:string,1341:string,1345:string,1351:string,1353:string,1359:string,1363:string,
> 1365:string,1367:string,1373:string,1377:string,1397:string,1401:string,1403:string,
> 1405:string,1409:string,1411:string,1413:string,1417:string,1419:string,1421:string,
> 1423:string,1431:string,1437:string,1439:string,144:string,1441:string,145:string,1451:string,
> 1453:string,146:string,1467:string,1469:string,1483:string,1485:string,1491:string,1493:string,
> 15:string,1505:string,1519:string,1521:string,1523:string,155:string,156:string,158:string,
> 1586:string,1678:string,169:string,17:string,177:string,1783:string,1785:string,1787:string,
> 1789:string,1793:string,1795:string,1797:string,1799:string,1801:string,1803:string,1805:string,
> 1807:string,1809:string,1811:string,1815:string,1818:string,1826:string,1829:string,1874:string,
> 1876:string,1878:string,1880:string,1882:string,207:string,215:string,226:string,2293:string,
> 232:string,239:string,241:string,245:string,2456:string,248:string,2482:string,2490:string,
> 2491:string,2492:string,2493:string,2495:string,25:string,2502:string,2503:string,2517:string,
> 2522:string,255:string,256:string,2573:string,2574:string,260:string,263:string,265:string,
> 276:string,278:string,286:string,293:string,296:string,299:string,3:string,30:string,307:string,
> 312:string,320:string,328:string,340:string,348:string,354:string,363:string,364:string,369:string,
> 375:string,376:string,383:string,401:string,405:string,409:string,412:string,42:string,423:string,444:string,445:string,446:string,447:string,450:string,451:string,452:string,453:string,455:string,458:string,461:string,463:string,464:string,51:string,543472980:string,57:string,75:string,90:string,92:string,AVMYN:string,AccessibilityAmenities:string,
> AddressPublic:string,Amenities:string,AmountofWaterFront:string,AppliancesIncluded:string,AppraiserInitials:string,AppraiserName:string,Basement:string,BathsFull:string,BathsHalf:string,BathsTotal:string,BedsTotal:string,Block:string,Book:string,BuildingNumber:string,City:string,CloseDate:string,ClosePrice:string,CoListAgentDirectWorkPhone:string,CoListAgentEmail:string,CoListAgentFullName:string,CoListAgentMLSID:string,CoListAgent_MUI:string,CoListOfficeMLSID:string,CoListOfficeName:string,CoListOfficePhone:string,
> CoListOffice_MUI:string,Color:string,CommentsYN:string,Construction:string,Cooling:string,CountyOrParish:string,DOM:string,Deed:string,Directions:string,Driveway:string,Electric:string,Equipment:string,Exterior:string,Fireplace:string,FloorNumber:string,Floors:string,FoundationMaterials:string,FullTaxAmount:string,Gas:string,HeatFuel:string,HeatSystem:string,Internet:string,ListAgentDirectWorkPhone:string,ListAgentEmail:string,ListAgentFullName:string,ListAgentMLSID:string,ListAgent_MUI:string,ListOfficeMLSID:string,ListOfficeName:string,ListOfficePhone:string,ListOffice_MUI:string,ListPrice:string,ListingContractDate:string,Location:string,Lot:string,LotSizeArea:string,MLSNumber:string,Make:string,MapNumber:string,MatrixModifiedDT:string,MobileHomeIncludes:string,Model:string,MonthlyAssociationFee:string,NeighborhoodAssoc:string,NumberOfRooms:string,OneBedroomUnits:string,OriginalListPrice:string,Page:string,Parking:string,PhotoCount:string,PhotoModificationTimestamp:string,PostalCode:string,PropertySubType:string,PropertyType:string,PublicAddress:string,RecreationalWater:string,Remarks:string,Restrictions:string,RoadFrontage:string,Roads:string,Roof:string,SchoolDistrict:string,Seasonal:string,SellingAgentDirectWorkPhone:string,SellingAgentEmail:string,SellingAgentFullName:string,SellingAgentMLSID:string,SellingAgent_MUI:string,SellingOfficeMLSID:string,SellingOfficeName:string,SellingOfficePhone:string,SellingOffice_MUI:string,SerialNumber:string,Site:string,SoldTerms:string,
> SourceofSquareFootage:string,SqFtFinishedTotal:string,StateOrProvinc,
> `basement` string
> ) STORED AS parquet;
> Please help out in case if there is any workaround for it or any other restriction details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)