[jira] [Created] (HAWQ-1674) Pxf error "expecting union description" when accessing partitioned hive table with avro schema

["David (JIRA)" <ji...@apache.org>]

David created HAWQ-1674:
---------------------------

             Summary: Pxf error "expecting union description" when accessing partitioned hive table with avro schema
                 Key: HAWQ-1674
                 URL: https://issues.apache.org/jira/browse/HAWQ-1674
             Project: Apache HAWQ
          Issue Type: Bug
          Components: PXF
            Reporter: David
            Assignee: Ed Espino


Hi,

 

i created a partitioned table in hive with an avro schema
{code:java}
{
    "type"     : "record",
    "name"     : "DavidSimpleData",
    "namespace": "namespace",
    "doc"      : "TestSchema",
    "fields"   : [
        {
            "name": "meta_extracted_at_time",
            "type": "long"
        },
        {
            "name": "status",
            "type": {
                "type"   : "enum",
                "name"   : "ToDoStatus",
                "symbols": [
                    "HIDDEN",
                    "ACTIONABLE",
                    "DONE",
                    "ARCHIVED",
                    "DELETED"
                ]
            }
        }
    ]
}{code}
 
{code:java}
create external table staging.david_simple
PARTITIONED BY(year int, month int)
row format serde 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
stored as inputformat 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
outputformat 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION '/user/talend/staging/david_simple'
tblproperties ('avro.schema.url'='http://schemaservice-stage.local:2876/schema-repo/david_simple/id/4'){code}
 

The hawq table is created with
{code:java}
create external table history.ext_test_david_simple (
    meta_extracted_at_time bigint,
    status varchar (100),
    year int,
    month int
    ) LOCATION ('pxf://pim01.local:51200/staging.david_simple?PROFILE=Hive')
    FORMAT 'custom' (formatter = 'pxfwritable_import');{code}
 

When selecting from the table The hawq error i get is

 
{code:java}
[XX000] ERROR: remote component error (500) from 'ip:51200': type Exception report message Found namespace.ToDoStatus, expecting union description The server encountered an internal error that prevented it from fulfilling this request. exception java.io.IOException: Found namespace.ToDoStatus, expecting union (libchurl.c:897) (seg11 pid03.cyberservices.local:40000 pid=371019) (dispatcher.c:1805) Detail: External table ext_test_david_simple{code}
 

Selecting on hive give correct results so it seems the pxf is producing the error somehow.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)