Binary ENUM in Parquet

[Michael Borst <mi...@mbr-targeting.com>]

Hi all,

when querying a parquet file that contains a binary field with original
type enum, I get the following error:
Error: SYSTEM ERROR: UnsupportedOperationException: Unsupported type ENUM

The file metadata is as follows:
Parquet Metadata: ParquetMetaData{FileMetaData{schema: message SomeMessage {
  ...
  repeated group someNestedField {
    ...
    repeated group anotherNestedField {
      ....
      optional binary anEnum (ENUM);

The exception is thrown here:
https://github.com/apache/drill/blob/1.1.0/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetGroupConverter.java#L244

I am using Drill 1.1.0.

Going through Jira, I noticed this ticket:
https://issues.apache.org/jira/browse/DRILL-1775
I assume this solves same problem for non-nested fields? Could this be
applied for my problem as well?

Best
Michael

-- 
Michael Borst
Big Data Engineer

michael@mbr-targeting.com
phone: +49 30 69817113

mbr targeting GmbH
Hobrechtstr. 65 | 12047 Berlin
Amtsgericht Charlottenburg | HRB 136489B
Geschäftsführer: Pablo Metz, Christoph Tavan

Re: Binary ENUM in Parquet

[Jinfeng Ni <ji...@gmail.com>]

DRILL-1175 has been merged, and should be in Drill 1.1.0. Based on the
patch, Drill simply converts binary ENUM to varbinary.  However, looks like
it did not fix the place you mentioned
(DrillParquetGroupConverter.java:#L244), where I think varbinary should be
used as well.  That's why you hit this error.

I'm CC to Jason, as he may have better idea about this issue.



On Tue, Aug 4, 2015 at 3:59 AM, Michael Borst <mi...@mbr-targeting.com>
wrote:

> Hi all,
>
> when querying a parquet file that contains a binary field with original
> type enum, I get the following error:
> Error: SYSTEM ERROR: UnsupportedOperationException: Unsupported type ENUM
>
> The file metadata is as follows:
> Parquet Metadata: ParquetMetaData{FileMetaData{schema: message SomeMessage
> {
>   ...
>   repeated group someNestedField {
>     ...
>     repeated group anotherNestedField {
>       ....
>       optional binary anEnum (ENUM);
>
> The exception is thrown here:
>
> https://github.com/apache/drill/blob/1.1.0/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetGroupConverter.java#L244
>
> I am using Drill 1.1.0.
>
> Going through Jira, I noticed this ticket:
> https://issues.apache.org/jira/browse/DRILL-1775
> I assume this solves same problem for non-nested fields? Could this be
> applied for my problem as well?
>
> Best
> Michael
>
> --
> Michael Borst
> Big Data Engineer
>
> michael@mbr-targeting.com
> phone: +49 30 69817113
>
> mbr targeting GmbH
> Hobrechtstr. 65 | 12047 Berlin
> Amtsgericht Charlottenburg | HRB 136489B
> Geschäftsführer: Pablo Metz, Christoph Tavan
>