[jira] [Commented] (DRILL-7518) Parquet INT64 Nullable Type Support

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-7518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010778#comment-17010778 ] 

ASF GitHub Bot commented on DRILL-7518:
---------------------------------------

arina-ielchiieva commented on pull request #1952: DRILL-7518: Support INT_64 for nullable INT64 in Parquet
URL: https://github.com/apache/drill/pull/1952
 
 
   Jira - [DRILL-7518](https://issues.apache.org/jira/browse/DRILL-7518).
   
   Sadly there is no file to reproduce the issue was attached. It's hard to generate such file using Jira code since such converted types are deprecated. Apparently, `pyspark` generates such files but did not try to generate file with it.
   
   Anyway, checked all supported converted types for int 32 and 64 according to documentation (https://github.com/apache/parquet-format/blob/master/LogicalTypes.md) and made sure all of them are supported for non-nullable and nullable readers. So far only one was missing.
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Parquet INT64 Nullable Type Support
> -----------------------------------
>
>                 Key: DRILL-7518
>                 URL: https://issues.apache.org/jira/browse/DRILL-7518
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.17.0
>         Environment: Tested on the apache/drill:1.17.0 docker image.
>            Reporter: David Severski
>            Assignee: Arina Ielchiieva
>            Priority: Major
>             Fix For: 1.18.0
>
>
> Querying a parquet file with fields of type INT64 generates an immediate error in the complex parquet reader with an error of "Unsupported nullable converted type INT_64 for primitive type INT64". Attempts to work around this via explicit CAST() and CONVERT_FROM() are unsuccessful. The suggestion from drill-users is that an implementation needs to be made at https://github.com/apache/drill/blob/9993fa3547b029db5fe33a2210fa6f07e8ac1990/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReaderFactory.java#L303.
>  
> If related, a similar INT32 type field in the same file exhibits this problem, but can be worked around via an explicit CAST() to INT.
>  
> At this time, I do not have a sanitized parquet file to submit as a reference example. :(
>  
> Reference thread on drill-users list: http://mail-archives.apache.org/mod_mbox/drill-user/202001.mbox/%3cCAJgUOA53LDKxqsH1fSVtJ+dk5421Eg4Aw4PaiM++8BfErxDvjQ@mail.gmail.com%3e



--
This message was sent by Atlassian Jira
(v8.3.4#803005)