[GitHub] [incubator-iceberg] chenjunjiedada edited a comment on issue #684: Support Decimal in Parquet predicates

[GitBox <gi...@apache.org>]

chenjunjiedada edited a comment on issue #684: Support Decimal in Parquet predicates
URL: https://github.com/apache/incubator-iceberg/issues/684#issuecomment-574006063
 
 
   @rdblue, I 'm trying to understand this, please correct me if I'm wrong.
   
   Currently, `ParquetFilter@predicate`  uses binary column predicate for decimal while decimal can be stored as int or long type in parquet. So we need to translate the reference to int or long as following: 
   ```java
           case DECIMAL:
             Type.PrimitiveType type = ref.type().asPrimitiveType();
             if (type instanceof Types.DecimalType) {
               int precision = ((Types.DecimalType)type).precision();
               if (precision <= DECIMAL_INT32_MAX_DIGITS) {
                 return pred(op, FilterApi.intColumn(path), getParquetPrimitive(lit));
               }
               else if (precision <= DECIMAL_INT64_MAX_DIGITS) {
                 return pred(op, FilterApi.longColumn(path), getParquetPrimitive(lit));
               } else {
                 return pred(op, FilterApi.binaryColumn(path), getParquetPrimitive(lit));
               }
             }
             return pred(op, FilterApi.binaryColumn(path), getParquetPrimitive(lit));
   ```
   Since DecimalLiteral uses a default comparator (null first + natural order), so it cannot compare the BigDecimal that stored as binary. So we need to define a customized comparator such as `BinaryDecimalComparator` and use it in `DecimalLiteral#comparator()`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org