[GitHub] [iceberg] nastra commented on a change in pull request #3249: Optimized spark vectorized read parquet decimal

[GitBox <gi...@apache.org>]

nastra commented on a change in pull request #3249:
URL: https://github.com/apache/iceberg/pull/3249#discussion_r728819327



##########
File path: arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java
##########
@@ -171,6 +177,8 @@ public VectorHolder read(VectorHolder reuse, int numValsToRead) {
             vectorizedColumnIterator.timestampMillisBatchReader().nextBatch(vec, typeWidth, nullabilityHolder);
             break;
           case UUID:
+          case FIXED_WIDTH_BINARY:
+          case FIXED_LENGTH_DECIMAL:

Review comment:
       yep those changes for `FIXED_WIDTH_BINARY` are correct. Note that support for `FIXED` is being added as part of https://github.com/apache/iceberg/pull/3029

##########
File path: arrow/src/main/java/org/apache/iceberg/arrow/vectorized/GenericArrowVectorAccessorFactory.java
##########
@@ -92,19 +93,21 @@ protected GenericArrowVectorAccessorFactory(
     Dictionary dictionary = holder.dictionary();
     boolean isVectorDictEncoded = holder.isDictionaryEncoded();
     FieldVector vector = holder.vector();
+    ColumnDescriptor desc = holder.descriptor();
+    // desc could be null when the holder is PositionVectorHolder
+    PrimitiveType primitive = desc == null ? null : desc.getPrimitiveType();
     if (isVectorDictEncoded) {
-      ColumnDescriptor desc = holder.descriptor();
-      PrimitiveType primitive = desc.getPrimitiveType();
       return getDictionaryVectorAccessor(dictionary, desc, vector, primitive);
     } else {
-      return getPlainVectorAccessor(vector);
+      return getPlainVectorAccessor(vector, primitive);
     }
   }
 
   private ArrowVectorAccessor<DecimalT, Utf8StringT, ArrayT, ChildVectorT> getDictionaryVectorAccessor(
       Dictionary dictionary,
       ColumnDescriptor desc,
       FieldVector vector, PrimitiveType primitive) {
+    Preconditions.checkState(primitive != null, "Primitive should not be null");

Review comment:
       PrimitiveType should not be null




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org