Re: [PR] Spark, Arrow, Parquet: Add vectorized read support for parquet RLE encoded data pages [iceberg]

["nastra (via GitHub)" <gi...@apache.org>]

nastra commented on code in PR #14853:
URL: https://github.com/apache/iceberg/pull/14853#discussion_r2793744990


##########
spark/v4.1/spark/src/test/java/org/apache/iceberg/spark/data/vectorized/parquet/TestParquetVectorizedReads.java:
##########
@@ -439,6 +448,33 @@ public void testUnsupportedReadsForParquetV2() throws Exception {
         .hasMessageEndingWith("Disable vectorized reads to read this table/file");
   }
 
+  @Test
+  public void testRLEEncodingOnlySupportsBooleanDataPage() {
+    MessageType schema =
+        new MessageType(
+            "test",
+            primitive(PrimitiveTypeName.INT32, Type.Repetition.OPTIONAL).id(1).named("int_col"));
+    ColumnDescriptor intColumnDesc = schema.getColumnDescription(new String[] {"int_col"});
+    ByteBufferInputStream stream = ByteBufferInputStream.wrap(ByteBuffer.allocate(0));
+
+    String expectedMessage =
+        "Cannot support vectorized reads for column "
+            + intColumnDesc
+            + " with encoding "
+            + Encoding.RLE
+            + ". Disable vectorized reads to read this table/file";
+
+    assertThatThrownBy(
+            () ->
+                new VectorizedPageIterator(intColumnDesc, "parquet-mr", false) {
+                  {
+                    initDataReader(Encoding.RLE, stream, 0);
+                  }
+                })
+        .isInstanceOf(UnsupportedOperationException.class)
+        .hasMessage(expectedMessage);

Review Comment:
   ```suggestion
       ColumnDescriptor intColumnDesc = schema.getColumnDescription(new String[] {"int_col"});
   
       assertThatThrownBy(
               () ->
                   new VectorizedPageIterator(intColumnDesc, "parquet-mr", false) {
                     {
                       initDataReader(
                           Encoding.RLE, ByteBufferInputStream.wrap(ByteBuffer.allocate(0)), 0);
                     }
                   })
           .isInstanceOf(UnsupportedOperationException.class)
           .hasMessage(
               "Cannot support vectorized reads for column %s with encoding %s. Disable vectorized reads to read this table/file",
               intColumnDesc, Encoding.RLE);
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org