You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "akashrn5 (via GitHub)" <gi...@apache.org> on 2026/10/05 17:27:20 UTC
[PR] [WIP]Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
akashrn5 opened a new pull request, #18387:
URL: https://github.com/apache/iceberg/pull/18387
Closes #18386
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#discussion_r4247147197
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/VectorizedSparkOrcReaders.java:
##########
@@ -156,7 +156,9 @@ public Converter primitive(Type.PrimitiveType iPrimitive, TypeDescription primit
break;
case BINARY:
primitiveValueReader =
- Type.TypeID.UUID == iPrimitive.typeId()
+ // iPrimitive is null for the variant metadata and value fields, which are
Review Comment:
I think we can omit this comment here and in the other places as it's AI generated and not adding lots of value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6311214254
> +1, thank you @akashrn5 and @nastra!
thank you for your review and approval @nastra @uros-b , @huaxingao please help to review and merge, thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6279149565
+1, thank you @akashrn5 and @nastra!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6364083138
@nastra @uros-b @huaxingao can you please hep to review and merge the PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "pvary (via GitHub) via issues" <is...@iceberg.apache.org>.
pvary commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6656068766
Move the fix into `OrcSchemaWithTypeVisitor.visitVariant`. It's one file in iceberg-orc instead of 9 copies across Spark versions. It also fixes the Flink reader, which would otherwise need the separate follow-up PR the author mentioned, plus Spark 3.5, which has the same code, and any future implementations. The PR's tests can stay as they are.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] ORC: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "pvary (via GitHub) via issues" <is...@iceberg.apache.org>.
pvary merged PR #18387:
URL: https://github.com/apache/iceberg/pull/18387
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6263192410
@nastra can you please have a look at the changes and help to review and merge? i have handled all the review comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on code in PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#discussion_r4312844216
##########
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestSparkOrcReader.java:
##########
@@ -124,4 +131,51 @@ public void testUnknownMapType() {
.isInstanceOf(IllegalArgumentException.class)
.hasMessageStartingWith("Cannot create MapType with unknown value type");
}
+
+ @Test
+ public void variantIsNotSupported() throws IOException {
+ File testFile = writeVariantOrcFile();
+
+ try (CloseableIterable<InternalRow> reader =
+ ORC.read(Files.localInput(testFile))
+ .project(VARIANT_SCHEMA)
+ .createReaderFunc(readOrcSchema -> new SparkOrcReader(VARIANT_SCHEMA, readOrcSchema))
+ .build()) {
+ assertThatThrownBy(reader::iterator)
+ .isInstanceOf(UnsupportedOperationException.class)
+ .hasMessage("Variant is not supported");
+ }
+ }
+
+ @Test
+ public void variantIsNotSupportedWhenVectorized() throws IOException {
+ File testFile = writeVariantOrcFile();
+
+ try (CloseableIterable<ColumnarBatch> reader =
+ ORC.read(Files.localInput(testFile))
+ .project(VARIANT_SCHEMA)
+ .createBatchedReaderFunc(
+ readOrcSchema ->
+ VectorizedSparkOrcReaders.buildReader(
+ VARIANT_SCHEMA, readOrcSchema, ImmutableMap.of()))
+ .build()) {
+ assertThatThrownBy(reader::iterator)
+ .isInstanceOf(UnsupportedOperationException.class)
+ .hasMessage("Variant is not supported");
+ }
+ }
+
+ private File writeVariantOrcFile() throws IOException {
+ File testFile = File.createTempFile("junit", null, temp.toFile());
+ assertThat(testFile.delete()).as("Delete should succeed").isTrue();
+
+ FileAppender<Record> writer =
Review Comment:
i used try with resource and created an actual record for the test, please review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6265485422
@akashrn5 can you please check whether we have the same issue with Flink's ORC reader/writer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#discussion_r4312663179
##########
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestSparkOrcReader.java:
##########
@@ -124,4 +131,51 @@ public void testUnknownMapType() {
.isInstanceOf(IllegalArgumentException.class)
.hasMessageStartingWith("Cannot create MapType with unknown value type");
}
+
+ @Test
+ public void variantIsNotSupported() throws IOException {
+ File testFile = writeVariantOrcFile();
+
+ try (CloseableIterable<InternalRow> reader =
+ ORC.read(Files.localInput(testFile))
+ .project(VARIANT_SCHEMA)
+ .createReaderFunc(readOrcSchema -> new SparkOrcReader(VARIANT_SCHEMA, readOrcSchema))
+ .build()) {
+ assertThatThrownBy(reader::iterator)
+ .isInstanceOf(UnsupportedOperationException.class)
+ .hasMessage("Variant is not supported");
+ }
+ }
+
+ @Test
+ public void variantIsNotSupportedWhenVectorized() throws IOException {
+ File testFile = writeVariantOrcFile();
+
+ try (CloseableIterable<ColumnarBatch> reader =
+ ORC.read(Files.localInput(testFile))
+ .project(VARIANT_SCHEMA)
+ .createBatchedReaderFunc(
+ readOrcSchema ->
+ VectorizedSparkOrcReaders.buildReader(
+ VARIANT_SCHEMA, readOrcSchema, ImmutableMap.of()))
+ .build()) {
+ assertThatThrownBy(reader::iterator)
+ .isInstanceOf(UnsupportedOperationException.class)
+ .hasMessage("Variant is not supported");
+ }
+ }
+
+ private File writeVariantOrcFile() throws IOException {
+ File testFile = File.createTempFile("junit", null, temp.toFile());
+ assertThat(testFile.delete()).as("Delete should succeed").isTrue();
+
+ FileAppender<Record> writer =
Review Comment:
can we use a try-finally block here and maybe try and write a record?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6266864044
> @akashrn5 can you please check whether we have the same issue with Flink's ORC reader/writer?
yes @nastra , i will raise a separate PR for flink instead of mixing it in spark module changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "akashrn5 (via GitHub) via issues" <is...@iceberg.apache.org>.
akashrn5 commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6609517765
> I will be out next week. Will try to come back after that
@pvary @huaxingao can you please have a look at it once since @nastra and @uros-b have already approved the changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on code in PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#discussion_r4247350801
##########
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestOrcWrite.java:
##########
@@ -56,4 +57,26 @@ public void splitOffsets() throws IOException {
writer.close();
assertThat(writer.splitOffsets()).as("Split offsets not present").isNotEmpty();
}
+
+ @Test
+ public void variantIsNotSupported() throws IOException {
Review Comment:
added the test case @nastra
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#discussion_r4247143109
##########
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestOrcWrite.java:
##########
@@ -56,4 +57,26 @@ public void splitOffsets() throws IOException {
writer.close();
assertThat(writer.splitOffsets()).as("Split offsets not present").isNotEmpty();
}
+
+ @Test
+ public void variantIsNotSupported() throws IOException {
Review Comment:
this only covers the write path. Can we also have a test for the two read paths that are modified?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra closed pull request #18387: Spark: Fix NPE when reading/writing variant type with ORC format
URL: https://github.com/apache/iceberg/pull/18387
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] ORC: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "akashrn5 (via GitHub) via issues" <is...@iceberg.apache.org>.
akashrn5 commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6684172021
> Move the fix into `OrcSchemaWithTypeVisitor.visitVariant`. It's one file in iceberg-orc instead of 9 copies across Spark versions. It also fixes the Flink reader, which would otherwise need the separate follow-up PR the author mentioned, plus Spark 3.5, which has the same code, and any future implementations. The PR's tests can stay as they are.
Thank you @pvary , makes sense. I have updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] ORC: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "pvary (via GitHub) via issues" <is...@iceberg.apache.org>.
pvary commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6684966478
Merged to main.
Thanks @akashrn5 for the PR and @uros-b and @nastra for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6122309056
Hi @nastra @uros-b can anyone please have a look at this PR? or if you can ask anyone else to review this. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#discussion_r4247152452
##########
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestOrcWrite.java:
##########
@@ -56,4 +57,26 @@ public void splitOffsets() throws IOException {
writer.close();
assertThat(writer.splitOffsets()).as("Split offsets not present").isNotEmpty();
}
+
+ @Test
+ public void variantIsNotSupported() throws IOException {
+ Schema variantSchema =
+ new Schema(
+ optional(1, "id", Types.IntegerType.get()),
+ optional(2, "payload", Types.VariantType.get()));
+
+ File testFile = File.createTempFile("junit", null, temp.toFile());
+ assertThat(testFile.delete()).as("Delete should succeed").isTrue();
+
+ // The ORC variant mapping visits the metadata and value fields with a null Iceberg type, so
+ // this must report that variant is unsupported instead of throwing a NullPointerException.
Review Comment:
I think we can remove this generated comment as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on code in PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#discussion_r4247355803
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/VectorizedSparkOrcReaders.java:
##########
@@ -156,7 +156,9 @@ public Converter primitive(Type.PrimitiveType iPrimitive, TypeDescription primit
break;
case BINARY:
primitiveValueReader =
- Type.TypeID.UUID == iPrimitive.typeId()
+ // iPrimitive is null for the variant metadata and value fields, which are
Review Comment:
done
##########
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestOrcWrite.java:
##########
@@ -56,4 +57,26 @@ public void splitOffsets() throws IOException {
writer.close();
assertThat(writer.splitOffsets()).as("Split offsets not present").isNotEmpty();
}
+
+ @Test
+ public void variantIsNotSupported() throws IOException {
+ Schema variantSchema =
+ new Schema(
+ optional(1, "id", Types.IntegerType.get()),
+ optional(2, "payload", Types.VariantType.get()));
+
+ File testFile = File.createTempFile("junit", null, temp.toFile());
+ assertThat(testFile.delete()).as("Delete should succeed").isTrue();
+
+ // The ORC variant mapping visits the metadata and value fields with a null Iceberg type, so
+ // this must report that variant is unsupported instead of throwing a NullPointerException.
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE in insert op for ORC format with variant type in v3 tables [iceberg]
Posted by "akashrn5 (via GitHub)" <gi...@apache.org>.
akashrn5 commented on code in PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#discussion_r4247350801
##########
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestOrcWrite.java:
##########
@@ -56,4 +57,26 @@ public void splitOffsets() throws IOException {
writer.close();
assertThat(writer.splitOffsets()).as("Split offsets not present").isNotEmpty();
}
+
+ @Test
+ public void variantIsNotSupported() throws IOException {
Review Comment:
added the test case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Fix NPE when reading/writing variant type with ORC format [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #18387:
URL: https://github.com/apache/iceberg/pull/18387#issuecomment-6373175177
I will be out next week. Will try to come back after that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org