You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "cshuo (via GitHub)" <gi...@apache.org> on 2026/07/10 08:03:16 UTC
[PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
cshuo opened a new pull request, #19242:
URL: https://github.com/apache/hudi/pull/19242
### Describe the issue this Pull Request addresses
Flink Lance base-file reader/writer only supported primitive columns, preventing tables containing nested `ROW` and `ARRAY` data from being written or read correctly. Lance native logs were also unnecessarily disabled for table version 10+.
### Summary and Changelog
- Add recursive Flink `RowData` ↔ Arrow conversion for `ROW` and `ARRAY`, including nested schemas, null values, and Arrow list offsets.
- Enable Lance native logs for write version 10+, while rejecting native Lance column-stats indexing because Lance metadata statistics are unsupported.
- Remove the unused `HoodieTableConfig` argument from `CommonClientUtils.shouldWriteNativeLogs` and update common, Flink, and Spark call sites.
- Add unit coverage for nested Arrow conversion and MAP rejection.
- Add Lance nested-type upsert integration coverage for both COW and MOR tables.
- Keep `MAP` unsupported pending Lance Java writer support, tracked by https://github.com/lance-format/lance/issues/3620.
### Impact
- **Functional impact**: Flink Lance tables can persist and read nested `ROW` and `ARRAY` columns; Lance tables at write version 10+ use native logs.
- **Maintainability**: Native-log selection is now solely version-based and no longer carries an unused table-config dependency.
- **Extensibility**: Recursive conversion establishes the structure needed for additional complex types once Lance supports their Arrow encoding.
### Risk Level
Medium. The change affects Arrow serialization and native log routing for Lance. Coverage includes nested schema/value round trips, MAP rejection, and COW/MOR upsert integration tests.
### Documentation Update
none.
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3565809373
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/HoodieFlinkLanceArrowUtils.java:
##########
@@ -245,51 +286,122 @@ private static ArrowType toArrowType(LogicalType type) {
return new ArrowType.Timestamp(TimeUnit.MICROSECOND, null);
case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
return new ArrowType.Timestamp(TimeUnit.MICROSECOND, "UTC");
+ case ROW:
+ return ArrowType.Struct.INSTANCE;
+ case ARRAY:
+ return ArrowType.List.INSTANCE;
default:
throw unsupported(type);
}
}
- private static LogicalType toLogicalType(ArrowType arrowType) {
+ private static LogicalType toLogicalType(Field field, String path) {
+ ArrowType arrowType = field.getType();
+ LogicalType logicalType;
if (arrowType instanceof ArrowType.Bool) {
- return new BooleanType();
+ logicalType = new BooleanType();
} else if (arrowType instanceof ArrowType.Int) {
ArrowType.Int intType = (ArrowType.Int) arrowType;
switch (intType.getBitWidth()) {
case 8:
- return new TinyIntType();
+ logicalType = new TinyIntType();
+ break;
case 16:
- return new SmallIntType();
+ logicalType = new SmallIntType();
+ break;
case 32:
- return new IntType();
+ logicalType = new IntType();
+ break;
case 64:
- return new BigIntType();
+ logicalType = new BigIntType();
+ break;
default:
throw new HoodieNotSupportedException("Unsupported Arrow int width for Lance Flink reader: " + intType.getBitWidth());
}
} else if (arrowType instanceof ArrowType.FloatingPoint) {
ArrowType.FloatingPoint fp = (ArrowType.FloatingPoint) arrowType;
- return fp.getPrecision() == FloatingPointPrecision.SINGLE
+ logicalType = fp.getPrecision() == FloatingPointPrecision.SINGLE
? new FloatType()
: new DoubleType();
} else if (arrowType instanceof ArrowType.Utf8) {
- return new VarCharType();
+ logicalType = new VarCharType();
} else if (arrowType instanceof ArrowType.Binary) {
- return new VarBinaryType();
+ logicalType = new VarBinaryType();
} else if (arrowType instanceof ArrowType.Date) {
- return new DateType();
+ logicalType = new DateType();
} else if (arrowType instanceof ArrowType.Time) {
- return new TimeType();
+ logicalType = new TimeType();
} else if (arrowType instanceof ArrowType.Decimal) {
ArrowType.Decimal decimal = (ArrowType.Decimal) arrowType;
- return new DecimalType(decimal.getPrecision(), decimal.getScale());
+ logicalType = new DecimalType(decimal.getPrecision(), decimal.getScale());
} else if (arrowType instanceof ArrowType.Timestamp) {
ArrowType.Timestamp timestamp = (ArrowType.Timestamp) arrowType;
- return timestamp.getTimezone() == null
+ logicalType = timestamp.getTimezone() == null
? new TimestampType(6)
: new LocalZonedTimestampType(6);
+ } else if (arrowType instanceof ArrowType.Struct) {
+ List<RowType.RowField> fields = new ArrayList<>(field.getChildren().size());
+ for (Field child : field.getChildren()) {
+ fields.add(new RowType.RowField(child.getName(), toLogicalType(child, path + "." + child.getName())));
+ }
+ logicalType = new RowType(field.isNullable(), fields);
+ } else if (arrowType instanceof ArrowType.List) {
+ Field element = onlyChild(field, path, "list");
+ logicalType = new ArrayType(field.isNullable(), toLogicalType(element, path + "[]"));
+ } else {
+ throw new HoodieNotSupportedException("Unsupported Arrow type for Lance Flink reader at '" + path + "': " + arrowType);
+ }
+ return logicalType.copy(field.isNullable());
+ }
+
+ private static void writeRow(RowType rowType, StructVector vector, int rowId, RowData row, boolean utcTimestamp) {
+ vector.setIndexDefined(rowId);
+ for (int i = 0; i < rowType.getFieldCount(); i++) {
+ LogicalType childType = rowType.getTypeAt(i);
+ Object childValue = RowData.createFieldGetter(childType, i).getFieldOrNull(row);
+ writeValue(childType, (FieldVector) vector.getChildByOrdinal(i), rowId, childValue, utcTimestamp);
}
- throw new HoodieNotSupportedException("Unsupported Arrow type for Lance Flink reader: " + arrowType);
+ }
+
+ private static void writeArray(ArrayType arrayType, ListVector vector, int rowId, ArrayData array, boolean utcTimestamp) {
+ int startIndex = vector.startNewValue(rowId);
+ LogicalType elementType = arrayType.getElementType();
+ ArrayData.ElementGetter elementGetter = ArrayData.createElementGetter(elementType);
+ FieldVector dataVector = vector.getDataVector();
+ for (int i = 0; i < array.size(); i++) {
+ writeValue(elementType, dataVector, startIndex + i, elementGetter.getElementOrNull(array, i), utcTimestamp);
+ }
+ vector.endValue(rowId, array.size());
+ }
+
+ private static RowData readRow(RowType rowType, StructVector vector, int rowId) {
+ GenericRowData row = new GenericRowData(rowType.getFieldCount());
+ for (int i = 0; i < rowType.getFieldCount(); i++) {
+ row.setField(i, readValue(rowType.getTypeAt(i), vector.getChildByOrdinal(i), rowId));
+ }
+ return row;
+ }
+
+ private static ArrayData readArray(ArrayType arrayType, ListVector vector, int rowId) {
+ int startIndex = vector.getElementStartIndex(rowId);
+ int endIndex = vector.getElementEndIndex(rowId);
+ Object[] values = new Object[endIndex - startIndex];
+ FieldVector dataVector = vector.getDataVector();
+ for (int i = 0; i < values.length; i++) {
+ values[i] = readValue(arrayType.getElementType(), dataVector, startIndex + i);
+ }
+ return new GenericArrayData(values);
+ }
+
+ private static Field onlyChild(Field field, String path, String type) {
+ if (field.getChildren().size() != 1) {
+ throw invalidArrowSchema(path, type + " must contain exactly one child field");
+ }
+ return field.getChildren().get(0);
+ }
Review Comment:
🤖 nit: `type` as a `String` parameter name is a bit confusing here since every other `type` in this file refers to a `LogicalType` or `ArrowType` object — could you rename it to `typeName` (or `arrowKind`) to make it clear this is just a label for the error message?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3575359243
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java:
##########
@@ -220,7 +220,7 @@ void testStreamReadFromSpecifiedCommitWithChangelog(HoodieCDCSupplementalLogging
.option(FlinkOptions.READ_STREAMING_SKIP_COMPACT, false)
.option(FlinkOptions.CDC_ENABLED, true)
.option(FlinkOptions.SUPPLEMENTAL_LOGGING_MODE, mode.name())
- .option(FlinkOptions.READ_SOURCE_V2_ENABLED, useSourceV2)
+ .option(FlinkOptions.READ_SOURCE_V2_ENABLED, true)
Review Comment:
`useSourceV2` is still supplied as both `false` and `true` for every supplemental logging mode, but forcing `true` here makes each pair of parameterized invocations identical and removes all Source V1 CDC coverage. This is also unrelated to the Lance nested-type change. Could we restore `useSourceV2` here (or, if Source V1 is intentionally no longer supported, remove the boolean parameter/cases in a separate change with the rationale)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3575462162
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java:
##########
@@ -220,7 +220,7 @@ void testStreamReadFromSpecifiedCommitWithChangelog(HoodieCDCSupplementalLogging
.option(FlinkOptions.READ_STREAMING_SKIP_COMPACT, false)
.option(FlinkOptions.CDC_ENABLED, true)
.option(FlinkOptions.SUPPLEMENTAL_LOGGING_MODE, mode.name())
- .option(FlinkOptions.READ_SOURCE_V2_ENABLED, useSourceV2)
+ .option(FlinkOptions.READ_SOURCE_V2_ENABLED, true)
Review Comment:
yeah, this is an accidental change, will revert it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3575338385
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/HoodieFlinkLanceArrowUtils.java:
##########
@@ -113,63 +123,77 @@ public static void writeValue(LogicalType type, FieldVector vector, int rowId, R
}
public static void writeValue(LogicalType type, FieldVector vector, int rowId, RowData rowData, int ordinal, boolean utcTimestamp) {
- if (rowData.isNullAt(ordinal)) {
+ Object value = RowData.createFieldGetter(type, ordinal).getFieldOrNull(rowData);
Review Comment:
Yes. `RowDataArrowWriter.write` invokes this once per field per record. `createFieldGetter` constructs capturing lambdas (nullable types add another wrapper), and the `Object` return also boxes primitive values, so this regresses all existing primitive Lance writes rather than only nested writes. Could we prebuild/cache a recursive writer/accessor tree once from `rowType` (including the nested `FieldGetter`s and `ElementGetter`s), or otherwise keep the specialized primitive accessors in the hot loop?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3575338385
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/HoodieFlinkLanceArrowUtils.java:
##########
@@ -113,63 +123,77 @@ public static void writeValue(LogicalType type, FieldVector vector, int rowId, R
}
public static void writeValue(LogicalType type, FieldVector vector, int rowId, RowData rowData, int ordinal, boolean utcTimestamp) {
- if (rowData.isNullAt(ordinal)) {
+ Object value = RowData.createFieldGetter(type, ordinal).getFieldOrNull(rowData);
Review Comment:
Yes. `RowDataArrowWriter.write` invokes this once per field per record. `createFieldGetter` constructs capturing lambdas (nullable types add another wrapper), and the `Object` return also boxes primitive values, so this regresses all existing primitive Lance writes rather than only nested writes. Could we prebuild/cache a recursive writer/accessor tree once from `rowType` (including the nested `FieldGetter`s and `ElementGetter`s), or otherwise keep the specialized primitive accessors in the hot loop?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #19242:
URL: https://github.com/apache/hudi/pull/19242
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19242:
URL: https://github.com/apache/hudi/pull/19242#issuecomment-4933331268
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "97e532d8306c1cfb638fcb0e9a59ef4a46689da5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "97e532d8306c1cfb638fcb0e9a59ef4a46689da5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 97e532d8306c1cfb638fcb0e9a59ef4a46689da5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3572805230
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/lance/LanceRowDataWriter.java:
##########
@@ -0,0 +1,482 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io.storage.row.lance;
+
+import org.apache.hudi.exception.HoodieNotSupportedException;
+
+import org.apache.arrow.vector.BigIntVector;
+import org.apache.arrow.vector.BitVector;
+import org.apache.arrow.vector.DateDayVector;
+import org.apache.arrow.vector.DecimalVector;
+import org.apache.arrow.vector.FieldVector;
+import org.apache.arrow.vector.Float4Vector;
+import org.apache.arrow.vector.Float8Vector;
+import org.apache.arrow.vector.IntVector;
+import org.apache.arrow.vector.SmallIntVector;
+import org.apache.arrow.vector.TimeMilliVector;
+import org.apache.arrow.vector.TimeStampVector;
+import org.apache.arrow.vector.TinyIntVector;
+import org.apache.arrow.vector.VarBinaryVector;
+import org.apache.arrow.vector.VarCharVector;
+import org.apache.arrow.vector.complex.ListVector;
+import org.apache.arrow.vector.complex.StructVector;
+import org.apache.flink.table.data.ArrayData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.TimestampData;
+import org.apache.flink.table.types.logical.ArrayType;
+import org.apache.flink.table.types.logical.DecimalType;
+import org.apache.flink.table.types.logical.LogicalType;
+import org.apache.flink.table.types.logical.RowType;
+
+import java.util.List;
+
+import static org.apache.flink.table.types.logical.utils.LogicalTypeChecks.getPrecision;
+
+/**
+ * Writes Flink {@link RowData} values into Arrow vectors used by Lance base files.
+ *
+ * <p>The type-specific writer tree is created once for the supplied Arrow vectors. Record writes
+ * then use direct {@link RowData} and {@link ArrayData} accessors without creating field or element
+ * getters on the record-level path.
+ *
+ * <p>Supports primitive types, {@code ROW}, and {@code ARRAY}. {@code MAP} is not supported by the
+ * current Lance Java writer. Enable it once Lance adds Java writer support for {@code MAP}.
+ *
+ * @see <a href="https://github.com/lance-format/lance/issues/3620">Lance issue #3620</a>
+ */
+public class LanceRowDataWriter {
+
+ private final FieldWriter[] fieldWriters;
+
+ public LanceRowDataWriter(RowType rowType, List<FieldVector> vectors, boolean utcTimestamp) {
+ this.fieldWriters = new FieldWriter[rowType.getFieldCount()];
+ for (int i = 0; i < fieldWriters.length; i++) {
+ fieldWriters[i] = createWriter(rowType.getTypeAt(i), vectors.get(i), utcTimestamp);
+ }
+ }
+
+ public void write(RowData row, int rowId) {
+ for (int i = 0; i < fieldWriters.length; i++) {
+ fieldWriters[i].write(row, i, rowId);
+ }
+ }
+
+ private static FieldWriter createWriter(LogicalType type, FieldVector vector, boolean utcTimestamp) {
+ switch (type.getTypeRoot()) {
+ case BOOLEAN:
+ return new BooleanWriter((BitVector) vector);
+ case TINYINT:
+ return new TinyIntWriter((TinyIntVector) vector);
+ case SMALLINT:
+ return new SmallIntWriter((SmallIntVector) vector);
+ case INTEGER:
+ return new IntWriter((IntVector) vector);
+ case DATE:
+ return new DateWriter((DateDayVector) vector);
+ case TIME_WITHOUT_TIME_ZONE:
+ return new TimeWriter((TimeMilliVector) vector);
+ case BIGINT:
+ return new BigIntWriter((BigIntVector) vector);
+ case FLOAT:
+ return new FloatWriter((Float4Vector) vector);
+ case DOUBLE:
+ return new DoubleWriter((Float8Vector) vector);
+ case CHAR:
+ case VARCHAR:
+ return new StringWriter((VarCharVector) vector);
+ case BINARY:
+ case VARBINARY:
+ return new BinaryWriter((VarBinaryVector) vector);
+ case DECIMAL:
+ return new DecimalWriter((DecimalVector) vector, (DecimalType) type);
+ case TIMESTAMP_WITHOUT_TIME_ZONE:
+ case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
+ return new TimestampWriter((TimeStampVector) vector, getPrecision(type), utcTimestamp);
+ case ROW:
+ RowType rowType = (RowType) type;
+ StructVector structVector = (StructVector) vector;
+ FieldWriter[] fieldWriters = new FieldWriter[rowType.getFieldCount()];
+ for (int i = 0; i < fieldWriters.length; i++) {
Review Comment:
🤖 nit: could you rename this local variable to `childWriters` (or `nestedWriters`)? It shares the exact name of the outer instance field `fieldWriters`, and even though `createWriter` is static, a reader scanning the method will do a double-take to confirm these are unrelated.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3569959201
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/HoodieFlinkLanceArrowUtils.java:
##########
@@ -113,63 +123,77 @@ public static void writeValue(LogicalType type, FieldVector vector, int rowId, R
}
public static void writeValue(LogicalType type, FieldVector vector, int rowId, RowData rowData, int ordinal, boolean utcTimestamp) {
- if (rowData.isNullAt(ordinal)) {
+ Object value = RowData.createFieldGetter(type, ordinal).getFieldOrNull(rowData);
Review Comment:
is this a per-row action?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(flink): support nested data type for flink lance reader/writer [hudi]
Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3575455503
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/HoodieFlinkLanceArrowUtils.java:
##########
@@ -113,63 +123,77 @@ public static void writeValue(LogicalType type, FieldVector vector, int rowId, R
}
public static void writeValue(LogicalType type, FieldVector vector, int rowId, RowData rowData, int ordinal, boolean utcTimestamp) {
- if (rowData.isNullAt(ordinal)) {
+ Object value = RowData.createFieldGetter(type, ordinal).getFieldOrNull(rowData);
Review Comment:
Fixed by adding a new `LanceRowDataWriter`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org