You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "guluo2016 (via GitHub)" <gi...@apache.org> on 2025/12/07 15:46:07 UTC
[PR] [arrow] Fix incorrect index usage in ArrowFieldWriters.TimeWriter [paimon]
guluo2016 opened a new pull request, #6768:
URL: https://github.com/apache/paimon/pull/6768
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #6767
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [arrow] Fix incorrect index usage in ArrowFieldWriters.TimeWriter [paimon]
Posted by "kaori-seasons (via GitHub)" <gi...@apache.org>.
kaori-seasons commented on code in PR #6768:
URL: https://github.com/apache/paimon/pull/6768#discussion_r2601300085
##########
paimon-arrow/src/test/java/org/apache/paimon/arrow/vector/ArrowFormatWriterTest.java:
##########
@@ -433,6 +438,55 @@ public void testWriteRowArrayTwice() {
}
}
+ @Test
+ public void testTimeFieldWriterWithOffset() {
+ RowType rowType =
+ new RowType(
+ Collections.singletonList(
+ new DataField(0, "time_field", DataTypes.TIME())));
+ try (RootAllocator allocator = new RootAllocator();
+ VectorSchemaRoot vsr = ArrowUtils.createVectorSchemaRoot(rowType, allocator)) {
+ ArrowFieldWriter[] fieldWriters = ArrowUtils.createArrowFieldWriters(vsr, rowType);
+
+ IntColumnVector timeVec =
+ new IntColumnVector() {
+ final int[] values = new int[] {0, 1000, 2000, 3000, 4000};
Review Comment:
Current situation: Similar index mixing (using `i` to read source data, or using `row` to check the Arrow vector) can easily occur repeatedly in multiple field writers, leading to hidden bugs.
Recommendation:
Recommendation: Define and strictly distinguish between `sourceIndex` (e.g., `rowIndex = startIndex + i`) and `targetIndex(i)` in the implementation.
Within the loop body of each writer, use `sourceIndex` to read data from the `ColumnVector`, and use `targetIndex` to write data to the `ArrowVector`; use `sourceIndex` to check for null values.
Example:
```
// Assume startIndex, batchRows, columnVector, and timeMilliVector are known.
for (int i = 0; i < batchRows; i++) {
int sourceIndex = startIndex + i; // 从 columnVector 读取的索引
int targetIndex = i; // 写到 Arrow 向量的位置
if (columnVector.isNullAt(sourceIndex)) {
timeMilliVector.setNull(targetIndex);
} else {
int value = ((IntColumnVector) columnVector).getInt(sourceIndex);
timeMilliVector.setSafe(targetIndex, value);
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [arrow] Fix incorrect index usage in ArrowFieldWriters.TimeWriter [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #6768:
URL: https://github.com/apache/paimon/pull/6768#issuecomment-4527135664
Already fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [arrow] Fix incorrect index usage in ArrowFieldWriters.TimeWriter [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed pull request #6768: [arrow] Fix incorrect index usage in ArrowFieldWriters.TimeWriter
URL: https://github.com/apache/paimon/pull/6768
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [arrow] Fix incorrect index usage in ArrowFieldWriters.TimeWriter [paimon]
Posted by "guluo2016 (via GitHub)" <gi...@apache.org>.
guluo2016 commented on PR #6768:
URL: https://github.com/apache/paimon/pull/6768#issuecomment-3626529774
@yuzelin @Zouxxyy @JingsongLi Can you review this pr when you have time, thanks!
If I missed anything, please let me know, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [arrow] Fix incorrect index usage in ArrowFieldWriters.TimeWriter [paimon]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6768:
URL: https://github.com/apache/paimon/pull/6768#issuecomment-4402158574
This pull request has had no activity for 90 days. If you'd like to keep it open, please push a new commit or leave a comment. Thanks for the contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [arrow] Fix incorrect index usage in ArrowFieldWriters.TimeWriter [paimon]
Posted by "guluo2016 (via GitHub)" <gi...@apache.org>.
guluo2016 commented on code in PR #6768:
URL: https://github.com/apache/paimon/pull/6768#discussion_r2602114985
##########
paimon-arrow/src/test/java/org/apache/paimon/arrow/vector/ArrowFormatWriterTest.java:
##########
@@ -433,6 +438,55 @@ public void testWriteRowArrayTwice() {
}
}
+ @Test
+ public void testTimeFieldWriterWithOffset() {
+ RowType rowType =
+ new RowType(
+ Collections.singletonList(
+ new DataField(0, "time_field", DataTypes.TIME())));
+ try (RootAllocator allocator = new RootAllocator();
+ VectorSchemaRoot vsr = ArrowUtils.createVectorSchemaRoot(rowType, allocator)) {
+ ArrowFieldWriter[] fieldWriters = ArrowUtils.createArrowFieldWriters(vsr, rowType);
+
+ IntColumnVector timeVec =
+ new IntColumnVector() {
+ final int[] values = new int[] {0, 1000, 2000, 3000, 4000};
Review Comment:
I didn’t modify `ArrowFieldWriters.java` to keep it consistent with the rest of the code.
Updating the test case to improve readability, thanks for the review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org