You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/25 19:29:10 UTC
[GitHub] [arrow] lidavidm commented on a change in pull request #12692: ARROW-16005: [Java] Fix ArrayConsumer when using ArrowVectorIterator
lidavidm commented on a change in pull request #12692:
URL: https://github.com/apache/arrow/pull/12692#discussion_r835556364
##########
File path: java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/consumer/ArrayConsumer.java
##########
@@ -90,13 +97,12 @@ public void consume(ResultSet resultSet) throws SQLException, IOException {
int count = 0;
try (ResultSet rs = array.getResultSet()) {
while (rs.next()) {
- ensureInnerVectorCapacity(innerVectorIndex + count + 1);
Review comment:
It's because of how a ListVector is laid out in memory. The list `[[1, 2], [], [3, 4, 5]]` is represented as the child vector `[1, 2, 3, 4, 5]` and the offsets `[0, 2, 2, 5]`. `ensureInnerVectorCapacity` is resizing the child vector, so when we call `consume` for the last element, we want to ensure the child vector has enough capacity for the current elements, along with all the previous elements, and it looks like that's what `innerVectorIndex` is tracking.
In other words when we call `consume` for what will be `[3, 4, 5]` we need to ensure the child vector has space for at least 3, 4, 5, … elements not 1, 2, 3… elements.
##########
File path: java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/ArrowVectorIterator.java
##########
@@ -134,11 +122,26 @@ private VectorSchemaRoot createVectorSchemaRoot() {
}
throw new RuntimeException("Error occurred while creating schema root.", e);
}
+
+ // ensure consumers have been initialized
+ ensureInitialized(root);
return root;
}
+ private void ensureInitialized(VectorSchemaRoot root) throws SQLException {
+ if (!initialized) {
Review comment:
It seems this isn't right if `!config.isReuseVectorSchemaRoot()` because we'll have recreated the root. Since `ensureInitialized` is only ever called when creating a new root, I don't think we need to guard this with `initialized`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org