You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/11/04 03:26:38 UTC
[GitHub] [arrow] sarfata opened a new pull request, #14587: fix: RangeError crash in Vector.toArray()
sarfata opened a new pull request, #14587:
URL: https://github.com/apache/arrow/pull/14587
This fixes a bug which happens when a Vector has been created with multiple data blobs and at least one of them has been padded.
See https://observablehq.com/d/14488c116b338560 for a reproduction of the error and more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] sarfata commented on pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
sarfata commented on PR #14587:
URL: https://github.com/apache/arrow/pull/14587#issuecomment-1344859770
@domoritz fixed.
🤦 I did some last minute cleanup and added this test to show exactly the key condition for this bug but ended up with a failing test. Sorry about this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14587: fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14587:
URL: https://github.com/apache/arrow/pull/14587#issuecomment-1302908837
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
Opening JIRAs ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename pull request title in the following format?
ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14587:
URL: https://github.com/apache/arrow/pull/14587#issuecomment-1302912507
:warning: Ticket **has not been started in JIRA**, please click 'Start Progress'.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] domoritz merged pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
domoritz merged PR #14587:
URL: https://github.com/apache/arrow/pull/14587
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] sarfata commented on pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
sarfata commented on PR #14587:
URL: https://github.com/apache/arrow/pull/14587#issuecomment-1344679514
@domoritz @trxcllnt Thanks for your reviews.
I have added a unit test for my scenario. I also realized that my fix did not work when stride is not 1 so I have added a test (and fix) for that too.
I believe one of you will need to approve running the test workflow again before merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] trxcllnt commented on a diff in pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
trxcllnt commented on code in PR #14587:
URL: https://github.com/apache/arrow/pull/14587#discussion_r1044859014
##########
js/test/unit/vector/vector-tests.ts:
##########
@@ -211,6 +211,34 @@ describe(`ListVector`, () => {
});
});
+describe(`toArray()`, () => {
+ test(`when some data blobs have been padded`, () => {
+ const d1 = vectorFromArray([...Array(16).keys()]);
+ const d2 = vectorFromArray([...Array(10).keys()]);
+
+ // Padding has been added
+ expect(d2.length < d2.data.length)
+
+ const vector = new Vector([d1, d2]);
+
+ // This used to crash with "RangeError: offset is out of bounds"
+ // https://issues.apache.org/jira/browse/ARROW-18247
+ const array = vector.toArray();
+ expect(array).toHaveLength(26);
+ });
+
+ test(`when stride is 2`, () => {
+ let d1 = vectorFromArray([0, 1, 2], new Timestamp(TimeUnit.MILLISECOND)).data[0];
+ let d2 = vectorFromArray([3, 4, 5], new Timestamp(TimeUnit.MILLISECOND)).data[0];
+
+ const vector = new Vector([d1, d2]);
+
+ let array = Array.from(vector.toArray());
+ expect(array).toHaveLength(6 * 2);
+ expect(Array.from(array)).toMatchObject([0, 0, 1, 0, 2, 0, 3, 0, 4, 0, 5, 0]);
+ })
Review Comment:
```suggestion
});
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] trxcllnt commented on a diff in pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
trxcllnt commented on code in PR #14587:
URL: https://github.com/apache/arrow/pull/14587#discussion_r1044858609
##########
js/test/unit/vector/vector-tests.ts:
##########
@@ -211,6 +211,34 @@ describe(`ListVector`, () => {
});
});
+describe(`toArray()`, () => {
+ test(`when some data blobs have been padded`, () => {
+ const d1 = vectorFromArray([...Array(16).keys()]);
+ const d2 = vectorFromArray([...Array(10).keys()]);
Review Comment:
```suggestion
const d1 = vectorFromArray([...new Array(16).keys()]);
const d2 = vectorFromArray([...new Array(10).keys()]);
```
##########
js/test/unit/vector/vector-tests.ts:
##########
@@ -211,6 +211,34 @@ describe(`ListVector`, () => {
});
});
+describe(`toArray()`, () => {
+ test(`when some data blobs have been padded`, () => {
+ const d1 = vectorFromArray([...Array(16).keys()]);
+ const d2 = vectorFromArray([...Array(10).keys()]);
+
+ // Padding has been added
+ expect(d2.length < d2.data.length)
Review Comment:
```suggestion
expect(d2.length < d2.data.length);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14587:
URL: https://github.com/apache/arrow/pull/14587#issuecomment-1302912498
https://issues.apache.org/jira/browse/ARROW-18247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] trxcllnt commented on a diff in pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
trxcllnt commented on code in PR #14587:
URL: https://github.com/apache/arrow/pull/14587#discussion_r1044858708
##########
js/test/unit/vector/vector-tests.ts:
##########
@@ -211,6 +211,34 @@ describe(`ListVector`, () => {
});
});
+describe(`toArray()`, () => {
+ test(`when some data blobs have been padded`, () => {
+ const d1 = vectorFromArray([...Array(16).keys()]);
+ const d2 = vectorFromArray([...Array(10).keys()]);
+
+ // Padding has been added
+ expect(d2.length < d2.data.length)
Review Comment:
```suggestion
expect(d2.length).toBeLessThan(d2.data.length);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] domoritz commented on pull request #14587: ARROW-18247: [JS] fix: RangeError crash in Vector.toArray()
Posted by GitBox <gi...@apache.org>.
domoritz commented on PR #14587:
URL: https://github.com/apache/arrow/pull/14587#issuecomment-1344844493
Looks like a test is failing.
<img width="656" alt="Screenshot 2022-12-09 at 17 21 24" src="https://user-images.githubusercontent.com/589034/206805101-1f774fc9-c8d3-4e22-a6fe-008394c4dbfd.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org