You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/25 20:27:43 UTC
[GitHub] [arrow] isaacbrodsky opened a new pull request #12718: [JS] RecordBatches that are compressed should throw an error
isaacbrodsky opened a new pull request #12718:
URL: https://github.com/apache/arrow/pull/12718
While https://issues.apache.org/jira/browse/ARROW-8674 is not implemented, ArrowJS will silently provide corrupt (not uncompressed) data to applications when compressed files are passed in. This happens to be the default from Python, e.g. with `df.to_feather('my_file')`.
Opening as a draft as I didn't find how to add a sample file for testing that this behaves as expected. I'd appreciate any pointers on which part of the test suite we can assert this behavior in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] domoritz commented on a change in pull request #12718: ARROW-16099: [JS] RecordBatches that are compressed should throw an error
Posted by GitBox <gi...@apache.org>.
domoritz commented on a change in pull request #12718:
URL: https://github.com/apache/arrow/pull/12718#discussion_r841131656
##########
File path: js/src/ipc/metadata/message.ts
##########
@@ -297,6 +297,9 @@ function decodeSchema(_schema: _Schema, dictionaries: Map<number, DataType> = ne
/** @ignore */
function decodeRecordBatch(batch: _RecordBatch, version = MetadataVersion.V4) {
+ if (batch.compression() !== null) {
+ throw new Error("Record batch compression not implemented");
Review comment:
```suggestion
throw new Error('Record batch compression not implemented');
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] domoritz closed pull request #12718: ARROW-16099: [JS] RecordBatches that are compressed should throw an error
Posted by GitBox <gi...@apache.org>.
domoritz closed pull request #12718:
URL: https://github.com/apache/arrow/pull/12718
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #12718: ARROW-16099: [JS] RecordBatches that are compressed should throw an error
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #12718:
URL: https://github.com/apache/arrow/pull/12718#issuecomment-1086757886
Benchmark runs are scheduled for baseline = 33cb5bd919b8988a8292674651518e6c3edc9823 and contender = f5166fe21969d19adff23fc840ed1d7511348bad. f5166fe21969d19adff23fc840ed1d7511348bad is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/2b41e995f04c4926855d33d0e9730283...99421fc84f89442e89dddebda7412f8d/)
[Scheduled] [test-mac-arm](https://conbench.ursa.dev/compare/runs/748bf40a9b624edebe5f92ed1dc62c43...fae039ffeacc4b21bba0b9b8d066b744/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/3e7172d2a4ba416c815fb6704676b279...7bc03ebc30304bfca55eca6fecb84776/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/cb5cb20c782e49c99d4f2814485de9e8...ed84dba97a20430fa61323e56fc90b5d/)
Buildkite builds:
[Scheduled] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/434| `f5166fe2` ec2-t3-xlarge-us-east-2>
[Scheduled] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/419| `f5166fe2` test-mac-arm>
[Scheduled] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/429| `f5166fe2` ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/433| `33cb5bd9` ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/418| `33cb5bd9` test-mac-arm>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/428| `33cb5bd9` ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12718: ARROW-16099: [JS] RecordBatches that are compressed should throw an error
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12718:
URL: https://github.com/apache/arrow/pull/12718#issuecomment-1086757886
Benchmark runs are scheduled for baseline = 33cb5bd919b8988a8292674651518e6c3edc9823 and contender = f5166fe21969d19adff23fc840ed1d7511348bad. f5166fe21969d19adff23fc840ed1d7511348bad is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/2b41e995f04c4926855d33d0e9730283...99421fc84f89442e89dddebda7412f8d/)
[Scheduled] [test-mac-arm](https://conbench.ursa.dev/compare/runs/748bf40a9b624edebe5f92ed1dc62c43...fae039ffeacc4b21bba0b9b8d066b744/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/3e7172d2a4ba416c815fb6704676b279...7bc03ebc30304bfca55eca6fecb84776/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/cb5cb20c782e49c99d4f2814485de9e8...ed84dba97a20430fa61323e56fc90b5d/)
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/434| `f5166fe2` ec2-t3-xlarge-us-east-2>
[Scheduled] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/419| `f5166fe2` test-mac-arm>
[Scheduled] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/429| `f5166fe2` ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/433| `33cb5bd9` ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/418| `33cb5bd9` test-mac-arm>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/428| `33cb5bd9` ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12718: ARROW-16099: [JS] RecordBatches that are compressed should throw an error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12718:
URL: https://github.com/apache/arrow/pull/12718#issuecomment-1086231433
https://issues.apache.org/jira/browse/ARROW-16099
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #12718: ARROW-16099: [JS] RecordBatches that are compressed should throw an error
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #12718:
URL: https://github.com/apache/arrow/pull/12718#issuecomment-1086757886
Benchmark runs are scheduled for baseline = 33cb5bd919b8988a8292674651518e6c3edc9823 and contender = f5166fe21969d19adff23fc840ed1d7511348bad. f5166fe21969d19adff23fc840ed1d7511348bad is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/2b41e995f04c4926855d33d0e9730283...99421fc84f89442e89dddebda7412f8d/)
[Failed :arrow_down:0.33% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/748bf40a9b624edebe5f92ed1dc62c43...fae039ffeacc4b21bba0b9b8d066b744/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/3e7172d2a4ba416c815fb6704676b279...7bc03ebc30304bfca55eca6fecb84776/)
[Finished :arrow_down:0.13% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/cb5cb20c782e49c99d4f2814485de9e8...ed84dba97a20430fa61323e56fc90b5d/)
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/434| `f5166fe2` ec2-t3-xlarge-us-east-2>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/419| `f5166fe2` test-mac-arm>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/429| `f5166fe2` ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/433| `33cb5bd9` ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/418| `33cb5bd9` test-mac-arm>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/428| `33cb5bd9` ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12718: [JS] RecordBatches that are compressed should throw an error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12718:
URL: https://github.com/apache/arrow/pull/12718#issuecomment-1079409827
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
Opening JIRAs ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename pull request title in the following format?
ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org