You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/12/13 17:11:12 UTC
[GitHub] [arrow] jorgecarleitao opened a new pull request #11941: ARROW-15074: Clarified that LZ4 contains a single frame
jorgecarleitao opened a new pull request #11941:
URL: https://github.com/apache/arrow/pull/11941
See associated discussion at https://issues.apache.org/jira/browse/ARROW-15074
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#issuecomment-999748552
Benchmark runs are scheduled for baseline = 7e04d529ad261a6c644bdb998949c5c3cc21d195 and contender = 7baa65ced97714d0d9562c4fe74125718dae6b6e. 7baa65ced97714d0d9562c4fe74125718dae6b6e is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d891566b21734b34aeeaa31a644b1f42...ad53f6f9117948538adfcf95ecfac034/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/23e5cf226f944a91a51028372b3abb29...fe247c970958448f9d3c1301eb4712ae/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/f4662406b47c4e9a82b1fcf3bcd3c5e5...d614ea763915420db593b987b0e36ab3/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #11941: ARROW-15074: Clarified that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#issuecomment-992690141
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#issuecomment-999748552
Benchmark runs are scheduled for baseline = 7e04d529ad261a6c644bdb998949c5c3cc21d195 and contender = 7baa65ced97714d0d9562c4fe74125718dae6b6e. 7baa65ced97714d0d9562c4fe74125718dae6b6e is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d891566b21734b34aeeaa31a644b1f42...ad53f6f9117948538adfcf95ecfac034/)
[Failed :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/23e5cf226f944a91a51028372b3abb29...fe247c970958448f9d3c1301eb4712ae/)
[Failed :arrow_down:0.09% :arrow_up:0.04%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/f4662406b47c4e9a82b1fcf3bcd3c5e5...d614ea763915420db593b987b0e36ab3/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#issuecomment-996984041
At least the C++ implementation, yes. Basically, each LZ4 frame is meant to be an independent stream, and you can streaming compression/decompression even with a single frame.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#issuecomment-999748552
Benchmark runs are scheduled for baseline = 7e04d529ad261a6c644bdb998949c5c3cc21d195 and contender = 7baa65ced97714d0d9562c4fe74125718dae6b6e. 7baa65ced97714d0d9562c4fe74125718dae6b6e is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d891566b21734b34aeeaa31a644b1f42...ad53f6f9117948538adfcf95ecfac034/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/23e5cf226f944a91a51028372b3abb29...fe247c970958448f9d3c1301eb4712ae/)
[Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/f4662406b47c4e9a82b1fcf3bcd3c5e5...d614ea763915420db593b987b0e36ab3/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#discussion_r768634474
##########
File path: format/Message.fbs
##########
@@ -70,7 +70,7 @@ enum BodyCompressionMethod:byte {
/// bodies. Intended for use with RecordBatch but could be used for other
/// message types
table BodyCompression {
- /// Compressor library
+ /// Compressor library. Note that for LZ4_FRAME, it must contain a single frame
Review comment:
```suggestion
/// Compressor library.
/// For LZ4_FRAME, each compressed buffer must consist of a single frame.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] emkornfield commented on pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
emkornfield commented on pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#issuecomment-996928111
Why is this the case? Is this just adapting what implementations currently do?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #11941:
URL: https://github.com/apache/arrow/pull/11941
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao commented on a change in pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on a change in pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#discussion_r771126691
##########
File path: format/Message.fbs
##########
@@ -70,7 +70,7 @@ enum BodyCompressionMethod:byte {
/// bodies. Intended for use with RecordBatch but could be used for other
/// message types
table BodyCompression {
- /// Compressor library
+ /// Compressor library. Note that for LZ4_FRAME, it must contain a single frame
Review comment:
Good one. Committed. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#issuecomment-999748552
Benchmark runs are scheduled for baseline = 7e04d529ad261a6c644bdb998949c5c3cc21d195 and contender = 7baa65ced97714d0d9562c4fe74125718dae6b6e. 7baa65ced97714d0d9562c4fe74125718dae6b6e is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d891566b21734b34aeeaa31a644b1f42...ad53f6f9117948538adfcf95ecfac034/)
[Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/23e5cf226f944a91a51028372b3abb29...fe247c970958448f9d3c1301eb4712ae/)
[Failed :arrow_down:0.09% :arrow_up:0.04%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/f4662406b47c4e9a82b1fcf3bcd3c5e5...d614ea763915420db593b987b0e36ab3/)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou edited a comment on pull request #11941: ARROW-15074: [Format] Clarify that LZ4 contains a single frame
Posted by GitBox <gi...@apache.org>.
pitrou edited a comment on pull request #11941:
URL: https://github.com/apache/arrow/pull/11941#issuecomment-996984041
At least the C++ implementation, yes. Basically, each LZ4 frame is meant to be an independent stream, and you can do streaming compression/decompression even with a single frame.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org