You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/07/24 05:36:43 UTC
[GitHub] [arrow] westonpace opened a new pull request #10795: ARROW-13440: [C++] MapGenerator should optionally forward reentrant pressure
westonpace opened a new pull request #10795:
URL: https://github.com/apache/arrow/pull/10795
Added a basic mapping generator that does not queue incoming jobs. This allows it to forward async-reentrant pressure to the source. Fixed some issues in the CSV reader that were preventing it from running truly parallel. Performance is now significantly better but still not quite the same as the threaded reader. For the NY taxi dataset the streaming read time went from ~7 seconds to ~1.6 seconds. However, the file reader is still at ~0.8 seconds. I'll do more investigation later.
Leaving in draft as I want to extract a thread spawning generator I created into an independently tested thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #10795: ARROW-13155: [C++] MapGenerator should optionally forward reentrant pressure
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #10795:
URL: https://github.com/apache/arrow/pull/10795#issuecomment-889786927
Benchmark runs are scheduled for baseline = 03533fe89d6d29276ad3705a9f1fb56435d2c62c and contender = 53f08d28e06e9da33e5ea04075f0a1fd53e69277. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Failed] [ec2-t3-xlarge-us-east-2 (mimalloc)](https://conbench.ursa.dev/compare/runs/ad672e047a7d4ec7aab46caed0bc2c97...d9ab0b8e71624c4bad7e6f4f701d916d/)
[Scheduled] [ursa-i9-9960x (mimalloc)](https://conbench.ursa.dev/compare/runs/6b86e7c120114a53b89690e6cb49a859...25f21488a31f47daa79729655651f767/)
[Scheduled] [ursa-thinkcentre-m75q (mimalloc)](https://conbench.ursa.dev/compare/runs/af0f698407bb4eb3a62dbcbd03caedd5...d2a5e0fe6fe144b8aac767778c0f4307/)
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #10795: ARROW-13155: [C++] MapGenerator should optionally forward reentrant pressure
Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #10795:
URL: https://github.com/apache/arrow/pull/10795#issuecomment-889786927
Benchmark runs are scheduled for baseline = 03533fe89d6d29276ad3705a9f1fb56435d2c62c and contender = 53f08d28e06e9da33e5ea04075f0a1fd53e69277. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Scheduled] [ec2-t3-xlarge-us-east-2 (mimalloc)](https://conbench.ursa.dev/compare/runs/ad672e047a7d4ec7aab46caed0bc2c97...d9ab0b8e71624c4bad7e6f4f701d916d/)
[Scheduled] [ursa-i9-9960x (mimalloc)](https://conbench.ursa.dev/compare/runs/6b86e7c120114a53b89690e6cb49a859...25f21488a31f47daa79729655651f767/)
[Scheduled] [ursa-thinkcentre-m75q (mimalloc)](https://conbench.ursa.dev/compare/runs/af0f698407bb4eb3a62dbcbd03caedd5...d2a5e0fe6fe144b8aac767778c0f4307/)
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] westonpace commented on pull request #10795: ARROW-13155: [C++] MapGenerator should optionally forward reentrant pressure
Posted by GitBox <gi...@apache.org>.
westonpace commented on pull request #10795:
URL: https://github.com/apache/arrow/pull/10795#issuecomment-889786628
@ursabot please benchmark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] westonpace closed pull request #10795: ARROW-13155: [C++] MapGenerator should optionally forward reentrant pressure
Posted by GitBox <gi...@apache.org>.
westonpace closed pull request #10795:
URL: https://github.com/apache/arrow/pull/10795
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #10795: ARROW-13155: [C++] MapGenerator should optionally forward reentrant pressure
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #10795:
URL: https://github.com/apache/arrow/pull/10795#issuecomment-889786927
Benchmark runs are scheduled for baseline = 03533fe89d6d29276ad3705a9f1fb56435d2c62c and contender = 53f08d28e06e9da33e5ea04075f0a1fd53e69277. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Failed] [ec2-t3-xlarge-us-east-2 (mimalloc)](https://conbench.ursa.dev/compare/runs/ad672e047a7d4ec7aab46caed0bc2c97...d9ab0b8e71624c4bad7e6f4f701d916d/)
[Failed] [ursa-i9-9960x (mimalloc)](https://conbench.ursa.dev/compare/runs/6b86e7c120114a53b89690e6cb49a859...25f21488a31f47daa79729655651f767/)
[Scheduled] [ursa-thinkcentre-m75q (mimalloc)](https://conbench.ursa.dev/compare/runs/af0f698407bb4eb3a62dbcbd03caedd5...d2a5e0fe6fe144b8aac767778c0f4307/)
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot edited a comment on pull request #10795: ARROW-13155: [C++] MapGenerator should optionally forward reentrant pressure
Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #10795:
URL: https://github.com/apache/arrow/pull/10795#issuecomment-889786927
Benchmark runs are scheduled for baseline = 03533fe89d6d29276ad3705a9f1fb56435d2c62c and contender = 53f08d28e06e9da33e5ea04075f0a1fd53e69277. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Failed] [ec2-t3-xlarge-us-east-2 (mimalloc)](https://conbench.ursa.dev/compare/runs/ad672e047a7d4ec7aab46caed0bc2c97...d9ab0b8e71624c4bad7e6f4f701d916d/)
[Failed] [ursa-i9-9960x (mimalloc)](https://conbench.ursa.dev/compare/runs/6b86e7c120114a53b89690e6cb49a859...25f21488a31f47daa79729655651f767/)
[Failed] [ursa-thinkcentre-m75q (mimalloc)](https://conbench.ursa.dev/compare/runs/af0f698407bb4eb3a62dbcbd03caedd5...d2a5e0fe6fe144b8aac767778c0f4307/)
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #10795: ARROW-13155: [C++] MapGenerator should optionally forward reentrant pressure
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #10795:
URL: https://github.com/apache/arrow/pull/10795#issuecomment-888528264
https://issues.apache.org/jira/browse/ARROW-13155
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #10795: ARROW-13440: [C++] MapGenerator should optionally forward reentrant pressure
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #10795:
URL: https://github.com/apache/arrow/pull/10795#issuecomment-886003438
https://issues.apache.org/jira/browse/ARROW-13440
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org