You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "westonpace (via GitHub)" <gi...@apache.org> on 2023/04/12 14:04:49 UTC
[GitHub] [arrow] westonpace opened a new pull request, #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
westonpace opened a new pull request, #35075:
URL: https://github.com/apache/arrow/pull/35075
### Rationale for this change
Fixes a bug in the throttled scheduler.
### What changes are included in this PR?
The throttled scheduler will no longer recurse in the ContinueTasks loop if the continued task was immediately finished.
### Are these changes tested?
Yes, I added a new stress test that exposed the stack overflow very reliably on a standard Linux system.
### Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] snizovtsev commented on pull request #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
Posted by "snizovtsev (via GitHub)" <gi...@apache.org>.
snizovtsev commented on PR #35075:
URL: https://github.com/apache/arrow/pull/35075#issuecomment-1506247247
I've tested the branch on my workload and can confirm that stack overflows crashes had gone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35075:
URL: https://github.com/apache/arrow/pull/35075#issuecomment-1505339224
* Closes: #34539
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35075:
URL: https://github.com/apache/arrow/pull/35075#issuecomment-1505339350
:warning: GitHub issue #34539 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] westonpace commented on pull request #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on PR #35075:
URL: https://github.com/apache/arrow/pull/35075#issuecomment-1505339731
CC @benibus for review since you faced something very similar recently :smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] westonpace commented on pull request #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on PR #35075:
URL: https://github.com/apache/arrow/pull/35075#issuecomment-1507111376
@snizovtsev thanks for testing this out. I've gone ahead and merged so it should get picked up in the next 12.0.0 release candidate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] westonpace merged pull request #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace merged PR #35075:
URL: https://github.com/apache/arrow/pull/35075
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #35075:
URL: https://github.com/apache/arrow/pull/35075#issuecomment-1510033169
Benchmark runs are scheduled for baseline = a1d1373c78bf8dcc5e064b9d069fc2be71a91506 and contender = 196222dbd543d6931f4a1432845add97be0db802. 196222dbd543d6931f4a1432845add97be0db802 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/18897e5a083a4e73b755d4cd4dd9232f...ff97c2c2c09f4edd97501dc11bbe2143/)
[Failed] [test-mac-arm](https://conbench.ursa.dev/compare/runs/2c2a65c89de04cee817b67c73d80d3f8...9491c508a418449583978ddb1b99d16d/)
[Finished :arrow_down:6.12% :arrow_up:0.77%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/52dc61836c934cbf949252ed137389ef...5b518ea078fe472fb1be388b3788e62f/)
[Finished :arrow_down:1.54% :arrow_up:0.03%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/4ddae8411b464da29bea7a4413f814a5...cea02221017c4819b278517a1cac0696/)
Buildkite builds:
[Finished] [`196222db` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2708)
[Failed] [`196222db` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/2742)
[Finished] [`196222db` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2706)
[Finished] [`196222db` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/2733)
[Finished] [`a1d1373c` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2707)
[Failed] [`a1d1373c` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/2741)
[Finished] [`a1d1373c` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2705)
[Finished] [`a1d1373c` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/2732)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #35075: GH-34539: [C++] Fix throttled scheduler to avoid stack overflow in dataset writer
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #35075:
URL: https://github.com/apache/arrow/pull/35075#issuecomment-1510033356
['Python', 'R'] benchmarks have high level of regressions.
[ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/52dc61836c934cbf949252ed137389ef...5b518ea078fe472fb1be388b3788e62f/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org