You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "AngersZhuuuu (via GitHub)" <gi...@apache.org> on 2023/02/21 08:36:57 UTC
[GitHub] [incubator-celeborn] AngersZhuuuu opened a new pull request, #1258: [CELEBORN-321] When register shuffle failed, DataPushQueue should directly take the task queue to avoid NPE
AngersZhuuuu opened a new pull request, #1258:
URL: https://github.com/apache/incubator-celeborn/pull/1258
### What changes were proposed in this pull request?
ShuffleClientI side log when register shuffle failed
```
23/02/21 12:07:41 WARN ShuffleClientImpl: LifecycleManager request slots return StatusCode{value=ReserveSlotFailed}, retry again, remain retry times 2
23/02/21 12:07:44 WARN ShuffleClientImpl: LifecycleManager request slots return StatusCode{value=ReserveSlotFailed}, retry again, remain retry times 1
23/02/21 12:07:48 WARN ShuffleClientImpl: LifecycleManager request slots return StatusCode{value=ReserveSlotFailed}, retry again, remain retry times 0
23/02/21 12:07:51 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[DataPusher-458,5,main]
java.lang.NullPointerException
at com.aliyun.emr.rss.client.write.DataPushQueue.takePushTask(DataPushQueue.java:97)
at com.aliyun.emr.rss.client.write.DataPusher$1.run(DataPusher.java:123)
23/02/21 12:07:51 WARN ShuffleClientImpl: LifecycleManager request slots return StatusCode{value=ReserveSlotFailed}, retry again, remain retry times 2
23/02/21 12:07:51 INFO BlockManager: BlockManager stopped
23/02/21 12:07:51 INFO ShutdownHookManager: Shutdown hook called
```
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] codecov[bot] commented on pull request #1258: [CELEBORN-321] When register shuffle failed, DataPushQueue should directly take the task queue to avoid NPE
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #1258:
URL: https://github.com/apache/incubator-celeborn/pull/1258#issuecomment-1438067153
# [Codecov](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1258](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8b05263) into [main](https://codecov.io/gh/apache/incubator-celeborn/commit/b09b85521a3492eba7eac5b2a4c14bbecd3b8b46?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b09b855) will **decrease** coverage by `0.03%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## main #1258 +/- ##
============================================
- Coverage 27.17% 27.13% -0.03%
- Complexity 811 813 +2
============================================
Files 214 214
Lines 18315 18353 +38
Branches 1988 1997 +9
============================================
+ Hits 4975 4978 +3
- Misses 13014 13050 +36
+ Partials 326 325 -1
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/celeborn/client/write/DataPushQueue.java](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9jZWxlYm9ybi9jbGllbnQvd3JpdGUvRGF0YVB1c2hRdWV1ZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...deploy/master/clustermeta/AbstractMetaManager.java](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bWFzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9jZWxlYm9ybi9zZXJ2aWNlL2RlcGxveS9tYXN0ZXIvY2x1c3Rlcm1ldGEvQWJzdHJhY3RNZXRhTWFuYWdlci5qYXZh) | `85.64% <0.00%> (-2.43%)` | :arrow_down: |
| [.../org/apache/celeborn/client/ShuffleClientImpl.java](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9jZWxlYm9ybi9jbGllbnQvU2h1ZmZsZUNsaWVudEltcGwuamF2YQ==) | `18.21% <0.00%> (-0.06%)` | :arrow_down: |
| [...born/common/protocol/message/ControlMessages.scala](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvY2VsZWJvcm4vY29tbW9uL3Byb3RvY29sL21lc3NhZ2UvQ29udHJvbE1lc3NhZ2VzLnNjYWxh) | `0.14% <0.00%> (-<0.01%)` | :arrow_down: |
| [...a/org/apache/celeborn/common/meta/WorkerInfo.scala](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvY2VsZWJvcm4vY29tbW9uL21ldGEvV29ya2VySW5mby5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [.../org/apache/celeborn/client/LifecycleManager.scala](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvY2VsZWJvcm4vY2xpZW50L0xpZmVjeWNsZU1hbmFnZXIuc2NhbGE=) | `0.00% <0.00%> (ø)` | |
| [...apache/celeborn/service/deploy/master/Master.scala](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bWFzdGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvY2VsZWJvcm4vc2VydmljZS9kZXBsb3kvbWFzdGVyL01hc3Rlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...orn/common/network/server/BufferStreamManager.java](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9jZWxlYm9ybi9jb21tb24vbmV0d29yay9zZXJ2ZXIvQnVmZmVyU3RyZWFtTWFuYWdlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...leborn/service/deploy/worker/storage/Flusher.scala](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-d29ya2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvY2VsZWJvcm4vc2VydmljZS9kZXBsb3kvd29ya2VyL3N0b3JhZ2UvRmx1c2hlci5zY2FsYQ==) | `54.13% <0.00%> (+0.50%)` | :arrow_up: |
| [.../celeborn/service/deploy/worker/FetchHandler.scala](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-d29ya2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvY2VsZWJvcm4vc2VydmljZS9kZXBsb3kvd29ya2VyL0ZldGNoSGFuZGxlci5zY2FsYQ==) | `37.23% <0.00%> (+1.06%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/apache/incubator-celeborn/pull/1258?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] zy-jordan commented on pull request #1258: [CELEBORN-321] When register shuffle failed, DataPushQueue should directly take the task queue to avoid NPE
Posted by "zy-jordan (via GitHub)" <gi...@apache.org>.
zy-jordan commented on PR #1258:
URL: https://github.com/apache/incubator-celeborn/pull/1258#issuecomment-1438098524
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] AngersZhuuuu merged pull request #1258: [CELEBORN-321] When register shuffle failed, DataPushQueue should directly take the task queue to avoid NPE
Posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org>.
AngersZhuuuu merged PR #1258:
URL: https://github.com/apache/incubator-celeborn/pull/1258
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org