You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "AngersZhuuuu (via GitHub)" <gi...@apache.org> on 2023/03/14 10:27:29 UTC
[GitHub] [incubator-celeborn] AngersZhuuuu opened a new pull request, #1349: [CELEBORN-423] Format http request result
AngersZhuuuu opened a new pull request, #1349:
URL: https://github.com/apache/incubator-celeborn/pull/1349
### What changes were proposed in this pull request?
Format http request result
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] waitinfuture commented on a diff in pull request #1349: [CELEBORN-423] Format http request result
Posted by "waitinfuture (via GitHub)" <gi...@apache.org>.
waitinfuture commented on code in PR #1349:
URL: https://github.com/apache/incubator-celeborn/pull/1349#discussion_r1135641915
##########
master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala:
##########
@@ -698,88 +698,97 @@ private[celeborn] class Master(
override def getWorkerInfo: String = {
val sb = new StringBuilder
+ sb.append("====================== Workers Info in Master ===========================")
workersSnapShot.asScala.foreach { w =>
- sb.append("==========WorkerInfos in Master==========\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}in Master\n")
sb.append(w).append("\n")
+ sb.append("\n")
val workerInfo = requestGetWorkerInfos(w.endpoint)
.workerInfos.asJava
.get(0)
- sb.append("==========WorkerInfos in Workers==========\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}in Worker\n")
sb.append(workerInfo).append("\n")
+ sb.append("\n")
if (w.hasSameInfoWith(workerInfo)) {
- sb.append("Consist!").append("\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}status consist!\n")
} else {
- sb.append("[ERROR] Inconsistent!").append("\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}status not consist!\n")
}
+ sb.append("======================================================================")
}
sb.toString()
}
override def getLostWorkers: String = {
val sb = new StringBuilder
- sb.append("========== Lost WorkerInfos in Master==========\n")
- lostWorkersSnapshot.asScala.map { case (worker, time) =>
- sb.append(s"${worker.toUniqueId()} $time\n")
+ sb.append("======================= Lost Workers in Master ========================\n")
+ lostWorkersSnapshot.asScala.foreach { case (worker, time) =>
+ sb.append(s"${worker.toUniqueId().padTo(50, " ")}$time\n")
}
sb.toString()
}
override def getBlacklistedWorkers: String = {
val sb = new StringBuilder
- sb.append("==========Blacklisted WorkerInfos in Master==========\n")
- statusSystem.blacklist.asScala.map { worker =>
+ sb.append("==================== Blacklisted Workers in Master =====================\n")
+ statusSystem.blacklist.asScala.foreach { worker =>
sb.append(s"${worker.toUniqueId()}\n")
}
sb.toString()
}
override def getThreadDump: String = {
val sb = new StringBuilder
- val threadDump = Utils.getThreadDump()
- sb.append("==========Master ThreadDump==========\n")
- sb.append(threadDump).append("\n")
- workersSnapShot.asScala.foreach(w => {
Review Comment:
So we don't return workers' threaddump anymore, do you think we need a single request to return all workers' threaddump?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] codecov[bot] commented on pull request #1349: [CELEBORN-423] Format http request result
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #1349:
URL: https://github.com/apache/incubator-celeborn/pull/1349#issuecomment-1467843811
## [Codecov](https://codecov.io/gh/apache/incubator-celeborn/pull/1349?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1349](https://codecov.io/gh/apache/incubator-celeborn/pull/1349?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (65c3a38) into [main](https://codecov.io/gh/apache/incubator-celeborn/commit/3907d702125d408feaa3f2f74f8233a380fb3c9f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3907d70) will **decrease** coverage by `0.04%`.
> The diff coverage is `0.00%`.
> :exclamation: Current head 65c3a38 differs from pull request most recent head c0fb754. Consider uploading reports for the commit c0fb754 to get more accurate results
```diff
@@ Coverage Diff @@
## main #1349 +/- ##
============================================
- Coverage 29.82% 29.78% -0.04%
+ Complexity 918 916 -2
============================================
Files 221 221
Lines 18526 18536 +10
Branches 1982 1981 -1
============================================
- Hits 5524 5519 -5
- Misses 12610 12623 +13
- Partials 392 394 +2
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-celeborn/pull/1349?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...apache/celeborn/service/deploy/master/Master.scala](https://codecov.io/gh/apache/incubator-celeborn/pull/1349?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bWFzdGVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvY2VsZWJvcm4vc2VydmljZS9kZXBsb3kvbWFzdGVyL01hc3Rlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
| [...apache/celeborn/service/deploy/worker/Worker.scala](https://codecov.io/gh/apache/incubator-celeborn/pull/1349?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-d29ya2VyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvY2VsZWJvcm4vc2VydmljZS9kZXBsb3kvd29ya2VyL1dvcmtlci5zY2FsYQ==) | `0.00% <0.00%> (ø)` | |
... and [2 files with indirect coverage changes](https://codecov.io/gh/apache/incubator-celeborn/pull/1349/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] waitinfuture commented on a diff in pull request #1349: [CELEBORN-423] Format http request result
Posted by "waitinfuture (via GitHub)" <gi...@apache.org>.
waitinfuture commented on code in PR #1349:
URL: https://github.com/apache/incubator-celeborn/pull/1349#discussion_r1136467782
##########
master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala:
##########
@@ -698,88 +698,97 @@ private[celeborn] class Master(
override def getWorkerInfo: String = {
val sb = new StringBuilder
+ sb.append("====================== Workers Info in Master ===========================")
workersSnapShot.asScala.foreach { w =>
- sb.append("==========WorkerInfos in Master==========\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}in Master\n")
sb.append(w).append("\n")
+ sb.append("\n")
val workerInfo = requestGetWorkerInfos(w.endpoint)
.workerInfos.asJava
.get(0)
- sb.append("==========WorkerInfos in Workers==========\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}in Worker\n")
sb.append(workerInfo).append("\n")
+ sb.append("\n")
if (w.hasSameInfoWith(workerInfo)) {
- sb.append("Consist!").append("\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}status consist!\n")
} else {
- sb.append("[ERROR] Inconsistent!").append("\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}status not consist!\n")
}
+ sb.append("======================================================================")
}
sb.toString()
}
override def getLostWorkers: String = {
val sb = new StringBuilder
- sb.append("========== Lost WorkerInfos in Master==========\n")
- lostWorkersSnapshot.asScala.map { case (worker, time) =>
- sb.append(s"${worker.toUniqueId()} $time\n")
+ sb.append("======================= Lost Workers in Master ========================\n")
+ lostWorkersSnapshot.asScala.foreach { case (worker, time) =>
+ sb.append(s"${worker.toUniqueId().padTo(50, " ")}$time\n")
}
sb.toString()
}
override def getBlacklistedWorkers: String = {
val sb = new StringBuilder
- sb.append("==========Blacklisted WorkerInfos in Master==========\n")
- statusSystem.blacklist.asScala.map { worker =>
+ sb.append("==================== Blacklisted Workers in Master =====================\n")
+ statusSystem.blacklist.asScala.foreach { worker =>
sb.append(s"${worker.toUniqueId()}\n")
}
sb.toString()
}
override def getThreadDump: String = {
val sb = new StringBuilder
- val threadDump = Utils.getThreadDump()
- sb.append("==========Master ThreadDump==========\n")
- sb.append(threadDump).append("\n")
- workersSnapShot.asScala.foreach(w => {
Review Comment:
reasonable to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] waitinfuture merged pull request #1349: [CELEBORN-423] Format http request result
Posted by "waitinfuture (via GitHub)" <gi...@apache.org>.
waitinfuture merged PR #1349:
URL: https://github.com/apache/incubator-celeborn/pull/1349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] AngersZhuuuu commented on a diff in pull request #1349: [CELEBORN-423] Format http request result
Posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org>.
AngersZhuuuu commented on code in PR #1349:
URL: https://github.com/apache/incubator-celeborn/pull/1349#discussion_r1136462777
##########
master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala:
##########
@@ -698,88 +698,97 @@ private[celeborn] class Master(
override def getWorkerInfo: String = {
val sb = new StringBuilder
+ sb.append("====================== Workers Info in Master ===========================")
workersSnapShot.asScala.foreach { w =>
- sb.append("==========WorkerInfos in Master==========\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}in Master\n")
sb.append(w).append("\n")
+ sb.append("\n")
val workerInfo = requestGetWorkerInfos(w.endpoint)
.workerInfos.asJava
.get(0)
- sb.append("==========WorkerInfos in Workers==========\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}in Worker\n")
sb.append(workerInfo).append("\n")
+ sb.append("\n")
if (w.hasSameInfoWith(workerInfo)) {
- sb.append("Consist!").append("\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}status consist!\n")
} else {
- sb.append("[ERROR] Inconsistent!").append("\n")
+ sb.append(s"${w.toUniqueId().padTo(50, " ")}status not consist!\n")
}
+ sb.append("======================================================================")
}
sb.toString()
}
override def getLostWorkers: String = {
val sb = new StringBuilder
- sb.append("========== Lost WorkerInfos in Master==========\n")
- lostWorkersSnapshot.asScala.map { case (worker, time) =>
- sb.append(s"${worker.toUniqueId()} $time\n")
+ sb.append("======================= Lost Workers in Master ========================\n")
+ lostWorkersSnapshot.asScala.foreach { case (worker, time) =>
+ sb.append(s"${worker.toUniqueId().padTo(50, " ")}$time\n")
}
sb.toString()
}
override def getBlacklistedWorkers: String = {
val sb = new StringBuilder
- sb.append("==========Blacklisted WorkerInfos in Master==========\n")
- statusSystem.blacklist.asScala.map { worker =>
+ sb.append("==================== Blacklisted Workers in Master =====================\n")
+ statusSystem.blacklist.asScala.foreach { worker =>
sb.append(s"${worker.toUniqueId()}\n")
}
sb.toString()
}
override def getThreadDump: String = {
val sb = new StringBuilder
- val threadDump = Utils.getThreadDump()
- sb.append("==========Master ThreadDump==========\n")
- sb.append(threadDump).append("\n")
- workersSnapShot.asScala.foreach(w => {
Review Comment:
> So we don't return workers' threaddump anymore, do you think we need a single request to return all workers' threaddump?
The message is so noisy and I don't think we need to return all messages together, even if we want to build a machine admin website, we always open one machine's stack... won't open it together...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org