You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2023/01/18 15:09:17 UTC

[GitHub] [kyuubi] zzzzming95 opened a new issue, #4186: [Improvement] spark showProgress can briefly output info of the job

zzzzming95 opened a new issue, #4186:
URL: https://github.com/apache/kyuubi/issues/4186

   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   
   
   ### Search before asking
   
   - [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What would you like to be improved?
   
   When kyuubi.session.engine.spark.showProgress setting true, the user executing the job will print information about the progress of the current Stage. like this:
   <img width="1439" alt="image" src="https://user-images.githubusercontent.com/13965087/213206998-e6c33962-5da4-4ee4-9c8a-c357649957da.png">
   
   We can also briefly output the information of the job, like this :
   <img width="1294" alt="image" src="https://user-images.githubusercontent.com/13965087/213207484-9ffbc74c-ae92-4061-aadc-c96cc905efbf.png">
   
   
   ### How should we improve?
   
   <img width="1294" alt="image" src="https://user-images.githubusercontent.com/13965087/213207503-078f2076-1386-48f3-a9da-b9c042c7ff49.png">
   
   
   ### Are you willing to submit PR?
   
   - [X] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
   - [ ] No. I cannot submit a PR at this time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [kyuubi] davidyuan1223 commented on issue #4186: [Improvement] spark showProgress can briefly output info of the job

Posted by "davidyuan1223 (via GitHub)" <gi...@apache.org>.
davidyuan1223 commented on issue #4186:
URL: https://github.com/apache/kyuubi/issues/4186#issuecomment-1735302026

   i want improve this issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

Re: [I] [Improvement] spark showProgress can briefly output info of the job [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed issue #4186: [Improvement] spark showProgress can briefly output info of the job
URL: https://github.com/apache/kyuubi/issues/4186


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

[GitHub] [kyuubi] davidyuan1223 commented on issue #4186: [Improvement] spark showProgress can briefly output info of the job

Posted by "davidyuan1223 (via GitHub)" <gi...@apache.org>.
davidyuan1223 commented on issue #4186:
URL: https://github.com/apache/kyuubi/issues/4186#issuecomment-1738883087

   @cxzl25 hello,i have some question about this issue, the stageInfo has not property about jobId,and i found a code block abount miss logs about logs in code
   
   `  // For broadcast, Spark will introduce a new runId as SPARK_JOB_GROUP_ID, see:
     // https://github.com/apache/spark/pull/24595, So we will miss these logs.
     // TODO: Fix this until the below ticket resolved
     // https://issues.apache.org/jira/browse/SPARK-34064`
    
     if we want add job info in the log, we need add about [jobId,Seq[StageInfo]] map,when a stage is submitted, it will check the submitted stageInfo is in the Seq[StageInfo], is that right?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org