You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by lvdongr <gi...@git.apache.org> on 2017/08/18 03:25:30 UTC
[GitHub] spark pull request #18987: [SPARK-21775][Core]Dynamic Log Level Settings for...
GitHub user lvdongr opened a pull request:
https://github.com/apache/spark/pull/18987
[SPARK-21775][Core]Dynamic Log Level Settings for executors
## What changes were proposed in this pull request?
Someimes we want to change the log level of executor when our application has already deployed, to see detail infomation or decrease the log items. Changing the log4j configure file is not convenient,so We add the ability to set log level settings for a running executor.
## How was this patch tested?
manual tests
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lvdongr/spark SPARK-21775
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18987.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18987
----
commit 01af8ce69afeade8bb034c6965de0f3738f12fd5
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-03-08T04:09:40Z
[SPARK-19863][DStream] Whether or not use CachedKafkaConsumer need to be configured, when you use DirectKafkaInputDStream to connect the kafka in a Spark Streaming application has been successfully created.
commit b6daeec664d757999e257e56fed3844db51515e2
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-03-11T06:35:57Z
Merge remote-tracking branch 'apache/master'
commit e0e47b1da93b90210e44abc6e90655d3028555ec
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-04-12T07:20:01Z
Merge remote-tracking branch 'apache/master'
commit f4ab88111c5b8e9700eacc1acfa3858aed45124e
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-07-27T01:54:56Z
isklakldsng branch 'apache/master'
commit 463e570f9e05f785834e27bd535cfbb3b7cb7dfb
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-07-27T12:09:47Z
Merge remote-tracking branch 'apache/master'
commit 0e1b7f6d8e436ca243f78e3cbf064f591557b6c0
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-07-28T01:34:48Z
Merge remote-tracking branch 'apache/master'
commit 9a9972125ae8f7d90f5567f5b561f2c0ca16cfe7
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-07-28T02:50:23Z
refresh the master branch for kafkaconsumer
commit 637900b576b8c4d9e04a808a078e481a99751d03
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-07-31T03:08:29Z
Merge remote-tracking branch 'apache/master'
commit 04aafed076cb704a100eb7dc45b5cfda6438193b
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-08-17T11:41:58Z
Merge remote-tracking branch 'apache/master'
commit f88debcfccf9d1cd5c436321ff8cf444539dfd6c
Author: lvdongr <lv...@zte.com.cn>
Date: 2017-08-18T03:21:50Z
[SPARK-21775][Core]Dynamic Log Level Settings for executors
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18987: [SPARK-21775][Core]Dynamic Log Level Settings for execut...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18987
We won't merge this, it's too much overhead for little gain. You can close this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18987: [SPARK-21775][Core]Dynamic Log Level Settings for execut...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18987
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18987: [SPARK-21775][Core]Dynamic Log Level Settings for...
Posted by lvdongr <gi...@git.apache.org>.
Github user lvdongr closed the pull request at:
https://github.com/apache/spark/pull/18987
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18987: [SPARK-21775][Core]Dynamic Log Level Settings for execut...
Posted by lvdongr <gi...@git.apache.org>.
Github user lvdongr commented on the issue:
https://github.com/apache/spark/pull/18987
ok. Thank you all the same for your review @srowen @jerryshao @ajbozarth .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18987: [SPARK-21775][Core]Dynamic Log Level Settings for execut...
Posted by ajbozarth <gi...@git.apache.org>.
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/18987
I agree with @srowen and I'm not a fan out the UI either
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18987: [SPARK-21775][Core]Dynamic Log Level Settings for execut...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/18987
This seems like a super specific requirement, AFAIK there's no Hadoop related project support dynamically changing log level, do we really need this feature is Spark, do we have other workaround without changing the code? From my aspect, changing a mount of the code to support such specific requirement usually hard to persuade the community.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18987: [SPARK-21775][Core]Dynamic Log Level Settings for execut...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18987
This seems like a ton of complexity. I don't think this functionality is worth nearly this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18987: [SPARK-21775][Core]Dynamic Log Level Settings for execut...
Posted by lvdongr <gi...@git.apache.org>.
Github user lvdongr commented on the issue:
https://github.com/apache/spark/pull/18987
The log level setting is a very useful function.Our team is doing a spark application and when we want to see the debug log, we have to restart the application every time. So we develop this function.
The complexity lays on the ui display and set the log level. But we can choose not to show the setting Button on UI, and just give a api or restful interface for user to access this function.Then the change on spark is no much.
And Storm (http://storm.apache.org/) has the same function,too. @srowen @jerryshao @ajbozarth
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org