You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by liu-zhaokun <gi...@git.apache.org> on 2017/07/11 08:00:27 UTC
[GitHub] spark pull request #18599: [SPARK-21372] spark writes one log file even I se...
GitHub user liu-zhaokun opened a pull request:
https://github.com/apache/spark/pull/18599
[SPARK-21372] spark writes one log file even I set the number of spark_rotate_log to 0
[https://issues.apache.org/jira/browse/SPARK-21372](https://issues.apache.org/jira/browse/SPARK-21372)
Spark writes one log file even I set the number of spark_rotate_log to 0.Meanwhile,the number of spark_rotate_log also can set to a string type and other type.So I think we should make some changes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liu-zhaokun/spark new07111544
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18599.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18599
----
commit 7c71bc7e1d835dd1990a55d3c4213ae079991990
Author: liuzhaokun <li...@zte.com.cn>
Date: 2017-07-11T07:58:26Z
[SPARK-21372] spark writes one log file even I set the number of spark_rotate_log to 0
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by liu-zhaokun <gi...@git.apache.org>.
Github user liu-zhaokun commented on the issue:
https://github.com/apache/spark/pull/18599
@srowen
If I set num to 0,there shouldn't create a logfile or we should make some deal for 0，and If I set num some invaild symbol,the function doesn't work normally,right？I am sorry for my poor English.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by liu-zhaokun <gi...@git.apache.org>.
Github user liu-zhaokun commented on the issue:
https://github.com/apache/spark/pull/18599
@srowen
I think spark provides the param,so I can pass it in the script.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18599
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18599
AFAIK the function `spark_rotate_log` is never used with `num` param set, it would be interesting to see how this issue is triggered, could you share your script please? Thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18599
Did you see any failure in production environment caused by this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18599: [SPARK-21372] spark writes one log file even I se...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18599
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by liu-zhaokun <gi...@git.apache.org>.
Github user liu-zhaokun commented on the issue:
https://github.com/apache/spark/pull/18599
@jiangxb1987
When I want to change the number of log file,I test the scenario that num=0,and spark writes one log file.When I test the scenario that set num to Invaild symbol，the logic is wrong.So I think we should modify it to guarantee this feature.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by liu-zhaokun <gi...@git.apache.org>.
Github user liu-zhaokun commented on the issue:
https://github.com/apache/spark/pull/18599
@srowen
The param "num" which defined in function spark_rotate_log() isn't used,if users don't modify the script,it will never be invoked,so could I create a new PR to delete the param and replace it with a fixed number 5?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18599: [SPARK-21372] spark writes one log file even I se...
Posted by liu-zhaokun <gi...@git.apache.org>.
Github user liu-zhaokun commented on a diff in the pull request:
https://github.com/apache/spark/pull/18599#discussion_r126847486
--- Diff: sbin/spark-daemon.sh ---
@@ -78,6 +78,12 @@ spark_rotate_log ()
if [ -n "$2" ]; then
num=$2
--- End diff --
@srowen
There provide a param,num, to set the number of logfile,but it wasn't used in line 179.Since it doesn't work,can I remove the param "num" and use it just as a variable？
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18599
@liu-zhaokun I don't understand what bug you are referring to. We'd need to understand how an unmodified script displays some incorrect behavior.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by liu-zhaokun <gi...@git.apache.org>.
Github user liu-zhaokun commented on the issue:
https://github.com/apache/spark/pull/18599
@jiangxb1987
I want to change the number of log file ,so I passed num param manually by modify code in this file.In fact,this bug is there，even I don't trigger it,right?Shouldn't we modify it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18599
How would you set this with normal operation?
If you change the script to set a value that wasn't expected at that point given how the script works, of course it may fail. That's not a bug.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18599: [SPARK-21372] spark writes one log file even I set the n...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18599
Let's close this. You aren't providing a scenario where an input causes a problem, and I think are still saying the error happens when you modify the script.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18599: [SPARK-21372] spark writes one log file even I se...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/18599#discussion_r126636677
--- Diff: sbin/spark-daemon.sh ---
@@ -78,6 +78,12 @@ spark_rotate_log ()
if [ -n "$2" ]; then
num=$2
fi
+
+ expr $num + 0 &>/dev/null
+ if [ $? -ne 0 ] || [ $num -eq 0 ]; then
--- End diff --
Are you trying to verify if `num` is a number? I get it but you would already get a `bash` error in `$num -eq 0` if so and we don't otherwise validate script input that way. Isn't this forcing num to be nonzero, when I thought the issue was handling the case of 0 correctly?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18599: [SPARK-21372] spark writes one log file even I se...
Posted by liu-zhaokun <gi...@git.apache.org>.
Github user liu-zhaokun commented on a diff in the pull request:
https://github.com/apache/spark/pull/18599#discussion_r126639515
--- Diff: sbin/spark-daemon.sh ---
@@ -78,6 +78,12 @@ spark_rotate_log ()
if [ -n "$2" ]; then
num=$2
fi
+
+ expr $num + 0 &>/dev/null
+ if [ $? -ne 0 ] || [ $num -eq 0 ]; then
--- End diff --
@srowen
Yes, we should avoid this scenario that num isn't number.If num isn't a number,num will be set to 5.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org