You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Michi Mutsuzaki (JIRA)" <ji...@apache.org> on 2008/12/01 07:54:44 UTC
[jira] Created: (HIVE-88) hadoop doesn't use
conf/hive-log4j.properties
hadoop doesn't use conf/hive-log4j.properties
---------------------------------------------
Key: HIVE-88
URL: https://issues.apache.org/jira/browse/HIVE-88
Project: Hadoop Hive
Issue Type: Improvement
Affects Versions: 0.20.0
Reporter: Michi Mutsuzaki
Priority: Minor
Fix For: 0.20.0
hadoop-0.20.0-dev-core.jar contains log4j.properties file, and I think that's the one hadoop is picking up. I modified both conf/hive-log4j.properties and hadoopcore/conf/log4j.properties, but hadoop still printed INFO messages to stderr.
Pasting relevant posts from the mailing list below:
Michi Mutsuzaki <mi...@cs.stanford.edu> Fri, Nov 28, 2008 at 7:14 PM
To: hive-users@publists.facebook.com
Hello,
When I do "ant test" under ql directory, I get many log messages to stderr.
[junit] 08/11/28 19:04:14 INFO exec.MapOperator: Got partitions: null
[junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Initializing Self
[junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Using tag = -1
[junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
Sort order is ""
[junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
Sort order is ""
....
I tried setting log level to ERROR in conf/hive-log4j.properties, but these info lines still show up. How can I get rid of them?
Thanks!
--Michi
Joydeep Sen Sarma <js...@facebook.com> Fri, Nov 28, 2008 at 10:49 PM
To: "michi@cs.stanford.edu" <mi...@cs.stanford.edu>, "hive-users@publists.facebook.com" <hi...@publists.facebook.com>
When we run the tests - we run in hadoop 'local' mode - and in this mode, we run map-reduce jobs by invoking 'hadoop jar ... ExecDriver' cmd line. this was done because we had some issues submitting map-reduce jobs directly (from same jvm) in local mode that we could not resolve.
The issue is that when we invoke 'hadoop jar ... ExecDriver' - we don't control log4j via hive-log4j. one thing u can try is changing the hadoop's log4j.properties that hive is picking up (probably hadoopcore/conf/log4j.properties).
Revisiting this after a long time - I think this can be fixed with some changes to MapRedTask.java (need to add hive-log4j.properties to hadoop classpath here and then reset log4j using this in execdriver). Feel free to file a jira if this is too irritating ..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-88) hadoop doesn't use
conf/hive-log4j.properties
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joydeep Sen Sarma reassigned HIVE-88:
-------------------------------------
Assignee: Joydeep Sen Sarma
> hadoop doesn't use conf/hive-log4j.properties
> ---------------------------------------------
>
> Key: HIVE-88
> URL: https://issues.apache.org/jira/browse/HIVE-88
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Configuration
> Affects Versions: 0.20.0
> Reporter: Michi Mutsuzaki
> Assignee: Joydeep Sen Sarma
> Priority: Minor
> Fix For: 0.20.0
>
>
> hadoop-0.20.0-dev-core.jar contains log4j.properties file, and I think that's the one hadoop is picking up. I modified both conf/hive-log4j.properties and hadoopcore/conf/log4j.properties, but hadoop still printed INFO messages to stderr.
> Pasting relevant posts from the mailing list below:
> Michi Mutsuzaki <mi...@cs.stanford.edu> Fri, Nov 28, 2008 at 7:14 PM
> To: hive-users@publists.facebook.com
> Hello,
> When I do "ant test" under ql directory, I get many log messages to stderr.
> [junit] 08/11/28 19:04:14 INFO exec.MapOperator: Got partitions: null
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Initializing Self
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Using tag = -1
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> ....
> I tried setting log level to ERROR in conf/hive-log4j.properties, but these info lines still show up. How can I get rid of them?
> Thanks!
> --Michi
> Joydeep Sen Sarma <js...@facebook.com> Fri, Nov 28, 2008 at 10:49 PM
> To: "michi@cs.stanford.edu" <mi...@cs.stanford.edu>, "hive-users@publists.facebook.com" <hi...@publists.facebook.com>
> When we run the tests - we run in hadoop 'local' mode - and in this mode, we run map-reduce jobs by invoking 'hadoop jar ... ExecDriver' cmd line. this was done because we had some issues submitting map-reduce jobs directly (from same jvm) in local mode that we could not resolve.
> The issue is that when we invoke 'hadoop jar ... ExecDriver' - we don't control log4j via hive-log4j. one thing u can try is changing the hadoop's log4j.properties that hive is picking up (probably hadoopcore/conf/log4j.properties).
> Revisiting this after a long time - I think this can be fixed with some changes to MapRedTask.java (need to add hive-log4j.properties to hadoop classpath here and then reset log4j using this in execdriver). Feel free to file a jira if this is too irritating ..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-88) hadoop doesn't use
conf/hive-log4j.properties
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashish Thusoo updated HIVE-88:
------------------------------
Component/s: Configuration
> hadoop doesn't use conf/hive-log4j.properties
> ---------------------------------------------
>
> Key: HIVE-88
> URL: https://issues.apache.org/jira/browse/HIVE-88
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Configuration
> Affects Versions: 0.20.0
> Reporter: Michi Mutsuzaki
> Priority: Minor
> Fix For: 0.20.0
>
>
> hadoop-0.20.0-dev-core.jar contains log4j.properties file, and I think that's the one hadoop is picking up. I modified both conf/hive-log4j.properties and hadoopcore/conf/log4j.properties, but hadoop still printed INFO messages to stderr.
> Pasting relevant posts from the mailing list below:
> Michi Mutsuzaki <mi...@cs.stanford.edu> Fri, Nov 28, 2008 at 7:14 PM
> To: hive-users@publists.facebook.com
> Hello,
> When I do "ant test" under ql directory, I get many log messages to stderr.
> [junit] 08/11/28 19:04:14 INFO exec.MapOperator: Got partitions: null
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Initializing Self
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Using tag = -1
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> ....
> I tried setting log level to ERROR in conf/hive-log4j.properties, but these info lines still show up. How can I get rid of them?
> Thanks!
> --Michi
> Joydeep Sen Sarma <js...@facebook.com> Fri, Nov 28, 2008 at 10:49 PM
> To: "michi@cs.stanford.edu" <mi...@cs.stanford.edu>, "hive-users@publists.facebook.com" <hi...@publists.facebook.com>
> When we run the tests - we run in hadoop 'local' mode - and in this mode, we run map-reduce jobs by invoking 'hadoop jar ... ExecDriver' cmd line. this was done because we had some issues submitting map-reduce jobs directly (from same jvm) in local mode that we could not resolve.
> The issue is that when we invoke 'hadoop jar ... ExecDriver' - we don't control log4j via hive-log4j. one thing u can try is changing the hadoop's log4j.properties that hive is picking up (probably hadoopcore/conf/log4j.properties).
> Revisiting this after a long time - I think this can be fixed with some changes to MapRedTask.java (need to add hive-log4j.properties to hadoop classpath here and then reset log4j using this in execdriver). Feel free to file a jira if this is too irritating ..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-88) hadoop doesn't use
conf/hive-log4j.properties
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson updated HIVE-88:
--------------------------------
Fix Version/s: (was: 0.2.0)
0.4.0
> hadoop doesn't use conf/hive-log4j.properties
> ---------------------------------------------
>
> Key: HIVE-88
> URL: https://issues.apache.org/jira/browse/HIVE-88
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Configuration
> Reporter: Michi Mutsuzaki
> Assignee: Joydeep Sen Sarma
> Priority: Minor
> Fix For: 0.4.0
>
>
> hadoop-0.20.0-dev-core.jar contains log4j.properties file, and I think that's the one hadoop is picking up. I modified both conf/hive-log4j.properties and hadoopcore/conf/log4j.properties, but hadoop still printed INFO messages to stderr.
> Pasting relevant posts from the mailing list below:
> Michi Mutsuzaki <mi...@cs.stanford.edu> Fri, Nov 28, 2008 at 7:14 PM
> To: hive-users@publists.facebook.com
> Hello,
> When I do "ant test" under ql directory, I get many log messages to stderr.
> [junit] 08/11/28 19:04:14 INFO exec.MapOperator: Got partitions: null
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Initializing Self
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Using tag = -1
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> ....
> I tried setting log level to ERROR in conf/hive-log4j.properties, but these info lines still show up. How can I get rid of them?
> Thanks!
> --Michi
> Joydeep Sen Sarma <js...@facebook.com> Fri, Nov 28, 2008 at 10:49 PM
> To: "michi@cs.stanford.edu" <mi...@cs.stanford.edu>, "hive-users@publists.facebook.com" <hi...@publists.facebook.com>
> When we run the tests - we run in hadoop 'local' mode - and in this mode, we run map-reduce jobs by invoking 'hadoop jar ... ExecDriver' cmd line. this was done because we had some issues submitting map-reduce jobs directly (from same jvm) in local mode that we could not resolve.
> The issue is that when we invoke 'hadoop jar ... ExecDriver' - we don't control log4j via hive-log4j. one thing u can try is changing the hadoop's log4j.properties that hive is picking up (probably hadoopcore/conf/log4j.properties).
> Revisiting this after a long time - I think this can be fixed with some changes to MapRedTask.java (need to add hive-log4j.properties to hadoop classpath here and then reset log4j using this in execdriver). Feel free to file a jira if this is too irritating ..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-88) hadoop doesn't use
conf/hive-log4j.properties
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joydeep Sen Sarma resolved HIVE-88.
-----------------------------------
Resolution: Duplicate
> hadoop doesn't use conf/hive-log4j.properties
> ---------------------------------------------
>
> Key: HIVE-88
> URL: https://issues.apache.org/jira/browse/HIVE-88
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Configuration
> Reporter: Michi Mutsuzaki
> Assignee: Joydeep Sen Sarma
> Priority: Minor
>
> hadoop-0.20.0-dev-core.jar contains log4j.properties file, and I think that's the one hadoop is picking up. I modified both conf/hive-log4j.properties and hadoopcore/conf/log4j.properties, but hadoop still printed INFO messages to stderr.
> Pasting relevant posts from the mailing list below:
> Michi Mutsuzaki <mi...@cs.stanford.edu> Fri, Nov 28, 2008 at 7:14 PM
> To: hive-users@publists.facebook.com
> Hello,
> When I do "ant test" under ql directory, I get many log messages to stderr.
> [junit] 08/11/28 19:04:14 INFO exec.MapOperator: Got partitions: null
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Initializing Self
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Using tag = -1
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> ....
> I tried setting log level to ERROR in conf/hive-log4j.properties, but these info lines still show up. How can I get rid of them?
> Thanks!
> --Michi
> Joydeep Sen Sarma <js...@facebook.com> Fri, Nov 28, 2008 at 10:49 PM
> To: "michi@cs.stanford.edu" <mi...@cs.stanford.edu>, "hive-users@publists.facebook.com" <hi...@publists.facebook.com>
> When we run the tests - we run in hadoop 'local' mode - and in this mode, we run map-reduce jobs by invoking 'hadoop jar ... ExecDriver' cmd line. this was done because we had some issues submitting map-reduce jobs directly (from same jvm) in local mode that we could not resolve.
> The issue is that when we invoke 'hadoop jar ... ExecDriver' - we don't control log4j via hive-log4j. one thing u can try is changing the hadoop's log4j.properties that hive is picking up (probably hadoopcore/conf/log4j.properties).
> Revisiting this after a long time - I think this can be fixed with some changes to MapRedTask.java (need to add hive-log4j.properties to hadoop classpath here and then reset log4j using this in execdriver). Feel free to file a jira if this is too irritating ..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-88) hadoop doesn't use
conf/hive-log4j.properties
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-88:
---------------------------
Fix Version/s: (was: 0.5.0)
> hadoop doesn't use conf/hive-log4j.properties
> ---------------------------------------------
>
> Key: HIVE-88
> URL: https://issues.apache.org/jira/browse/HIVE-88
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Configuration
> Reporter: Michi Mutsuzaki
> Assignee: Joydeep Sen Sarma
> Priority: Minor
>
> hadoop-0.20.0-dev-core.jar contains log4j.properties file, and I think that's the one hadoop is picking up. I modified both conf/hive-log4j.properties and hadoopcore/conf/log4j.properties, but hadoop still printed INFO messages to stderr.
> Pasting relevant posts from the mailing list below:
> Michi Mutsuzaki <mi...@cs.stanford.edu> Fri, Nov 28, 2008 at 7:14 PM
> To: hive-users@publists.facebook.com
> Hello,
> When I do "ant test" under ql directory, I get many log messages to stderr.
> [junit] 08/11/28 19:04:14 INFO exec.MapOperator: Got partitions: null
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Initializing Self
> [junit] 08/11/28 19:04:14 INFO exec.ReduceSinkOperator: Using tag = -1
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> [junit] 08/11/28 19:04:14 INFO thrift.TBinarySortableProtocol:
> Sort order is ""
> ....
> I tried setting log level to ERROR in conf/hive-log4j.properties, but these info lines still show up. How can I get rid of them?
> Thanks!
> --Michi
> Joydeep Sen Sarma <js...@facebook.com> Fri, Nov 28, 2008 at 10:49 PM
> To: "michi@cs.stanford.edu" <mi...@cs.stanford.edu>, "hive-users@publists.facebook.com" <hi...@publists.facebook.com>
> When we run the tests - we run in hadoop 'local' mode - and in this mode, we run map-reduce jobs by invoking 'hadoop jar ... ExecDriver' cmd line. this was done because we had some issues submitting map-reduce jobs directly (from same jvm) in local mode that we could not resolve.
> The issue is that when we invoke 'hadoop jar ... ExecDriver' - we don't control log4j via hive-log4j. one thing u can try is changing the hadoop's log4j.properties that hive is picking up (probably hadoopcore/conf/log4j.properties).
> Revisiting this after a long time - I think this can be fixed with some changes to MapRedTask.java (need to add hive-log4j.properties to hadoop classpath here and then reset log4j using this in execdriver). Feel free to file a jira if this is too irritating ..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.