You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kazuki Ohta (JIRA)" <ji...@apache.org> on 2011/05/01 11:45:03 UTC
[jira] [Created] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Enables HiveServer to accept -hiveconf option
---------------------------------------------
Key: HIVE-2139
URL: https://issues.apache.org/jira/browse/HIVE-2139
Project: Hive
Issue Type: Improvement
Components: CLI
Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
Reporter: Kazuki Ohta
Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
{code}
hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
Starting Hive Thrift Server
java.lang.NumberFormatException: For input string: "-hiveconf"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:449)
at java.lang.Integer.parseInt(Integer.java:499)
at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
{code}
Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067933#comment-13067933 ]
jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------
bq. On 2011-07-19 00:08:19, Carl Steinbach wrote:
bq. > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3382
bq. > <https://reviews.apache.org/r/958/diff/2/?file=26007#file26007line3382>
bq. >
bq. > This should be $METASTORE_PORT instead of HIVE_PORT (see metastore.sh).
I missed that, thanks.
bq. On 2011-07-19 00:08:19, Carl Steinbach wrote:
bq. > common/build.xml, line 30
bq. > <https://reviews.apache.org/r/958/diff/2/?file=26001#file26001line30>
bq. >
bq. > This target overrides the definition of 'compile' in build-common.xml, but neglects to include this action which copies over hive-log4j.properties:
bq. >
bq. > <copy todir="${build.classes}" failonerror="false">
bq. > <fileset dir="${src.dir}/conf"/>
bq. > </copy>
bq. >
bq. > If this is added in I think the missing hive-log4j.properties issue will be resolved.
Ok -- FYI I originally copied this section from cli/build.xml, not build-common.xml. (it's the same except for this missing copy)
- Patrick
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/#review1106
-----------------------------------------------------------
On 2011-07-15 23:51:55, Patrick Hunt wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/958/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-07-15 23:51:55)
bq.
bq.
bq. Review request for hive and Carl Steinbach.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
bq.
bq. There's a common HiveCli class that's used by both main classes.
bq.
bq. I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
bq.
bq. commands such as the following now work as expected:
bq.
bq. bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
bq.
bq. as does the following which generates usage information:
bq.
bq. bin/hive --service hiveserver -h
bq.
bq. Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
bq.
bq.
bq. This addresses bug HIVE-2139.
bq. https://issues.apache.org/jira/browse/HIVE-2139
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java 728958d
bq. service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9
bq. ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 090ecfc
bq. ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 61393c8
bq. bin/ext/hiveserver.sh b5edce4
bq. bin/ext/metastore.sh db15f6e
bq. cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java bd9f6b1
bq. common/build.xml 0969e16
bq. common/ivy.xml bee3bf0
bq. common/src/java/org/apache/hadoop/hive/common/LogUtils.java PRE-CREATION
bq. common/src/java/org/apache/hadoop/hive/common/cli/CommonCliOptions.java PRE-CREATION
bq. hwi/src/java/org/apache/hadoop/hive/hwi/HWISessionItem.java 1bf8c97
bq. metastore/ivy.xml 83218fc
bq. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java e540c63
bq.
bq. Diff: https://reviews.apache.org/r/958/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I couldn't find any tests for these changes, so I verfied the changes manually.
bq.
bq.
bq. Thanks,
bq.
bq. Patrick
bq.
bq.
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated HIVE-2139:
-------------------------------
Status: Patch Available (was: Open)
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069163#comment-13069163 ]
Hudson commented on HIVE-2139:
------------------------------
Integrated in Hive-trunk-h0.21 #841 (See [https://builds.apache.org/job/Hive-trunk-h0.21/841/])
HIVE-2139. Enable HiveServer to accept -hiveconf option (Patrick Hunt via cws)
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1149311
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/LogUtils.java
* /hive/trunk/hwi/src/java/org/apache/hadoop/hive/hwi/HWISessionItem.java
* /hive/trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/cli/CommonCliOptions.java
* /hive/trunk/metastore/ivy.xml
* /hive/trunk/bin/ext/metastore.sh
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java
* /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
* /hive/trunk/common/ivy.xml
* /hive/trunk/common/build.xml
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/cli
* /hive/trunk/bin/ext/hiveserver.sh
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Fix For: 0.8.0
>
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2139:
---------------------------------
Resolution: Fixed
Fix Version/s: 0.8.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Patrick!
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Fix For: 0.8.0
>
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated HIVE-2139:
-------------------------------
Status: Patch Available (was: Open)
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated HIVE-2139:
-------------------------------
Attachment: HIVE-2139.patch
This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
There's a common HiveCli class that's used by both main classes.
I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
commands such as the following now work as expected:
bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
as does the following which generates usage information:
bin/hive --service hiveserver -h
Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
I couldn't find any tests for these changes, so I verfied the changes manually.
If these changes go through I'll update the wiki docs (which afaik are the only docs that need to be updated).
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066319#comment-13066319 ]
jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 91
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21668#file21668line91>
bq. >
bq. > We should try to avoid making the metastore dependent on ql. There's already an open ticket (HIVE-850) that covers the task of moving SessionState to common. Looks like now may be a good time to do this.
bq.
bq. Patrick Hunt wrote:
bq. I've refactored the log initialization code in order to allow reuse (moved into common). Took a look at doing more like 850, but it looks like that's going to be a much class to tease apart.
notice as part of this refactoring the log4j initialization method now throws an exception rather than silently ignoring the missing log4j.properties. When I built this (ant package) and run either the hiveserver or metastore it now complains about no log4j.properties found on the classpath (please give it a try). I added a log4j.properties to the conf dir and it was fine. It seems that the exec jar includes a log4j.properties, should other jars also? or it's just up to the user to address this?
- Patrick
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/#review919
-----------------------------------------------------------
On 2011-07-15 23:51:55, Patrick Hunt wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/958/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-07-15 23:51:55)
bq.
bq.
bq. Review request for hive and Carl Steinbach.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
bq.
bq. There's a common HiveCli class that's used by both main classes.
bq.
bq. I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
bq.
bq. commands such as the following now work as expected:
bq.
bq. bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
bq.
bq. as does the following which generates usage information:
bq.
bq. bin/hive --service hiveserver -h
bq.
bq. Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
bq.
bq.
bq. This addresses bug HIVE-2139.
bq. https://issues.apache.org/jira/browse/HIVE-2139
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java 728958d
bq. service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9
bq. ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 090ecfc
bq. ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 61393c8
bq. bin/ext/hiveserver.sh b5edce4
bq. bin/ext/metastore.sh db15f6e
bq. cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java bd9f6b1
bq. common/build.xml 0969e16
bq. common/ivy.xml bee3bf0
bq. common/src/java/org/apache/hadoop/hive/common/LogUtils.java PRE-CREATION
bq. common/src/java/org/apache/hadoop/hive/common/cli/CommonCliOptions.java PRE-CREATION
bq. hwi/src/java/org/apache/hadoop/hive/hwi/HWISessionItem.java 1bf8c97
bq. metastore/ivy.xml 83218fc
bq. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java e540c63
bq.
bq. Diff: https://reviews.apache.org/r/958/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I couldn't find any tests for these changes, so I verfied the changes manually.
bq.
bq.
bq. Thanks,
bq.
bq. Patrick
bq.
bq.
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2139:
---------------------------------
Status: Open (was: Patch Available)
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066312#comment-13066312 ]
jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > I ran into some problems compiling this patch. Please verify that it builds using the ant command 'ant clean package'.
I see. I added a dependency on commons-cli in hive common but didn't specify it explicitly. I've updated ivy dependencies for this.
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java, line 42
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21667#file21667line42>
bq. >
bq. > The name of this class is likely to generate confusion. Maybe change it to CommonCliOpts, or something else?
done
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java, line 29
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21667#file21667line29>
bq. >
bq. > The HiveMetaStore and HiveServer imports are unnecessary.
weird, "organize imports" adds this each time I run it, I removed it manually and it's fine.
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 91
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21668#file21668line91>
bq. >
bq. > We should try to avoid making the metastore dependent on ql. There's already an open ticket (HIVE-850) that covers the task of moving SessionState to common. Looks like now may be a good time to do this.
I've refactored the log initialization code in order to allow reuse (moved into common). Took a look at doing more like 850, but it looks like that's going to be a much class to tease apart.
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > bin/ext/hiveserver.sh, line 31
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21665#file21665line31>
bq. >
bq. > Does this mean that $HIVE_PORT takes precedence over another port specified using the -p switch? If so then I think the reverse makes more sense.
bq. >
bq. > Also, in order to make the precedence explicit, I think it would be good to move this logic to the HiveCli class, e.g. explicitly call System.getenv("HIVE_PORT") from HiveCli.
excellent idea, I didn't think to move the check into the code. done.
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3347
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21668#file21668line3347>
bq. >
bq. > cli.processHiveConf() copies all of the -hiveconf properties into the list of SystemProperties, which I agree we want to do before initializing the logging system, but subsequently we need to make sure that these same key/val properties are are also registered as HiveConf values so that they have the opportunity to override values specified in hive-default.xml and hive-site.xml. A similar trick is done for the CLI via the OptionsProcessor process_stage1() and process_stage2() methods.
I've refactored a bit to allow this. It's tricky given that you need access to HiveConf, which is buried in a couple places. I like the way it's come out in general, however we can't push it into common (the shared cli code) given it would add a dependency on hiveconf.
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > service/src/java/org/apache/hadoop/hive/service/HiveServer.java, line 76
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21669#file21669line76>
bq. >
bq. > hive.metastore.server.[min|max].threads already exists. We should add similar properties for controlling the min/max number of threads for HiveServer.
Happy to do it - but would you mind adding a jira assigned to me? Would like to limit the scope creep on this one.
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > service/src/java/org/apache/hadoop/hive/service/HiveServer.java, line 612
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21669#file21669line612>
bq. >
bq. > Need to read the -hiveconf properties into the SessionState's HiveConf.
is this a bug in the existing code? -- in HiveServerHandler constructor, both the call to super and the constructor itself seem to be creating "new HiveConf" object. See how I've addressed this in the updated patch.
bq. On 2011-06-27 20:51:48, Carl Steinbach wrote:
bq. > service/src/java/org/apache/hadoop/hive/service/HiveServer.java, line 616
bq. > <https://reviews.apache.org/r/958/diff/1/?file=21669#file21669line616>
bq. >
bq. > We should also set options.maxWorkerThreads. Looks like the default value for Thrift is Integer.MAX_VALUE.
added
- Patrick
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/#review919
-----------------------------------------------------------
On 2011-06-24 22:12:48, Patrick Hunt wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/958/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-24 22:12:48)
bq.
bq.
bq. Review request for hive and Carl Steinbach.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
bq.
bq. There's a common HiveCli class that's used by both main classes.
bq.
bq. I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
bq.
bq. commands such as the following now work as expected:
bq.
bq. bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
bq.
bq. as does the following which generates usage information:
bq.
bq. bin/hive --service hiveserver -h
bq.
bq. Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
bq.
bq.
bq. This addresses bug HIVE-2139.
bq. https://issues.apache.org/jira/browse/HIVE-2139
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bin/ext/hiveserver.sh b5edce4
bq. bin/ext/metastore.sh db15f6e
bq. common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java PRE-CREATION
bq. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java bc58bd5
bq. service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9
bq.
bq. Diff: https://reviews.apache.org/r/958/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I couldn't find any tests for these changes, so I verfied the changes manually.
bq.
bq.
bq. Thanks,
bq.
bq. Patrick
bq.
bq.
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066314#comment-13066314 ]
jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/
-----------------------------------------------------------
(Updated 2011-07-15 23:51:55.625387)
Review request for hive and Carl Steinbach.
Changes
-------
Updated based on Carl's feedback.
All unit tests passed except for "org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1" which I don't think is me.
I also exercised on the command line and it all seemed to work.
Summary
-------
This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
There's a common HiveCli class that's used by both main classes.
I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
commands such as the following now work as expected:
bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
as does the following which generates usage information:
bin/hive --service hiveserver -h
Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
This addresses bug HIVE-2139.
https://issues.apache.org/jira/browse/HIVE-2139
Diffs (updated)
-----
ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java 728958d
service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 090ecfc
ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 61393c8
bin/ext/hiveserver.sh b5edce4
bin/ext/metastore.sh db15f6e
cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java bd9f6b1
common/build.xml 0969e16
common/ivy.xml bee3bf0
common/src/java/org/apache/hadoop/hive/common/LogUtils.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/cli/CommonCliOptions.java PRE-CREATION
hwi/src/java/org/apache/hadoop/hive/hwi/HWISessionItem.java 1bf8c97
metastore/ivy.xml 83218fc
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java e540c63
Diff: https://reviews.apache.org/r/958/diff
Testing
-------
I couldn't find any tests for these changes, so I verfied the changes manually.
Thanks,
Patrick
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055748#comment-13055748 ]
jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/#review919
-----------------------------------------------------------
I ran into some problems compiling this patch. Please verify that it builds using the ant command 'ant clean package'.
bin/ext/hiveserver.sh
<https://reviews.apache.org/r/958/#comment1973>
Does this mean that $HIVE_PORT takes precedence over another port specified using the -p switch? If so then I think the reverse makes more sense.
Also, in order to make the precedence explicit, I think it would be good to move this logic to the HiveCli class, e.g. explicitly call System.getenv("HIVE_PORT") from HiveCli.
common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java
<https://reviews.apache.org/r/958/#comment1975>
The HiveMetaStore and HiveServer imports are unnecessary.
common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java
<https://reviews.apache.org/r/958/#comment1974>
The name of this class is likely to generate confusion. Maybe change it to CommonCliOpts, or something else?
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/958/#comment1976>
We should try to avoid making the metastore dependent on ql. There's already an open ticket (HIVE-850) that covers the task of moving SessionState to common. Looks like now may be a good time to do this.
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/958/#comment1977>
cli.processHiveConf() copies all of the -hiveconf properties into the list of SystemProperties, which I agree we want to do before initializing the logging system, but subsequently we need to make sure that these same key/val properties are are also registered as HiveConf values so that they have the opportunity to override values specified in hive-default.xml and hive-site.xml. A similar trick is done for the CLI via the OptionsProcessor process_stage1() and process_stage2() methods.
service/src/java/org/apache/hadoop/hive/service/HiveServer.java
<https://reviews.apache.org/r/958/#comment1978>
hive.metastore.server.[min|max].threads already exists. We should add similar properties for controlling the min/max number of threads for HiveServer.
service/src/java/org/apache/hadoop/hive/service/HiveServer.java
<https://reviews.apache.org/r/958/#comment1979>
Need to read the -hiveconf properties into the SessionState's HiveConf.
service/src/java/org/apache/hadoop/hive/service/HiveServer.java
<https://reviews.apache.org/r/958/#comment1980>
We should also set options.maxWorkerThreads. Looks like the default value for Thrift is Integer.MAX_VALUE.
- Carl
On 2011-06-24 22:12:48, Patrick Hunt wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/958/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-24 22:12:48)
bq.
bq.
bq. Review request for hive and Carl Steinbach.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
bq.
bq. There's a common HiveCli class that's used by both main classes.
bq.
bq. I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
bq.
bq. commands such as the following now work as expected:
bq.
bq. bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
bq.
bq. as does the following which generates usage information:
bq.
bq. bin/hive --service hiveserver -h
bq.
bq. Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
bq.
bq.
bq. This addresses bug HIVE-2139.
bq. https://issues.apache.org/jira/browse/HIVE-2139
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bin/ext/hiveserver.sh b5edce4
bq. bin/ext/metastore.sh db15f6e
bq. common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java PRE-CREATION
bq. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java bc58bd5
bq. service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9
bq.
bq. Diff: https://reviews.apache.org/r/958/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I couldn't find any tests for these changes, so I verfied the changes manually.
bq.
bq.
bq. Thanks,
bq.
bq. Patrick
bq.
bq.
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated HIVE-2139:
-------------------------------
Status: Patch Available (was: Open)
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067934#comment-13067934 ]
jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/
-----------------------------------------------------------
(Updated 2011-07-19 20:15:07.278811)
Review request for hive and Carl Steinbach.
Changes
-------
Updated with Carl's comments.
Summary
-------
This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
There's a common HiveCli class that's used by both main classes.
I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
commands such as the following now work as expected:
bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
as does the following which generates usage information:
bin/hive --service hiveserver -h
Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
This addresses bug HIVE-2139.
https://issues.apache.org/jira/browse/HIVE-2139
Diffs (updated)
-----
common/build.xml 0969e16
common/ivy.xml bee3bf0
common/src/java/org/apache/hadoop/hive/common/LogUtils.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/cli/CommonCliOptions.java PRE-CREATION
hwi/src/java/org/apache/hadoop/hive/hwi/HWISessionItem.java 1bf8c97
metastore/ivy.xml 83218fc
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java e540c63
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 090ecfc
ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 61393c8
ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java 728958d
service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9
cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java bd9f6b1
bin/ext/hiveserver.sh b5edce4
bin/ext/metastore.sh db15f6e
Diff: https://reviews.apache.org/r/958/diff
Testing
-------
I couldn't find any tests for these changes, so I verfied the changes manually.
Thanks,
Patrick
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068173#comment-13068173 ]
Carl Steinbach commented on HIVE-2139:
--------------------------------------
+1. Will commit if tests pass.
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated HIVE-2139:
-------------------------------
Attachment: HIVE-2139.patch
Updated based on Carl's feedback.
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated HIVE-2139:
-------------------------------
Attachment: HIVE-2139.patch
Updated with Carl's comments.
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069328#comment-13069328 ]
Carl Steinbach commented on HIVE-2139:
--------------------------------------
@Patrick: Good idea. I created a page on the wiki for notes like this: https://cwiki.apache.org/confluence/display/Hive/HiveChangeLog
Please add a blurb there and be sure to include a link back to this ticket. Thanks!
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Fix For: 0.8.0
>
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069251#comment-13069251 ]
Patrick Hunt commented on HIVE-2139:
------------------------------------
Ok, I'll update that. Do you all have some common way to handle documenting the fact that pre-0.8.0 it's one way, post 0.8.0 it's another? Is there an example you can point me to?
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Fix For: 0.8.0
>
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054710#comment-13054710 ]
jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/
-----------------------------------------------------------
Review request for hive and Carl Steinbach.
Summary
-------
This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
There's a common HiveCli class that's used by both main classes.
I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
commands such as the following now work as expected:
bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
as does the following which generates usage information:
bin/hive --service hiveserver -h
Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
This addresses bug HIVE-2139.
https://issues.apache.org/jira/browse/HIVE-2139
Diffs
-----
bin/ext/hiveserver.sh b5edce4
bin/ext/metastore.sh db15f6e
common/src/java/org/apache/hadoop/hive/common/cli/HiveCli.java PRE-CREATION
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java bc58bd5
service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9
Diff: https://reviews.apache.org/r/958/diff
Testing
-------
I couldn't find any tests for these changes, so I verfied the changes manually.
Thanks,
Patrick
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069184#comment-13069184 ]
Patrick Hunt commented on HIVE-2139:
------------------------------------
Should I update the docs for this? Where? If so any guidelines for doing so? (version differences for example)
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Fix For: 0.8.0
>
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2139) Enables HiveServer to accept -hiveconf
option
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2139:
---------------------------------
Status: Open (was: Patch Available)
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt reassigned HIVE-2139:
----------------------------------
Assignee: Patrick Hunt
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067411#comment-13067411 ]
jiraposter@reviews.apache.org commented on HIVE-2139:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/958/#review1106
-----------------------------------------------------------
common/build.xml
<https://reviews.apache.org/r/958/#comment2245>
This target overrides the definition of 'compile' in build-common.xml, but neglects to include this action which copies over hive-log4j.properties:
<copy todir="${build.classes}" failonerror="false">
<fileset dir="${src.dir}/conf"/>
</copy>
If this is added in I think the missing hive-log4j.properties issue will be resolved.
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/958/#comment2244>
This should be $METASTORE_PORT instead of HIVE_PORT (see metastore.sh).
- Carl
On 2011-07-15 23:51:55, Patrick Hunt wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/958/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-07-15 23:51:55)
bq.
bq.
bq. Review request for hive and Carl Steinbach.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This patch updates HiveServer and HiveMetastore to add proper cli handling - similar to that used in CliDriver (ie GnuParser).
bq.
bq. There's a common HiveCli class that's used by both main classes.
bq.
bq. I've attempted to make the cli's backward compatible with the prior command line processing. Notice I've "deprecated" (via warnings, but the code still runs) if the old style CLI usage is used.
bq.
bq. commands such as the following now work as expected:
bq.
bq. bin/hive --service hiveserver -t 200 -p 12000 --hiveconf hive.root.logger=DEBUG,console
bq.
bq. as does the following which generates usage information:
bq.
bq. bin/hive --service hiveserver -h
bq.
bq. Note: HiveMetastore as not initializing log4j, I updated the code to do similar to HiveServer (otw the hiveconf hive.root.logger option above didn't work).
bq.
bq.
bq. This addresses bug HIVE-2139.
bq. https://issues.apache.org/jira/browse/HIVE-2139
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ql/src/test/org/apache/hadoop/hive/ql/history/TestHiveHistory.java 728958d
bq. service/src/java/org/apache/hadoop/hive/service/HiveServer.java ea04be9
bq. ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 090ecfc
bq. ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 61393c8
bq. bin/ext/hiveserver.sh b5edce4
bq. bin/ext/metastore.sh db15f6e
bq. cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java bd9f6b1
bq. common/build.xml 0969e16
bq. common/ivy.xml bee3bf0
bq. common/src/java/org/apache/hadoop/hive/common/LogUtils.java PRE-CREATION
bq. common/src/java/org/apache/hadoop/hive/common/cli/CommonCliOptions.java PRE-CREATION
bq. hwi/src/java/org/apache/hadoop/hive/hwi/HWISessionItem.java 1bf8c97
bq. metastore/ivy.xml 83218fc
bq. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java e540c63
bq.
bq. Diff: https://reviews.apache.org/r/958/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. I couldn't find any tests for these changes, so I verfied the changes manually.
bq.
bq.
bq. Thanks,
bq.
bq. Patrick
bq.
bq.
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Attachments: HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2139) Enables HiveServer to accept
-hiveconf option
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069245#comment-13069245 ]
Carl Steinbach commented on HIVE-2139:
--------------------------------------
I think doc updates should go here:
https://cwiki.apache.org/confluence/display/Hive/HiveServer
> Enables HiveServer to accept -hiveconf option
> ---------------------------------------------
>
> Key: HIVE-2139
> URL: https://issues.apache.org/jira/browse/HIVE-2139
> Project: Hive
> Issue Type: Improvement
> Components: CLI
> Environment: Linux + CDH3u0 (Hive 0.7.0+27.1-2~lucid-cdh3)
> Reporter: Kazuki Ohta
> Assignee: Patrick Hunt
> Fix For: 0.8.0
>
> Attachments: HIVE-2139.patch, HIVE-2139.patch, HIVE-2139.patch
>
>
> Currently, I'm trying to test HiveHBaseIntegration on HiveServer. But it doesn't seem to accept -hiveconf command.
> {code}
> hive --service hiveserver -hiveconf hbase.zookeeper.quorum=hdp0,hdp1,hdp2
> Starting Hive Thrift Server
> java.lang.NumberFormatException: For input string: "-hiveconf"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at org.apache.hadoop.hive.service.HiveServer.main(HiveServer.java:382)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> {code}
> Therefore, you need to throw the query like "set hbase.zookeeper.quorum=hdp0,hdp1,hdp2" everytime. It's not convenient for separating the configuration between server-side and client-side.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira