You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by Lefty Leverenz <le...@hortonworks.com> on 2014/10/01 09:27:07 UTC

Re: Review Request 25682: HIVE-6586 - Add new parameters to HiveConf.java after commit HIVE-6037

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25682/
-----------------------------------------------------------

(Updated Oct. 1, 2014, 7:27 a.m.)

Review request for hive, Carl Steinbach, Alan Gates, Navis Ryu, Prasad Mujumdar, and Sergey Shelukhin.

Changes
-------

Patch 2 adds descriptions for 7 parameters, removes angle brackets from descriptions of 2 parameters, improves descriptions of 24 parameters, and makes trivial edits and format changes to many more.
The non-trivial changes are listed in a comment on HIVE-6586, along with JIRAs and people who created the parameters. A generated hive-default.xml.template file is attached to HIVE-6586 in case you don't want to bother generating your own copy.

Looks like I uploaded wrong, using Add File ... so redoing it with Update Diff.

Bugs: HIVE-6586
https://issues.apache.org/jira/browse/HIVE-6586

Repository: hive

Description
-------

HIVE-6586 kept track of new configuration parameters and changes to parameter descriptions when HIVE-6037 moved parameter descriptions into HiveConf.java from hive-default.xml.template.

HIVE-6586.patch addresses all the fixes listed in the JIRA comments (except ones that had already been fixed), tidies up some line breaks, and makes minor edits to parameter descriptions. It also revises the descriptions of hive.txn.xxx, hive.compactor.xxx, hive.server2.async.exec.shutdown.timeout, and hive.security.authorization.createtable.owner.grants.

Diffs (updated)
-----

trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1628586

Diff: https://reviews.apache.org/r/25682/diff/

Testing
-------

Generated hive-default.xml.template (attached to HIVE-6586) from the new HiveConf.java and reviewed the changed parameter descriptions.

File Attachments (updated)
----------------

Patch 2, rebased and fixed some issues
https://reviews.apache.org/media/uploaded/files/2014/10/01/8e4b539e-2590-4d8e-b3b5-45175a051f9d__HIVE-6586.2.patch

Thanks,

Lefty Leverenz

Re: Review Request 25682: HIVE-6586 - Add new parameters to HiveConf.java after commit HIVE-6037

Posted by Lefty Leverenz <le...@gmail.com>.


> On Oct. 2, 2014, 10:08 p.m., Alan Gates wrote:
> > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 1322
> > <https://reviews.apache.org/r/25682/diff/2/?file=710033#file710033line1322>
> >
> >     Rather than saying "to turn on Hive transactions" this should read "as part of turning on Hive transactions".  This change alone won't turn on transactions.

Agreed.  How about hive.txn.manager?  ("To turn on Hive transactions, set to org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.")  Should all 3 parameters list the others that are required?  Oh ... make that 4 parameters, including hive.support.concurrency.

For example:  "Set this to true on one instance of the Thrift metastore service as part of turning on Hive transactions.  The parameters hive.txn.manager, hive.compactor.worker.threads, and hive.support.concurrency must also be set appropriately to turn on Hive transactions."


- Lefty


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25682/#review55256
-----------------------------------------------------------


On Oct. 1, 2014, 7:27 a.m., Lefty Leverenz wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25682/
> -----------------------------------------------------------
> 
> (Updated Oct. 1, 2014, 7:27 a.m.)
> 
> 
> Review request for hive, Carl Steinbach, Alan Gates, Navis Ryu, Prasad Mujumdar, and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-6586
>     https://issues.apache.org/jira/browse/HIVE-6586
> 
> 
> Repository: hive
> 
> 
> Description
> -------
> 
> HIVE-6586 kept track of new configuration parameters and changes to parameter descriptions when HIVE-6037 moved parameter descriptions into HiveConf.java from hive-default.xml.template.
> 
> HIVE-6586.patch addresses all the fixes listed in the JIRA comments (except ones that had already been fixed), tidies up some line breaks, and makes minor edits to parameter descriptions.  It also revises the descriptions of hive.txn.xxx, hive.compactor.xxx, hive.server2.async.exec.shutdown.timeout, and hive.security.authorization.createtable.owner.grants.
> 
> 
> Diffs
> -----
> 
>   trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1628586 
> 
> Diff: https://reviews.apache.org/r/25682/diff/
> 
> 
> Testing
> -------
> 
> Generated hive-default.xml.template (attached to HIVE-6586) from the new HiveConf.java and reviewed the changed parameter descriptions.
> 
> 
> File Attachments
> ----------------
> 
> Patch 2, rebased and fixed some issues
>   https://reviews.apache.org/media/uploaded/files/2014/10/01/8e4b539e-2590-4d8e-b3b5-45175a051f9d__HIVE-6586.2.patch
> 
> 
> Thanks,
> 
> Lefty Leverenz
> 
>

Re: Review Request 25682: HIVE-6586 - Add new parameters to HiveConf.java after commit HIVE-6037

Posted by Lefty Leverenz <le...@gmail.com>.


> On Oct. 2, 2014, 10:08 p.m., Alan Gates wrote:
> > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 1322
> > <https://reviews.apache.org/r/25682/diff/2/?file=710033#file710033line1322>
> >
> >     Rather than saying "to turn on Hive transactions" this should read "as part of turning on Hive transactions".  This change alone won't turn on transactions.
> 
> Lefty Leverenz wrote:
>     Agreed.  How about hive.txn.manager?  ("To turn on Hive transactions, set to org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.")  Should all 3 parameters list the others that are required?  Oh ... make that 4 parameters, including hive.support.concurrency.
>     
>     For example:  "Set this to true on one instance of the Thrift metastore service as part of turning on Hive transactions.  The parameters hive.txn.manager, hive.compactor.worker.threads, and hive.support.concurrency must also be set appropriately to turn on Hive transactions."
> 
> Alan Gates wrote:
>     I don't know how verbose you want to get in HiveConf.java.  It might make sense to detail all the required parameters under hive.txn.manager and then have pointers to it from the compactor ones.  I don't think you need pointers from hive.support.concurrency since that has other uses beyond just turning on transactions.

Good solution -- easy to maintain.  Thanks.


- Lefty


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25682/#review55256
-----------------------------------------------------------


On Oct. 1, 2014, 7:27 a.m., Lefty Leverenz wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25682/
> -----------------------------------------------------------
> 
> (Updated Oct. 1, 2014, 7:27 a.m.)
> 
> 
> Review request for hive, Carl Steinbach, Alan Gates, Navis Ryu, Prasad Mujumdar, and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-6586
>     https://issues.apache.org/jira/browse/HIVE-6586
> 
> 
> Repository: hive
> 
> 
> Description
> -------
> 
> HIVE-6586 kept track of new configuration parameters and changes to parameter descriptions when HIVE-6037 moved parameter descriptions into HiveConf.java from hive-default.xml.template.
> 
> HIVE-6586.patch addresses all the fixes listed in the JIRA comments (except ones that had already been fixed), tidies up some line breaks, and makes minor edits to parameter descriptions.  It also revises the descriptions of hive.txn.xxx, hive.compactor.xxx, hive.server2.async.exec.shutdown.timeout, and hive.security.authorization.createtable.owner.grants.
> 
> 
> Diffs
> -----
> 
>   trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1628586 
> 
> Diff: https://reviews.apache.org/r/25682/diff/
> 
> 
> Testing
> -------
> 
> Generated hive-default.xml.template (attached to HIVE-6586) from the new HiveConf.java and reviewed the changed parameter descriptions.
> 
> 
> File Attachments
> ----------------
> 
> Patch 2, rebased and fixed some issues
>   https://reviews.apache.org/media/uploaded/files/2014/10/01/8e4b539e-2590-4d8e-b3b5-45175a051f9d__HIVE-6586.2.patch
> 
> 
> Thanks,
> 
> Lefty Leverenz
> 
>

Re: Review Request 25682: HIVE-6586 - Add new parameters to HiveConf.java after commit HIVE-6037

Posted by Alan Gates <ga...@hortonworks.com>.


> On Oct. 2, 2014, 10:08 p.m., Alan Gates wrote:
> > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 1322
> > <https://reviews.apache.org/r/25682/diff/2/?file=710033#file710033line1322>
> >
> >     Rather than saying "to turn on Hive transactions" this should read "as part of turning on Hive transactions".  This change alone won't turn on transactions.
> 
> Lefty Leverenz wrote:
>     Agreed.  How about hive.txn.manager?  ("To turn on Hive transactions, set to org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.")  Should all 3 parameters list the others that are required?  Oh ... make that 4 parameters, including hive.support.concurrency.
>     
>     For example:  "Set this to true on one instance of the Thrift metastore service as part of turning on Hive transactions.  The parameters hive.txn.manager, hive.compactor.worker.threads, and hive.support.concurrency must also be set appropriately to turn on Hive transactions."

I don't know how verbose you want to get in HiveConf.java.  It might make sense to detail all the required parameters under hive.txn.manager and then have pointers to it from the compactor ones.  I don't think you need pointers from hive.support.concurrency since that has other uses beyond just turning on transactions.


- Alan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25682/#review55256
-----------------------------------------------------------


On Oct. 1, 2014, 7:27 a.m., Lefty Leverenz wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25682/
> -----------------------------------------------------------
> 
> (Updated Oct. 1, 2014, 7:27 a.m.)
> 
> 
> Review request for hive, Carl Steinbach, Alan Gates, Navis Ryu, Prasad Mujumdar, and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-6586
>     https://issues.apache.org/jira/browse/HIVE-6586
> 
> 
> Repository: hive
> 
> 
> Description
> -------
> 
> HIVE-6586 kept track of new configuration parameters and changes to parameter descriptions when HIVE-6037 moved parameter descriptions into HiveConf.java from hive-default.xml.template.
> 
> HIVE-6586.patch addresses all the fixes listed in the JIRA comments (except ones that had already been fixed), tidies up some line breaks, and makes minor edits to parameter descriptions.  It also revises the descriptions of hive.txn.xxx, hive.compactor.xxx, hive.server2.async.exec.shutdown.timeout, and hive.security.authorization.createtable.owner.grants.
> 
> 
> Diffs
> -----
> 
>   trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1628586 
> 
> Diff: https://reviews.apache.org/r/25682/diff/
> 
> 
> Testing
> -------
> 
> Generated hive-default.xml.template (attached to HIVE-6586) from the new HiveConf.java and reviewed the changed parameter descriptions.
> 
> 
> File Attachments
> ----------------
> 
> Patch 2, rebased and fixed some issues
>   https://reviews.apache.org/media/uploaded/files/2014/10/01/8e4b539e-2590-4d8e-b3b5-45175a051f9d__HIVE-6586.2.patch
> 
> 
> Thanks,
> 
> Lefty Leverenz
> 
>

Re: Review Request 25682: HIVE-6586 - Add new parameters to HiveConf.java after commit HIVE-6037

Posted by Alan Gates <ga...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25682/#review55256
-----------------------------------------------------------



trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/25682/#comment95645>

    Rather than saying "to turn on Hive transactions" this should read "as part of turning on Hive transactions".  This change alone won't turn on transactions.



trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/25682/#comment95647>

    Same comment as above about turning on transactions.


- Alan Gates


On Oct. 1, 2014, 7:27 a.m., Lefty Leverenz wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25682/
> -----------------------------------------------------------
> 
> (Updated Oct. 1, 2014, 7:27 a.m.)
> 
> 
> Review request for hive, Carl Steinbach, Alan Gates, Navis Ryu, Prasad Mujumdar, and Sergey Shelukhin.
> 
> 
> Bugs: HIVE-6586
>     https://issues.apache.org/jira/browse/HIVE-6586
> 
> 
> Repository: hive
> 
> 
> Description
> -------
> 
> HIVE-6586 kept track of new configuration parameters and changes to parameter descriptions when HIVE-6037 moved parameter descriptions into HiveConf.java from hive-default.xml.template.
> 
> HIVE-6586.patch addresses all the fixes listed in the JIRA comments (except ones that had already been fixed), tidies up some line breaks, and makes minor edits to parameter descriptions.  It also revises the descriptions of hive.txn.xxx, hive.compactor.xxx, hive.server2.async.exec.shutdown.timeout, and hive.security.authorization.createtable.owner.grants.
> 
> 
> Diffs
> -----
> 
>   trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1628586 
> 
> Diff: https://reviews.apache.org/r/25682/diff/
> 
> 
> Testing
> -------
> 
> Generated hive-default.xml.template (attached to HIVE-6586) from the new HiveConf.java and reviewed the changed parameter descriptions.
> 
> 
> File Attachments
> ----------------
> 
> Patch 2, rebased and fixed some issues
>   https://reviews.apache.org/media/uploaded/files/2014/10/01/8e4b539e-2590-4d8e-b3b5-45175a051f9d__HIVE-6586.2.patch
> 
> 
> Thanks,
> 
> Lefty Leverenz
> 
>

Re: Review Request 25682: HIVE-6586 - Add new parameters to HiveConf.java after commit HIVE-6037

Posted by Lefty Leverenz <le...@hortonworks.com>.

(Updated Oct. 28, 2014, 6:06 a.m.)

Review request for hive, Alan Gates, Navis Ryu, Prasad Mujumdar, and Sergey Shelukhin.

Changes
-------

Patch 4 is based on trunk and addresses Alan Gate's review suggestions. HIVE-6586 has a list of all the changes and a template file.

Bugs: HIVE-6586
https://issues.apache.org/jira/browse/HIVE-6586

Repository: hive

Description (updated)
-------

HIVE-6586 kept track of new configuration parameters and changes to parameter descriptions when HIVE-6037 moved parameter descriptions into HiveConf.java from hive-default.xml.template.

Patch 3: HIVE-6586.3.patch omits minor edits and most of the line break changes, and addresses Alan Gates' review suggestions. For a complete list of changed parameter descriptions and a generated hive-default.xml.template file for patch 3, see the JIRA.

Patch 4: Same as patch 3, but trunk-based.

Diffs (updated)
-----

trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1634265

Diff: https://reviews.apache.org/r/25682/diff/

Testing
-------

Generated hive-default.xml.template (attached to HIVE-6586) from the new HiveConf.java and reviewed the changed parameter descriptions.

File Attachments
----------------

Patch 2, rebased and fixed some issues
https://reviews.apache.org/media/uploaded/files/2014/10/01/8e4b539e-2590-4d8e-b3b5-45175a051f9d__HIVE-6586.2.patch

Thanks,

Lefty Leverenz

Re: Review Request 25682: HIVE-6586 - Add new parameters to HiveConf.java after commit HIVE-6037

Posted by Lefty Leverenz <le...@hortonworks.com>.

(Updated Oct. 27, 2014, 8:33 a.m.)

Review request for hive, Alan Gates, Navis Ryu, Prasad Mujumdar, and Sergey Shelukhin.

Changes
-------

Patch 3 addresses Alan's comments and omits inessential changes (minor edits and line breaks). HIVE-6586 has a list of changes and a generated template file.

This patch is for branch-0.14 but when I named that as the base directory I got an error, so I called the base directory "trunk" and then the upload worked. The svn checkout was revision 1634265 but I left that out in the update process because I thought it might have caused the error.

A separate patch will be available for trunk eventually. (Right now I can't generate a template file in trunk due to a missing pom for org.apache.hive:hive-shims:jar:0.15.0-SNAPSHOT.)

Bugs: HIVE-6586
https://issues.apache.org/jira/browse/HIVE-6586

Repository: hive

Description
-------

HIVE-6586 kept track of new configuration parameters and changes to parameter descriptions when HIVE-6037 moved parameter descriptions into HiveConf.java from hive-default.xml.template.

Diffs (updated)
-----

trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1634265

Diff: https://reviews.apache.org/r/25682/diff/

Testing
-------

Generated hive-default.xml.template (attached to HIVE-6586) from the new HiveConf.java and reviewed the changed parameter descriptions.

File Attachments
----------------

Patch 2, rebased and fixed some issues
https://reviews.apache.org/media/uploaded/files/2014/10/01/8e4b539e-2590-4d8e-b3b5-45175a051f9d__HIVE-6586.2.patch

Thanks,

Lefty Leverenz

Re: Review Request 25682: HIVE-6586 - Add new parameters to HiveConf.java after commit HIVE-6037

Posted by Lefty Leverenz <le...@hortonworks.com>.

(Updated Oct. 27, 2014, 8:19 a.m.)

Review request for hive, Alan Gates, Navis Ryu, Prasad Mujumdar, and Sergey Shelukhin.

Bugs: HIVE-6586
https://issues.apache.org/jira/browse/HIVE-6586

Repository: hive

Description (updated)
-------

HIVE-6586 kept track of new configuration parameters and changes to parameter descriptions when HIVE-6037 moved parameter descriptions into HiveConf.java from hive-default.xml.template.

Diffs (updated)
-----

trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1634265

Diff: https://reviews.apache.org/r/25682/diff/

Testing
-------

Generated hive-default.xml.template (attached to HIVE-6586) from the new HiveConf.java and reviewed the changed parameter descriptions.

File Attachments
----------------

Patch 2, rebased and fixed some issues
https://reviews.apache.org/media/uploaded/files/2014/10/01/8e4b539e-2590-4d8e-b3b5-45175a051f9d__HIVE-6586.2.patch

Thanks,

Lefty Leverenz