Re: Review Request 14326: HIVE-4629. HS2 should support an API to retrieve query logs

[Thejas Nair <th...@hortonworks.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14326/#review44414
-----------------------------------------------------------



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/14326/#comment78792>

    It will be also useful to have the size unit indication in the name ( eg log.size.kb)
    



jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
<https://reviews.apache.org/r/14326/#comment78804>

    So far this has been considered an hive internal class (not an external user api). This change seems to be aiming at making this function available to non-hive users.
    I think that is OK, but then we should document that explicitly.
    



service/src/java/org/apache/hive/service/cli/CLIService.java
<https://reviews.apache.org/r/14326/#comment78817>

    I don't think the logs from commands such as getOperationStatus should go into this operation log.
    I don't think it will be useful for use cases such as Hue.
    



service/src/java/org/apache/hive/service/cli/log/LogManager.java
<https://reviews.apache.org/r/14326/#comment78816>

    I agree with Carl, I think this is better as an object in OperationManager rather than a service.
    



service/src/java/org/apache/hive/service/cli/log/LogManager.java
<https://reviews.apache.org/r/14326/#comment78815>

    Isn't this re-implementing the functionality of ThreadLocal ? Should we just use ThreadLocal for it ? Does that not work because log4j does the logging asynchronously ?
    


- Thejas Nair


On Sept. 25, 2013, 12:08 a.m., Shreepadma Venugopalan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14326/
> -----------------------------------------------------------
> 
> (Updated Sept. 25, 2013, 12:08 a.m.)
> 
> 
> Review request for hive and Brock Noland.
> 
> 
> Bugs: HIVE-4629
>     https://issues.apache.org/jira/browse/HIVE-4629
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Adds a new API to HS2, String getLog(OperationHandle opHandle) that returns the query log for a given operation handle. The log is maintained in memory as a circular buffer. The default size is 128 KB, but can be configured by the user. Logging is initialized if hive.server2.in.mem.logging is set to true.
> 
> Log object is created in executeStatement,getColumns,getTables,getSchemas,getCatalogs,getTypeInfo,getFunctions and destroyed in closeOperation, cancelOperation.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e971644 
>   conf/hive-default.xml.template 1ee756c 
>   jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java 2912ece 
>   jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java 09ab3c2 
>   service/if/TCLIService.thrift 6e20375 
>   service/src/java/org/apache/hive/service/cli/CLIService.java 1a7f338 
>   service/src/java/org/apache/hive/service/cli/CLIServiceClient.java 14ef54f 
>   service/src/java/org/apache/hive/service/cli/EmbeddedCLIServiceClient.java 9dca874 
>   service/src/java/org/apache/hive/service/cli/ICLIService.java f647ce6 
>   service/src/java/org/apache/hive/service/cli/log/LinkedStringBuffer.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/log/LogDivertAppender.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/log/LogManager.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/log/OperationLog.java PRE-CREATION 
>   service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 1f78a1d 
>   service/src/java/org/apache/hive/service/cli/session/HiveSession.java 00058cc 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 11c96b2 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java f392d62 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 2f2866f 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIServiceClient.java 9bb2a0f 
> 
> Diff: https://reviews.apache.org/r/14326/diff/
> 
> 
> Testing
> -------
> 
> Add a new unit test to test log retrieval.
> 
> 
> Thanks,
> 
> Shreepadma Venugopalan
> 
>