You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Hao (JIRA)" <ji...@apache.org> on 2014/06/11 04:34:01 UTC
[jira] [Updated] (SPARK-2106) Unify the HiveContext
[ https://issues.apache.org/jira/browse/SPARK-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng Hao updated SPARK-2106:
-----------------------------
Description:
I've been working on CLI for Catalyst, and from the CLI point of view, HiveContext may be required some changes as:
- SessionState management
`SessionState` instance creation & initialization should be done within the wrappers, e.g. in `SharkCliDriver` or `CLIService` etc. cause they know better how to load the user configuration and logger redirection etc. And in HiveContext, it can retrieve SessionState by calling `SessionState.get()`.
- HiveContext API may not be enough for CLI
Retrieving the schema from the output of `SELECT`; but `QueryExecution` is hidden for CLI.
Get each HQL result, besides the string based result, but also we need to know the error code, as well as the call stack if exceptions thrown.
was:
I've been working on CLI for Catalyst, and from the CLI point of view, HiveContext may be required some changes as:
- SessionState management
`SessionState` instance creation & initialization should be done within the wrappers, e.g. in `SharkCliDriver` or `CLIService` etc. cause they know better how to load the use configuration and logger output redirection etc. And in HiveContext, it can retrieve SessionState by calling `SessionState.get()`.
- HiveContext API may not be enough for CLI
Retrieving the schema from the output of `SELECT`; but `QueryExecution` is hidden for CLI.
Get each HQL result, besides the string based result, but also we need to know the error code, as well as the call stack if exceptions thrown.
> Unify the HiveContext
> ---------------------
>
> Key: SPARK-2106
> URL: https://issues.apache.org/jira/browse/SPARK-2106
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Cheng Hao
>
> I've been working on CLI for Catalyst, and from the CLI point of view, HiveContext may be required some changes as:
> - SessionState management
> `SessionState` instance creation & initialization should be done within the wrappers, e.g. in `SharkCliDriver` or `CLIService` etc. cause they know better how to load the user configuration and logger redirection etc. And in HiveContext, it can retrieve SessionState by calling `SessionState.get()`.
> - HiveContext API may not be enough for CLI
> Retrieving the schema from the output of `SELECT`; but `QueryExecution` is hidden for CLI.
> Get each HQL result, besides the string based result, but also we need to know the error code, as well as the call stack if exceptions thrown.
--
This message was sent by Atlassian JIRA
(v6.2#6252)