[jira] [Commented] (PHOENIX-2632) Easier Hive->Phoenix data movement

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15120348#comment-15120348 ] 

Josh Elser commented on PHOENIX-2632:
-------------------------------------

[~rgelhau], how do you see something like this getting included into Phoenix? A new Maven module that can sit downstream of phoenix-spark, maybe produce some uber-jar to make classpath stuff easier from the Phoenix side?

Any possibility to add some end-to-end tests? Such tests would be nice to have to help catch future breakages as they happen instead of realizing after a release when someone goes to use it.

In general, any tooling that can help get your data into Phoenix seems like a valuable addition to me. There are many ways to hammer that nail, but this seems like it would be a reasonably general-purpose one to provide.

> Easier Hive->Phoenix data movement
> ----------------------------------
>
>                 Key: PHOENIX-2632
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2632
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Randy Gelhausen
>
> Moving tables or query results from Hive into Phoenix today requires error prone manual schema re-definition inside HBase storage handler properties. 
> Since Hive and Phoenix support near equivalent types, it should be easier for users to pick a Hive table and load it (or derived query results) from it.
> I'm posting this to open design discussion, but also submit my own project https://github.com/randerzander/HiveToPhoenix for consideration as an early solution. It creates a Spark DataFrame from a Hive query, uses Phoenix JDBC to "create if not exists" a Phoenix equivalent table, and uses the phoenix-spark artifact to store the DataFrame into Phoenix.
> I'm eager to get feedback if this is interesting/useful to the Phoenix community.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)