You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/04/06 16:05:12 UTC

[jira] [Commented] (PHOENIX-1815) Use Spark Data Source API in phoenix-spark module

    [ https://issues.apache.org/jira/browse/PHOENIX-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481237#comment-14481237 ] 

ASF GitHub Bot commented on PHOENIX-1815:
-----------------------------------------

GitHub user jmahonin opened a pull request:

    https://github.com/apache/phoenix/pull/63

    PHOENIX-1815 Use Spark Data Source API in phoenix-spark module

    This allows using the SQLContext.load() functionality to create
    a Phoenix data frame, which also supports push-down on column
    or predicate filtering from Spark SQL.
    
    As well, DataFrame.save() is supported for persisting DataFrames
    back to Phoenix.
    
    This may work with Spark's standalone SQL server mode, but it
    hasn't been tested.
    
    ref:
    https://spark.apache.org/docs/latest/sql-programming-guide.html#data-sources
    
    https://databricks.com/blog/2015/01/09/spark-sql-data-sources-api-unified-data-access-for-the-spark-platform.html

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/FileTrek/phoenix PHOENIX-1815

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/phoenix/pull/63.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #63
    
----
commit c62178a1016a7885bd2c082fd1380c9a3023ca34
Author: Josh Mahonin <jm...@gmail.com>
Date:   2015-03-25T19:40:10Z

    PHOENIX-1815 Use Spark Data Source API in phoenix-spark module
    
    This allows using the SQLContext.load() functionality to create
    a Phoenix data frame, which also supports push-down on column
    or predicate filtering from Spark SQL.
    
    As well, DataFrame.save() is supported for persisting DataFrames
    back to Phoenix.
    
    This may work with Spark's standalone SQL server mode, but it
    hasn't been tested.
    
    ref:
    https://spark.apache.org/docs/latest/sql-programming-guide.html#data-sources
    
    https://databricks.com/blog/2015/01/09/spark-sql-data-sources-api-unified-data-access-for-the-spark-platform.html

----


> Use Spark Data Source API in phoenix-spark module
> -------------------------------------------------
>
>                 Key: PHOENIX-1815
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1815
>             Project: Phoenix
>          Issue Type: New Feature
>            Reporter: Josh Mahonin
>
> Spark 1.3.0 introduces a new 'Data Source' API to standardize load and save methods for different types of data sources.
> The phoenix-spark module should implement the same API for use as a pluggable data store in Spark.
> ref:
> https://spark.apache.org/docs/latest/sql-programming-guide.html#data-sources
>     
> https://databricks.com/blog/2015/01/09/spark-sql-data-sources-api-unified-data-access-for-the-spark-platform.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)