You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@kudu.apache.org by Benjamin Kim <bb...@gmail.com> on 2017/03/27 16:48:24 UTC

Spark on Kudu Roadmap

Hi,

Are there any plans for deeper integration with Spark especially Spark SQL? Is there a roadmap to look at, so I can know what to expect in the future?

Cheers,
Ben

Re: Spark on Kudu Roadmap

Posted by Benjamin Kim <bb...@gmail.com>.

Hi Mike,

Thanks for the link. I guess further, deeper Spark integration is slowly coming. But when, we will have to wait and see.

Cheers,
Ben
 

> On Mar 27, 2017, at 12:25 PM, Mike Percy <mp...@apache.org> wrote:
> 
> Hi Ben,
> I don't really know so I'll let someone else more familiar with the Spark integration chime in on that. However I searched the Kudu JIRA and I don't see a tracking ticket filed on this (the closest thing I could find was https://issues.apache.org/jira/browse/KUDU-1676 <https://issues.apache.org/jira/browse/KUDU-1676> ) so you may want to file a JIRA to help track this feature.
> 
> Mike
> 
> 
> On Mon, Mar 27, 2017 at 11:55 AM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
> Hi Mike,
> 
> I believe what we are looking for is this below. It is an often request use case.
> 
> Anyone know if the Spark package will ever allow for creating tables in Spark SQL?
> 
> Such as:
>        CREATE EXTERNAL TABLE <table-name>
>        USING org.apache.kudu.spark.kudu
>        OPTIONS (Map("kudu.master" -> “<kudu-master>", "kudu.table" -> “table-name”));
> 
> In this way, plain SQL can be used to do DDL, DML statements whether in Spark SQL code or using JDBC to interface with Spark SQL Thriftserver.
> 
> Thanks,
> Ben
> 
> 
> 
>> On Mar 27, 2017, at 11:01 AM, Mike Percy <mpercy@apache.org <ma...@apache.org>> wrote:
>> 
>> Hi Ben,
>> Is there anything in particular you are looking for?
>> 
>> Thanks,
>> Mike
>> 
>> On Mon, Mar 27, 2017 at 9:48 AM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>> 
>> Are there any plans for deeper integration with Spark especially Spark SQL? Is there a roadmap to look at, so I can know what to expect in the future?
>> 
>> Cheers,
>> Ben
>> 
> 
>

Re: Spark on Kudu Roadmap

Posted by Mike Percy <mp...@apache.org>.

Hi Ben,
I don't really know so I'll let someone else more familiar with the Spark
integration chime in on that. However I searched the Kudu JIRA and I don't
see a tracking ticket filed on this (the closest thing I could find was
https://issues.apache.org/jira/browse/KUDU-1676 ) so you may want to file a
JIRA to help track this feature.

Mike


On Mon, Mar 27, 2017 at 11:55 AM, Benjamin Kim <bb...@gmail.com> wrote:

> Hi Mike,
>
> I believe what we are looking for is this below. It is an often request
> use case.
>
> Anyone know if the Spark package will ever allow for creating tables in
> Spark SQL?
>
> Such as:
>        CREATE EXTERNAL TABLE <table-name>
>        USING org.apache.kudu.spark.kudu
>        OPTIONS (Map("kudu.master" -> “<kudu-master>", "kudu.table" ->
> “table-name”));
>
> In this way, plain SQL can be used to do DDL, DML statements whether in
> Spark SQL code or using JDBC to interface with Spark SQL Thriftserver.
>
>
> Thanks,
> Ben
>
>
>
> On Mar 27, 2017, at 11:01 AM, Mike Percy <mp...@apache.org> wrote:
>
> Hi Ben,
> Is there anything in particular you are looking for?
>
> Thanks,
> Mike
>
> On Mon, Mar 27, 2017 at 9:48 AM, Benjamin Kim <bb...@gmail.com> wrote:
>
>> Hi,
>>
>> Are there any plans for deeper integration with Spark especially Spark
>> SQL? Is there a roadmap to look at, so I can know what to expect in the
>> future?
>>
>> Cheers,
>> Ben
>
>
>
>

Re: Spark on Kudu Roadmap

Posted by Benjamin Kim <bb...@gmail.com>.

Hi Mike,

I believe what we are looking for is this below. It is an often request use case.

Anyone know if the Spark package will ever allow for creating tables in Spark SQL?

Such as:
       CREATE EXTERNAL TABLE <table-name>
       USING org.apache.kudu.spark.kudu
       OPTIONS (Map("kudu.master" -> “<kudu-master>", "kudu.table" -> “table-name”));

In this way, plain SQL can be used to do DDL, DML statements whether in Spark SQL code or using JDBC to interface with Spark SQL Thriftserver.

Thanks,
Ben

> On Mar 27, 2017, at 11:01 AM, Mike Percy <mp...@apache.org> wrote:
> 
> Hi Ben,
> Is there anything in particular you are looking for?
> 
> Thanks,
> Mike
> 
> On Mon, Mar 27, 2017 at 9:48 AM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
> Hi,
> 
> Are there any plans for deeper integration with Spark especially Spark SQL? Is there a roadmap to look at, so I can know what to expect in the future?
> 
> Cheers,
> Ben
>

Re: Spark on Kudu Roadmap

Posted by Mike Percy <mp...@apache.org>.

Hi Ben,
Is there anything in particular you are looking for?

Thanks,
Mike

On Mon, Mar 27, 2017 at 9:48 AM, Benjamin Kim <bb...@gmail.com> wrote:

> Hi,
>
> Are there any plans for deeper integration with Spark especially Spark
> SQL? Is there a roadmap to look at, so I can know what to expect in the
> future?
>
> Cheers,
> Ben