You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@livy.apache.org by simsicon <si...@gmail.com> on 2018/02/06 05:43:42 UTC

Is multiple languages support in 0.5.0 only work with rest-api?

Hi,

I am pretty new to livy, and I am looking into the livy to do some research
if livy is capable as infrastructure of a data mining platform, which means
we may want to pipe different language operators together, so I am excited
to see the multiple languages support in the latest release 0.5.0.

I wonder if we can use scala api to create a session, then post different
language statement to that session? Does this make any sense?

Thanks
Z

-- 

Thanks&Best Regards

Zhang Zheng

Re: Is multiple languages support in 0.5.0 only work with rest-api?

Posted by Saisai Shao <sa...@gmail.com>.

Sorry I don't fully understand your saying. I think you can use Scala/Java
API to do data transformation and register as a table, and then use Python
API to do ML pipeline, not sure what is your specific scenario.

Thanks
Jerry

2018-02-06 18:09 GMT+08:00 simsicon <si...@gmail.com>:

> Thanks Jerry, please let me give an example like, if I am going to submit
> an application that using Scala API, but I would like to prepend a
> customization transformation function using Python at the beginning of the
> application, what should I do exactly? And is SparkSession in SparkContext
>
> <https://livy.incubator.apache.org/docs/latest/api/scala/index.html#org.apache.livy.scalaapi.ScalaJobContext>refers
> to the same concept of session in REST api?
>
> On Tue, Feb 6, 2018 at 3:22 PM, Saisai Shao <sa...@gmail.com>
> wrote:
>
>> IIUC, I assume you're using JobAPI rather than REST APIs. Yes, you can
>> create a session using Java/Scala job API, and then manipulate with Python
>> APIs.
>>
>> Thanks
>> Jerry
>>
>> 2018-02-06 13:43 GMT+08:00 simsicon <si...@gmail.com>:
>>
>>> Hi,
>>>
>>> I am pretty new to livy, and I am looking into the livy to do some
>>> research if livy is capable as infrastructure of a data mining platform,
>>> which means we may want to pipe different language operators together, so I
>>> am excited to see the multiple languages support in the latest release
>>> 0.5.0.
>>>
>>> I wonder if we can use scala api to create a session, then post
>>> different language statement to that session? Does this make any sense?
>>>
>>> Thanks
>>> Z
>>>
>>> --
>>>
>>> Thanks&Best Regards
>>>
>>> Zhang Zheng
>>>
>>
>>
>
>
> --
>
> Thanks&Best Regards
>
> Zhang Zheng
>

Re: Is multiple languages support in 0.5.0 only work with rest-api?

Posted by simsicon <si...@gmail.com>.

Thanks Jerry, please let me give an example like, if I am going to submit
an application that using Scala API, but I would like to prepend a
customization transformation function using Python at the beginning of the
application, what should I do exactly? And is SparkSession in SparkContext
<https://livy.incubator.apache.org/docs/latest/api/scala/index.html#org.apache.livy.scalaapi.ScalaJobContext>refers
to the same concept of session in REST api?

On Tue, Feb 6, 2018 at 3:22 PM, Saisai Shao <sa...@gmail.com> wrote:

> IIUC, I assume you're using JobAPI rather than REST APIs. Yes, you can
> create a session using Java/Scala job API, and then manipulate with Python
> APIs.
>
> Thanks
> Jerry
>
> 2018-02-06 13:43 GMT+08:00 simsicon <si...@gmail.com>:
>
>> Hi,
>>
>> I am pretty new to livy, and I am looking into the livy to do some
>> research if livy is capable as infrastructure of a data mining platform,
>> which means we may want to pipe different language operators together, so I
>> am excited to see the multiple languages support in the latest release
>> 0.5.0.
>>
>> I wonder if we can use scala api to create a session, then post different
>> language statement to that session? Does this make any sense?
>>
>> Thanks
>> Z
>>
>> --
>>
>> Thanks&Best Regards
>>
>> Zhang Zheng
>>
>
>

-- 

Thanks&Best Regards

Zhang Zheng

Re: Is multiple languages support in 0.5.0 only work with rest-api?

Posted by Saisai Shao <sa...@gmail.com>.

IIUC, I assume you're using JobAPI rather than REST APIs. Yes, you can
create a session using Java/Scala job API, and then manipulate with Python
APIs.

Thanks
Jerry

2018-02-06 13:43 GMT+08:00 simsicon <si...@gmail.com>:

> Hi,
>
> I am pretty new to livy, and I am looking into the livy to do some
> research if livy is capable as infrastructure of a data mining platform,
> which means we may want to pipe different language operators together, so I
> am excited to see the multiple languages support in the latest release
> 0.5.0.
>
> I wonder if we can use scala api to create a session, then post different
> language statement to that session? Does this make any sense?
>
> Thanks
> Z
>
> --
>
> Thanks&Best Regards
>
> Zhang Zheng
>