You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by "Tan, Jialiang" <jt...@ea.com> on 2017/11/03 17:36:46 UTC
Livy interpreter Scala code string interpolation
Hi,
When writing Spark Scala code in zeppelin with Livy interpreter, string interpolation does not seem to work. For example:
val devices = sc.objectFile[(VertexId, DeviceAttr)](
s"s3a://${bucket}/${datasetS3Prefix}/${tableName} /dt=${end_dt}/tid=${tid}/${datasetS3Suffix}")
I think the dollar curly brackets were interpreted as Javascript template literals and hence disappear when Livy server gets the message and becomes:
val devices = sc.objectFile[(VertexId, DeviceAttr)](
s"s3a://///dt= /tid= / ")
At the same time a couple of input boxes appears at the bottom of this paragraph in Zeppelin UI having the corresponding titles in the original dollar curly brackets.
Is this a bug? How can we overcome the string interpolation issue?
Thanks,
Jialiang
Re: Livy interpreter Scala code string interpolation
Posted by Jeff Zhang <zj...@gmail.com>.
Spark only enable dynamic form for sql interpreter, while livy enable it
for interpreters. This is the bug of livy interpreter.
Tan, Jialiang <jt...@ea.com>于2017年11月4日周六 上午8:22写道：
> I think for Spark interpreter it can properly process the string
> interpolation without any problems caused by dynamic form.
>
>
>
> *From: *Mohit Jaggi <mo...@gmail.com>
> *Reply-To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
> *Date: *Friday, November 3, 2017 at 5:20 PM
>
>
> *To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
> *Subject: *Re: Livy interpreter Scala code string interpolation
>
>
>
> So for spark interpreter does ${x} create a dynamic textbox or is
> interpreted as string interpolation? Or is it context-sensitive and knows
> about s"...."?
>
>
>
> On Fri, Nov 3, 2017 at 4:59 PM, Jeff Zhang <zj...@gmail.com> wrote:
>
> Oops, this is a bug of livy interpreter, Could you help create a ticket ?
> Thanks
>
>
>
>
>
> Tan, Jialiang <jt...@ea.com>于2017年11月4日周六 上午7:50写道：
>
> But somehow Spark interpreter works without this problem. Why is that the
> case if the problem is on Zeppelin UI side?
>
>
>
> *From: *Jeff Zhang <zj...@gmail.com>
> *Reply-To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
> *Date: *Friday, November 3, 2017 at 4:48 PM
> *To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
> *Subject: *Re: Livy interpreter Scala code string interpolation
>
>
>
>
>
> Yes, this is due to the dynamic forms of zeppelin.
>
>
>
>
>
> Mohit Jaggi <mo...@gmail.com>于2017年11月4日周六 上午2:28写道：
>
> I run into similar issues with shell scripts that use ${var}. Can we use a
> different magic notation for Z, one that has a lower chance of colliding
> with shell scripts and Scala code? This will be hard to do due to the
> variety of interpreters supported. Perhaps this can be made configurable?
>
>
>
> On Fri, Nov 3, 2017 at 10:36 AM, Tan, Jialiang <jt...@ea.com> wrote:
>
> Hi,
>
>
>
> When writing Spark Scala code in zeppelin with Livy interpreter, string
> interpolation does not seem to work. For example:
>
>
>
> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>
> s"s3a://${bucket}/${datasetS3Prefix}/${tableName}
> /dt=${end_dt}/tid=${tid}/${datasetS3Suffix}")
>
>
>
> I think the dollar curly brackets were interpreted as Javascript template
> literals and hence disappear when Livy server gets the message and becomes:
>
>
>
> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>
> s"s3a://///dt= /tid= / ")
>
>
>
> At the same time a couple of input boxes appears at the bottom of this
> paragraph in Zeppelin UI having the corresponding titles in the original
> dollar curly brackets.
>
>
>
> Is this a bug? How can we overcome the string interpolation issue?
>
>
>
> Thanks,
>
> Jialiang
>
>
>
>
>
Re: Livy interpreter Scala code string interpolation
Posted by "Tan, Jialiang" <jt...@ea.com>.
I think for Spark interpreter it can properly process the string interpolation without any problems caused by dynamic form.
From: Mohit Jaggi <mo...@gmail.com>
Reply-To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Date: Friday, November 3, 2017 at 5:20 PM
To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Subject: Re: Livy interpreter Scala code string interpolation
So for spark interpreter does ${x} create a dynamic textbox or is interpreted as string interpolation? Or is it context-sensitive and knows about s"...."?
On Fri, Nov 3, 2017 at 4:59 PM, Jeff Zhang <zj...@gmail.com>> wrote:
Oops, this is a bug of livy interpreter, Could you help create a ticket ? Thanks
Tan, Jialiang <jt...@ea.com>>于2017年11月4日周六 上午7:50写道：
But somehow Spark interpreter works without this problem. Why is that the case if the problem is on Zeppelin UI side?
From: Jeff Zhang <zj...@gmail.com>>
Reply-To: "users@zeppelin.apache.org<ma...@zeppelin.apache.org>" <us...@zeppelin.apache.org>>
Date: Friday, November 3, 2017 at 4:48 PM
To: "users@zeppelin.apache.org<ma...@zeppelin.apache.org>" <us...@zeppelin.apache.org>>
Subject: Re: Livy interpreter Scala code string interpolation
Yes, this is due to the dynamic forms of zeppelin.
Mohit Jaggi <mo...@gmail.com>>于2017年11月4日周六 上午2:28写道：
I run into similar issues with shell scripts that use ${var}. Can we use a different magic notation for Z, one that has a lower chance of colliding with shell scripts and Scala code? This will be hard to do due to the variety of interpreters supported. Perhaps this can be made configurable?
On Fri, Nov 3, 2017 at 10:36 AM, Tan, Jialiang <jt...@ea.com>> wrote:
Hi,
When writing Spark Scala code in zeppelin with Livy interpreter, string interpolation does not seem to work. For example:
val devices = sc.objectFile[(VertexId, DeviceAttr)](
s"s3a://${bucket}/${datasetS3Prefix}/${tableName} /dt=${end_dt}/tid=${tid}/${datasetS3Suffix}")
I think the dollar curly brackets were interpreted as Javascript template literals and hence disappear when Livy server gets the message and becomes:
val devices = sc.objectFile[(VertexId, DeviceAttr)](
s"s3a://///dt= /tid= / ")
At the same time a couple of input boxes appears at the bottom of this paragraph in Zeppelin UI having the corresponding titles in the original dollar curly brackets.
Is this a bug? How can we overcome the string interpolation issue?
Thanks,
Jialiang
Re: Livy interpreter Scala code string interpolation
Posted by Mohit Jaggi <mo...@gmail.com>.
So for spark interpreter does ${x} create a dynamic textbox or is
interpreted as string interpolation? Or is it context-sensitive and knows
about s"...."?
On Fri, Nov 3, 2017 at 4:59 PM, Jeff Zhang <zj...@gmail.com> wrote:
> Oops, this is a bug of livy interpreter, Could you help create a ticket ?
> Thanks
>
>
> Tan, Jialiang <jt...@ea.com>于2017年11月4日周六 上午7:50写道：
>
>> But somehow Spark interpreter works without this problem. Why is that the
>> case if the problem is on Zeppelin UI side?
>>
>>
>>
>> *From: *Jeff Zhang <zj...@gmail.com>
>> *Reply-To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
>> *Date: *Friday, November 3, 2017 at 4:48 PM
>> *To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
>> *Subject: *Re: Livy interpreter Scala code string interpolation
>>
>>
>>
>>
>>
>> Yes, this is due to the dynamic forms of zeppelin.
>>
>>
>>
>>
>>
>> Mohit Jaggi <mo...@gmail.com>于2017年11月4日周六 上午2:28写道：
>>
>> I run into similar issues with shell scripts that use ${var}. Can we use
>> a different magic notation for Z, one that has a lower chance of colliding
>> with shell scripts and Scala code? This will be hard to do due to the
>> variety of interpreters supported. Perhaps this can be made configurable?
>>
>>
>>
>> On Fri, Nov 3, 2017 at 10:36 AM, Tan, Jialiang <jt...@ea.com> wrote:
>>
>> Hi,
>>
>>
>>
>> When writing Spark Scala code in zeppelin with Livy interpreter, string
>> interpolation does not seem to work. For example:
>>
>>
>>
>> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>>
>> s"s3a://${bucket}/${datasetS3Prefix}/${tableName}
>> /dt=${end_dt}/tid=${tid}/${datasetS3Suffix}")
>>
>>
>>
>> I think the dollar curly brackets were interpreted as Javascript template
>> literals and hence disappear when Livy server gets the message and becomes:
>>
>>
>>
>> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>>
>> s"s3a://///dt= /tid= / ")
>>
>>
>>
>> At the same time a couple of input boxes appears at the bottom of this
>> paragraph in Zeppelin UI having the corresponding titles in the original
>> dollar curly brackets.
>>
>>
>>
>> Is this a bug? How can we overcome the string interpolation issue?
>>
>>
>>
>> Thanks,
>>
>> Jialiang
>>
>>
>>
>>
Re: Livy interpreter Scala code string interpolation
Posted by Jeff Zhang <zj...@gmail.com>.
Oops, this is a bug of livy interpreter, Could you help create a ticket ?
Thanks
Tan, Jialiang <jt...@ea.com>于2017年11月4日周六 上午7:50写道：
> But somehow Spark interpreter works without this problem. Why is that the
> case if the problem is on Zeppelin UI side?
>
>
>
> *From: *Jeff Zhang <zj...@gmail.com>
> *Reply-To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
> *Date: *Friday, November 3, 2017 at 4:48 PM
> *To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
> *Subject: *Re: Livy interpreter Scala code string interpolation
>
>
>
>
>
> Yes, this is due to the dynamic forms of zeppelin.
>
>
>
>
>
> Mohit Jaggi <mo...@gmail.com>于2017年11月4日周六 上午2:28写道：
>
> I run into similar issues with shell scripts that use ${var}. Can we use a
> different magic notation for Z, one that has a lower chance of colliding
> with shell scripts and Scala code? This will be hard to do due to the
> variety of interpreters supported. Perhaps this can be made configurable?
>
>
>
> On Fri, Nov 3, 2017 at 10:36 AM, Tan, Jialiang <jt...@ea.com> wrote:
>
> Hi,
>
>
>
> When writing Spark Scala code in zeppelin with Livy interpreter, string
> interpolation does not seem to work. For example:
>
>
>
> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>
> s"s3a://${bucket}/${datasetS3Prefix}/${tableName}
> /dt=${end_dt}/tid=${tid}/${datasetS3Suffix}")
>
>
>
> I think the dollar curly brackets were interpreted as Javascript template
> literals and hence disappear when Livy server gets the message and becomes:
>
>
>
> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>
> s"s3a://///dt= /tid= / ")
>
>
>
> At the same time a couple of input boxes appears at the bottom of this
> paragraph in Zeppelin UI having the corresponding titles in the original
> dollar curly brackets.
>
>
>
> Is this a bug? How can we overcome the string interpolation issue?
>
>
>
> Thanks,
>
> Jialiang
>
>
>
>
Re: Livy interpreter Scala code string interpolation
Posted by "Tan, Jialiang" <jt...@ea.com>.
But somehow Spark interpreter works without this problem. Why is that the case if the problem is on Zeppelin UI side?
From: Jeff Zhang <zj...@gmail.com>
Reply-To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Date: Friday, November 3, 2017 at 4:48 PM
To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Subject: Re: Livy interpreter Scala code string interpolation
Yes, this is due to the dynamic forms of zeppelin.
Mohit Jaggi <mo...@gmail.com>>于2017年11月4日周六 上午2:28写道：
I run into similar issues with shell scripts that use ${var}. Can we use a different magic notation for Z, one that has a lower chance of colliding with shell scripts and Scala code? This will be hard to do due to the variety of interpreters supported. Perhaps this can be made configurable?
On Fri, Nov 3, 2017 at 10:36 AM, Tan, Jialiang <jt...@ea.com>> wrote:
Hi,
When writing Spark Scala code in zeppelin with Livy interpreter, string interpolation does not seem to work. For example:
val devices = sc.objectFile[(VertexId, DeviceAttr)](
s"s3a://${bucket}/${datasetS3Prefix}/${tableName} /dt=${end_dt}/tid=${tid}/${datasetS3Suffix}")
I think the dollar curly brackets were interpreted as Javascript template literals and hence disappear when Livy server gets the message and becomes:
val devices = sc.objectFile[(VertexId, DeviceAttr)](
s"s3a://///dt= /tid= / ")
At the same time a couple of input boxes appears at the bottom of this paragraph in Zeppelin UI having the corresponding titles in the original dollar curly brackets.
Is this a bug? How can we overcome the string interpolation issue?
Thanks,
Jialiang
Re: Livy interpreter Scala code string interpolation
Posted by Jeff Zhang <zj...@gmail.com>.
Yes, this is due to the dynamic forms of zeppelin.
Mohit Jaggi <mo...@gmail.com>于2017年11月4日周六 上午2:28写道：
> I run into similar issues with shell scripts that use ${var}. Can we use a
> different magic notation for Z, one that has a lower chance of colliding
> with shell scripts and Scala code? This will be hard to do due to the
> variety of interpreters supported. Perhaps this can be made configurable?
>
> On Fri, Nov 3, 2017 at 10:36 AM, Tan, Jialiang <jt...@ea.com> wrote:
>
>> Hi,
>>
>>
>>
>> When writing Spark Scala code in zeppelin with Livy interpreter, string
>> interpolation does not seem to work. For example:
>>
>>
>>
>> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>>
>> s"s3a://${bucket}/${datasetS3Prefix}/${tableName}
>> /dt=${end_dt}/tid=${tid}/${datasetS3Suffix}")
>>
>>
>>
>> I think the dollar curly brackets were interpreted as Javascript template
>> literals and hence disappear when Livy server gets the message and becomes:
>>
>>
>>
>> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>>
>> s"s3a://///dt= /tid= / ")
>>
>>
>>
>> At the same time a couple of input boxes appears at the bottom of this
>> paragraph in Zeppelin UI having the corresponding titles in the original
>> dollar curly brackets.
>>
>>
>>
>> Is this a bug? How can we overcome the string interpolation issue?
>>
>>
>>
>> Thanks,
>>
>> Jialiang
>>
>
>
Re: Livy interpreter Scala code string interpolation
Posted by Mohit Jaggi <mo...@gmail.com>.
I run into similar issues with shell scripts that use ${var}. Can we use a
different magic notation for Z, one that has a lower chance of colliding
with shell scripts and Scala code? This will be hard to do due to the
variety of interpreters supported. Perhaps this can be made configurable?
On Fri, Nov 3, 2017 at 10:36 AM, Tan, Jialiang <jt...@ea.com> wrote:
> Hi,
>
>
>
> When writing Spark Scala code in zeppelin with Livy interpreter, string
> interpolation does not seem to work. For example:
>
>
>
> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>
> s"s3a://${bucket}/${datasetS3Prefix}/${tableName}
> /dt=${end_dt}/tid=${tid}/${datasetS3Suffix}")
>
>
>
> I think the dollar curly brackets were interpreted as Javascript template
> literals and hence disappear when Livy server gets the message and becomes:
>
>
>
> val devices = sc.objectFile[(VertexId, DeviceAttr)](
>
> s"s3a://///dt= /tid= / ")
>
>
>
> At the same time a couple of input boxes appears at the bottom of this
> paragraph in Zeppelin UI having the corresponding titles in the original
> dollar curly brackets.
>
>
>
> Is this a bug? How can we overcome the string interpolation issue?
>
>
>
> Thanks,
>
> Jialiang
>