You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by Ankit Gupta <in...@gmail.com> on 2022/09/22 04:34:06 UTC
ZEPPELIN-5825 Zeppelin's spark interpreter erroring out when running with Standalone Spark cluster in cluster deploy mode
Hi Dev-Community
Looks like when running the zeppelin notebook with spark standalone cluster
in deploy cluster mode, it is erroring out with the below error in spark
driver program and zeppelin detaches itself with the spark driver.
https://issues.apache.org/jira/browse/ZEPPELIN-5825
I suspect it is because the driver is asking for resources from the master
at 6066 port. Since that is the REST master port which was configured from
zeppelin as the master but since this is cluster mode and the driver would
be running on the cluster itself and would be actually a part of the
cluster itself.
It appears it tries to reconnect with the spark master, which is why it
fails.
Can anyone think of any other possible scenarios in which the same might be
happening?
Thanks and regards
Ankit Prakash Gupta
Re: ZEPPELIN-5825 Zeppelin's spark interpreter erroring out when running with Standalone Spark cluster in cluster deploy mode
Posted by Ankit Gupta <in...@gmail.com>.
Hi Jeff
Apologies to poll again. Did you get a chance to check this out ?
Thanks and Regards
Ankit Prakash Gupta
info.ankitp@gmail.com
On Wed, Sep 28, 2022 at 5:52 PM Ankit Gupta <in...@gmail.com> wrote:
> Hi Jeff
>
> I have put more details on how to reproduce in the ticket. Please review
> and feel free to reach out if you have any questions.
>
> Thanks and Regards.
>
> Ankit Prakash Gupta
> info.ankitp@gmail.com
> +91 8750101321
>
> On Fri, Sep 23, 2022 at 10:41 AM Ankit Gupta <in...@gmail.com>
> wrote:
>
>> Hi Jeff
>>
>> Sure ! Let me containerise the solutions and put those in the jira ticket.
>>
>> Thanks and regards
>>
>> Ankit Prakash Gupta
>> info.ankitp@gmail.com
>> +91 8750101321
>>
>> On Thu, 22 Sep, 2022, 5:15 pm Jeff Zhang, <zj...@gmail.com> wrote:
>>
>>> Hi Ankit,
>>>
>>> Thanks for the reporting, could add how to reproduce step in the jira
>>> ticket?
>>>
>>> On Thu, Sep 22, 2022 at 6:04 PM Ankit Gupta <in...@gmail.com>
>>> wrote:
>>>
>>> > Hi Dev-Community
>>> >
>>> > Looks like when running the zeppelin notebook with spark standalone
>>> cluster
>>> > in deploy cluster mode, it is erroring out with the below error in
>>> spark
>>> > driver program and zeppelin detaches itself with the spark driver.
>>> >
>>> > https://issues.apache.org/jira/browse/ZEPPELIN-5825
>>> >
>>> > I suspect it is because the driver is asking for resources from the
>>> master
>>> > at 6066 port. Since that is the REST master port which was configured
>>> from
>>> > zeppelin as the master but since this is cluster mode and the driver
>>> would
>>> > be running on the cluster itself and would be actually a part of the
>>> > cluster itself.
>>> >
>>> > It appears it tries to reconnect with the spark master, which is why it
>>> > fails.
>>> >
>>> > Can anyone think of any other possible scenarios in which the same
>>> might be
>>> > happening?
>>> >
>>> > Thanks and regards
>>> >
>>> > Ankit Prakash Gupta
>>> >
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
Re: ZEPPELIN-5825 Zeppelin's spark interpreter erroring out when running with Standalone Spark cluster in cluster deploy mode
Posted by Ankit Gupta <in...@gmail.com>.
Hi Jeff
I have put more details on how to reproduce in the ticket. Please review
and feel free to reach out if you have any questions.
Thanks and Regards.
Ankit Prakash Gupta
info.ankitp@gmail.com
+91 8750101321
On Fri, Sep 23, 2022 at 10:41 AM Ankit Gupta <in...@gmail.com> wrote:
> Hi Jeff
>
> Sure ! Let me containerise the solutions and put those in the jira ticket.
>
> Thanks and regards
>
> Ankit Prakash Gupta
> info.ankitp@gmail.com
> +91 8750101321
>
> On Thu, 22 Sep, 2022, 5:15 pm Jeff Zhang, <zj...@gmail.com> wrote:
>
>> Hi Ankit,
>>
>> Thanks for the reporting, could add how to reproduce step in the jira
>> ticket?
>>
>> On Thu, Sep 22, 2022 at 6:04 PM Ankit Gupta <in...@gmail.com>
>> wrote:
>>
>> > Hi Dev-Community
>> >
>> > Looks like when running the zeppelin notebook with spark standalone
>> cluster
>> > in deploy cluster mode, it is erroring out with the below error in spark
>> > driver program and zeppelin detaches itself with the spark driver.
>> >
>> > https://issues.apache.org/jira/browse/ZEPPELIN-5825
>> >
>> > I suspect it is because the driver is asking for resources from the
>> master
>> > at 6066 port. Since that is the REST master port which was configured
>> from
>> > zeppelin as the master but since this is cluster mode and the driver
>> would
>> > be running on the cluster itself and would be actually a part of the
>> > cluster itself.
>> >
>> > It appears it tries to reconnect with the spark master, which is why it
>> > fails.
>> >
>> > Can anyone think of any other possible scenarios in which the same
>> might be
>> > happening?
>> >
>> > Thanks and regards
>> >
>> > Ankit Prakash Gupta
>> >
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
Re: ZEPPELIN-5825 Zeppelin's spark interpreter erroring out when running with Standalone Spark cluster in cluster deploy mode
Posted by Ankit Gupta <in...@gmail.com>.
Hi Jeff
Sure ! Let me containerise the solutions and put those in the jira ticket.
Thanks and regards
Ankit Prakash Gupta
info.ankitp@gmail.com
+91 8750101321
On Thu, 22 Sep, 2022, 5:15 pm Jeff Zhang, <zj...@gmail.com> wrote:
> Hi Ankit,
>
> Thanks for the reporting, could add how to reproduce step in the jira
> ticket?
>
> On Thu, Sep 22, 2022 at 6:04 PM Ankit Gupta <in...@gmail.com> wrote:
>
> > Hi Dev-Community
> >
> > Looks like when running the zeppelin notebook with spark standalone
> cluster
> > in deploy cluster mode, it is erroring out with the below error in spark
> > driver program and zeppelin detaches itself with the spark driver.
> >
> > https://issues.apache.org/jira/browse/ZEPPELIN-5825
> >
> > I suspect it is because the driver is asking for resources from the
> master
> > at 6066 port. Since that is the REST master port which was configured
> from
> > zeppelin as the master but since this is cluster mode and the driver
> would
> > be running on the cluster itself and would be actually a part of the
> > cluster itself.
> >
> > It appears it tries to reconnect with the spark master, which is why it
> > fails.
> >
> > Can anyone think of any other possible scenarios in which the same might
> be
> > happening?
> >
> > Thanks and regards
> >
> > Ankit Prakash Gupta
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: ZEPPELIN-5825 Zeppelin's spark interpreter erroring out when running with Standalone Spark cluster in cluster deploy mode
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Ankit,
Thanks for the reporting, could add how to reproduce step in the jira
ticket?
On Thu, Sep 22, 2022 at 6:04 PM Ankit Gupta <in...@gmail.com> wrote:
> Hi Dev-Community
>
> Looks like when running the zeppelin notebook with spark standalone cluster
> in deploy cluster mode, it is erroring out with the below error in spark
> driver program and zeppelin detaches itself with the spark driver.
>
> https://issues.apache.org/jira/browse/ZEPPELIN-5825
>
> I suspect it is because the driver is asking for resources from the master
> at 6066 port. Since that is the REST master port which was configured from
> zeppelin as the master but since this is cluster mode and the driver would
> be running on the cluster itself and would be actually a part of the
> cluster itself.
>
> It appears it tries to reconnect with the spark master, which is why it
> fails.
>
> Can anyone think of any other possible scenarios in which the same might be
> happening?
>
> Thanks and regards
>
> Ankit Prakash Gupta
>
--
Best Regards
Jeff Zhang