You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by st...@orange.com on 2020/02/18 08:32:00 UTC

Zeppelin performances with a lot of notebooks

Hello,

 

I’m facing performances issues with zeppelin. It seems that these
performances are mainly located on the browser side. I have actually a
Zeppelin instance with quite a lot of notebooks and many paragraphs inside,
and I find that Zeppelin refresh and rendering is really slow. It means for
example that the spark job is finished, but the Web UI shows that it is 20%
completed, and it takes a few minutes to reflect the real spark status.

So, I extracted one notebook, and imported it in another Zeppelin on the
same server. Actually, I have this:

-          A Zeppelin 0.8.0 which comes with Hortonworks data platform

-          A standalone Zeppelin 0.8.2 on the same server, with the same
configuration for memory

 

And then, my notebook imported in the standalone instance runs really
fastly. One difference I also have between the 2 instances is that the
standalone one has its notebook on local filesystem, while the HDP one has
its notebook on hdfs. For sure, hdfs access is slower than the local fs one.
Can this also be a possible origin of this issue?

 

Thanks for your help,

 

Stéphane

Re: Zeppelin performances with a lot of notebooks(Internet mail)

Posted by Jeff Zhang <zj...@gmail.com>.
0.9 will load note lazily, only load it when this note is opened.

lewinma(lewinma) <le...@tencent.com> 于2020年2月18日周二 下午4:54写道：

> Does 0.9 has some change?
>
>
>
> *From: *Jeff Zhang <zj...@gmail.com>
> *Reply-To: *"users@zeppelin.apache.org" <us...@zeppelin.apache.org>
> *Date: *Tuesday, February 18, 2020 at 16:53
> *To: *users <us...@zeppelin.apache.org>
> *Subject: *Re: Zeppelin performances with a lot of notebooks(Internet
> mail)
>
>
>
> Then I guess it is due to backend issue. Could you check the memory usage
> of the JVM process ? In 0.8, all the notes will be loaded in to memory,
> which might be the reason of slow performance.
>
>
>
> <st...@orange.com> 于2020年2月18日周二 下午4:38写道：
>
> Yes, it is slow in another tab, its slow even if I close and re-open by
> browser.
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjffdu@gmail.com]
> *Sent:* Tuesday, February 18, 2020 09:36
> *To:* users
> *Subject:* Re: Zeppelin performances with a lot of notebooks
>
>
>
> It's hard to say. Have you tried to open another tab ? If it is still
> slow, then it might be issue in backend.
>
>
>
> <st...@orange.com> 于2020年2月18日周二 下午4:32写道：
>
> Hello,
>
>
>
> I’m facing performances issues with zeppelin. It seems that these
> performances are mainly located on the browser side. I have actually a
> Zeppelin instance with quite a lot of notebooks and many paragraphs inside,
> and I find that Zeppelin refresh and rendering is really slow. It means for
> example that the spark job is finished, but the Web UI shows that it is 20%
> completed, and it takes a few minutes to reflect the real spark status.
>
> So, I extracted one notebook, and imported it in another Zeppelin on the
> same server. Actually, I have this:
>
> -          A Zeppelin 0.8.0 which comes with Hortonworks data platform
>
> -          A standalone Zeppelin 0.8.2 on the same server, with the same
> configuration for memory
>
>
>
> And then, my notebook imported in the standalone instance runs really
> fastly. One difference I also have between the 2 instances is that the
> standalone one has its notebook on local filesystem, while the HDP one has
> its notebook on hdfs. For sure, hdfs access is slower than the local fs
> one. Can this also be a possible origin of this issue?
>
>
>
> Thanks for your help,
>
>
>
> Stéphane
>
>
>
>
> --
>
> Best Regards
>
> Jeff Zhang
>
>
>
>
> --
>
> Best Regards
>
> Jeff Zhang
>


-- 
Best Regards

Jeff Zhang

Re: Zeppelin performances with a lot of notebooks(Internet mail)

Posted by "lewinma(lewinma)" <le...@tencent.com>.
Does 0.9 has some change?

From: Jeff Zhang <zj...@gmail.com>
Reply-To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Date: Tuesday, February 18, 2020 at 16:53
To: users <us...@zeppelin.apache.org>
Subject: Re: Zeppelin performances with a lot of notebooks(Internet mail)

Then I guess it is due to backend issue. Could you check the memory usage of the JVM process ? In 0.8, all the notes will be loaded in to memory, which might be the reason of slow performance.

<st...@orange.com>> 于2020年2月18日周二 下午4:38写道：
Yes, it is slow in another tab, its slow even if I close and re-open by browser.


From: Jeff Zhang [mailto:zjffdu@gmail.com<ma...@gmail.com>]
Sent: Tuesday, February 18, 2020 09:36
To: users
Subject: Re: Zeppelin performances with a lot of notebooks

It's hard to say. Have you tried to open another tab ? If it is still slow, then it might be issue in backend.

<st...@orange.com>> 于2020年2月18日周二 下午4:32写道：
Hello,

I’m facing performances issues with zeppelin. It seems that these performances are mainly located on the browser side. I have actually a Zeppelin instance with quite a lot of notebooks and many paragraphs inside, and I find that Zeppelin refresh and rendering is really slow. It means for example that the spark job is finished, but the Web UI shows that it is 20% completed, and it takes a few minutes to reflect the real spark status.
So, I extracted one notebook, and imported it in another Zeppelin on the same server. Actually, I have this:

-          A Zeppelin 0.8.0 which comes with Hortonworks data platform

-          A standalone Zeppelin 0.8.2 on the same server, with the same configuration for memory

And then, my notebook imported in the standalone instance runs really fastly. One difference I also have between the 2 instances is that the standalone one has its notebook on local filesystem, while the HDP one has its notebook on hdfs. For sure, hdfs access is slower than the local fs one. Can this also be a possible origin of this issue?

Thanks for your help,

Stéphane


--
Best Regards

Jeff Zhang


--
Best Regards

Jeff Zhang

RE: Zeppelin performances with a lot of notebooks

Posted by st...@orange.com.
What are your recommendations for the memory settings? Currently I have 2G for the Zeppelin JVM itself. I don’t have that much RAM on this VM to dedicate more for this process L

 

 

 

From: Jeff Zhang [mailto:zjffdu@gmail.com] 
Sent: Tuesday, February 18, 2020 09:53
To: users
Subject: Re: Zeppelin performances with a lot of notebooks

 

Then I guess it is due to backend issue. Could you check the memory usage of the JVM process ? In 0.8, all the notes will be loaded in to memory, which might be the reason of slow performance. 

 

<st...@orange.com> 于2020年2月18日周二 下午4:38写道：

Yes, it is slow in another tab, its slow even if I close and re-open by browser.

 

 

From: Jeff Zhang [mailto:zjffdu@gmail.com] 
Sent: Tuesday, February 18, 2020 09:36
To: users
Subject: Re: Zeppelin performances with a lot of notebooks

 

It's hard to say. Have you tried to open another tab ? If it is still slow, then it might be issue in backend. 

 

<st...@orange.com> 于2020年2月18日周二 下午4:32写道：

Hello,

 

I’m facing performances issues with zeppelin. It seems that these performances are mainly located on the browser side. I have actually a Zeppelin instance with quite a lot of notebooks and many paragraphs inside, and I find that Zeppelin refresh and rendering is really slow. It means for example that the spark job is finished, but the Web UI shows that it is 20% completed, and it takes a few minutes to reflect the real spark status.

So, I extracted one notebook, and imported it in another Zeppelin on the same server. Actually, I have this:

-          A Zeppelin 0.8.0 which comes with Hortonworks data platform

-          A standalone Zeppelin 0.8.2 on the same server, with the same configuration for memory

 

And then, my notebook imported in the standalone instance runs really fastly. One difference I also have between the 2 instances is that the standalone one has its notebook on local filesystem, while the HDP one has its notebook on hdfs. For sure, hdfs access is slower than the local fs one. Can this also be a possible origin of this issue?

 

Thanks for your help,

 

Stéphane




 

-- 

Best Regards

Jeff Zhang




 

-- 

Best Regards

Jeff Zhang

RE: Zeppelin performances with a lot of notebooks

Posted by st...@orange.com.
What is the best way to check this JVM memory usage?

 

From: Jeff Zhang [mailto:zjffdu@gmail.com] 
Sent: Tuesday, February 18, 2020 09:53
To: users
Subject: Re: Zeppelin performances with a lot of notebooks

 

Then I guess it is due to backend issue. Could you check the memory usage of the JVM process ? In 0.8, all the notes will be loaded in to memory, which might be the reason of slow performance. 

 

<st...@orange.com> 于2020年2月18日周二 下午4:38写道：

Yes, it is slow in another tab, its slow even if I close and re-open by browser.

 

 

From: Jeff Zhang [mailto:zjffdu@gmail.com] 
Sent: Tuesday, February 18, 2020 09:36
To: users
Subject: Re: Zeppelin performances with a lot of notebooks

 

It's hard to say. Have you tried to open another tab ? If it is still slow, then it might be issue in backend. 

 

<st...@orange.com> 于2020年2月18日周二 下午4:32写道：

Hello,

 

I’m facing performances issues with zeppelin. It seems that these performances are mainly located on the browser side. I have actually a Zeppelin instance with quite a lot of notebooks and many paragraphs inside, and I find that Zeppelin refresh and rendering is really slow. It means for example that the spark job is finished, but the Web UI shows that it is 20% completed, and it takes a few minutes to reflect the real spark status.

So, I extracted one notebook, and imported it in another Zeppelin on the same server. Actually, I have this:

-          A Zeppelin 0.8.0 which comes with Hortonworks data platform

-          A standalone Zeppelin 0.8.2 on the same server, with the same configuration for memory

 

And then, my notebook imported in the standalone instance runs really fastly. One difference I also have between the 2 instances is that the standalone one has its notebook on local filesystem, while the HDP one has its notebook on hdfs. For sure, hdfs access is slower than the local fs one. Can this also be a possible origin of this issue?

 

Thanks for your help,

 

Stéphane




 

-- 

Best Regards

Jeff Zhang




 

-- 

Best Regards

Jeff Zhang

Re: Zeppelin performances with a lot of notebooks

Posted by Jeff Zhang <zj...@gmail.com>.
Then I guess it is due to backend issue. Could you check the memory usage
of the JVM process ? In 0.8, all the notes will be loaded in to memory,
which might be the reason of slow performance.

<st...@orange.com> 于2020年2月18日周二 下午4:38写道：

> Yes, it is slow in another tab, its slow even if I close and re-open by
> browser.
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjffdu@gmail.com]
> *Sent:* Tuesday, February 18, 2020 09:36
> *To:* users
> *Subject:* Re: Zeppelin performances with a lot of notebooks
>
>
>
> It's hard to say. Have you tried to open another tab ? If it is still
> slow, then it might be issue in backend.
>
>
>
> <st...@orange.com> 于2020年2月18日周二 下午4:32写道：
>
> Hello,
>
>
>
> I’m facing performances issues with zeppelin. It seems that these
> performances are mainly located on the browser side. I have actually a
> Zeppelin instance with quite a lot of notebooks and many paragraphs inside,
> and I find that Zeppelin refresh and rendering is really slow. It means for
> example that the spark job is finished, but the Web UI shows that it is 20%
> completed, and it takes a few minutes to reflect the real spark status.
>
> So, I extracted one notebook, and imported it in another Zeppelin on the
> same server. Actually, I have this:
>
> -          A Zeppelin 0.8.0 which comes with Hortonworks data platform
>
> -          A standalone Zeppelin 0.8.2 on the same server, with the same
> configuration for memory
>
>
>
> And then, my notebook imported in the standalone instance runs really
> fastly. One difference I also have between the 2 instances is that the
> standalone one has its notebook on local filesystem, while the HDP one has
> its notebook on hdfs. For sure, hdfs access is slower than the local fs
> one. Can this also be a possible origin of this issue?
>
>
>
> Thanks for your help,
>
>
>
> Stéphane
>
>
>
>
> --
>
> Best Regards
>
> Jeff Zhang
>


-- 
Best Regards

Jeff Zhang

RE: Zeppelin performances with a lot of notebooks

Posted by st...@orange.com.
Yes, it is slow in another tab, its slow even if I close and re-open by browser.

 

 

From: Jeff Zhang [mailto:zjffdu@gmail.com] 
Sent: Tuesday, February 18, 2020 09:36
To: users
Subject: Re: Zeppelin performances with a lot of notebooks

 

It's hard to say. Have you tried to open another tab ? If it is still slow, then it might be issue in backend. 

 

<st...@orange.com> 于2020年2月18日周二 下午4:32写道：

Hello,

 

I’m facing performances issues with zeppelin. It seems that these performances are mainly located on the browser side. I have actually a Zeppelin instance with quite a lot of notebooks and many paragraphs inside, and I find that Zeppelin refresh and rendering is really slow. It means for example that the spark job is finished, but the Web UI shows that it is 20% completed, and it takes a few minutes to reflect the real spark status.

So, I extracted one notebook, and imported it in another Zeppelin on the same server. Actually, I have this:

-          A Zeppelin 0.8.0 which comes with Hortonworks data platform

-          A standalone Zeppelin 0.8.2 on the same server, with the same configuration for memory

 

And then, my notebook imported in the standalone instance runs really fastly. One difference I also have between the 2 instances is that the standalone one has its notebook on local filesystem, while the HDP one has its notebook on hdfs. For sure, hdfs access is slower than the local fs one. Can this also be a possible origin of this issue?

 

Thanks for your help,

 

Stéphane




 

-- 

Best Regards

Jeff Zhang

Re: Zeppelin performances with a lot of notebooks

Posted by Jeff Zhang <zj...@gmail.com>.
It's hard to say. Have you tried to open another tab ? If it is still slow,
then it might be issue in backend.

<st...@orange.com> 于2020年2月18日周二 下午4:32写道：

> Hello,
>
>
>
> I’m facing performances issues with zeppelin. It seems that these
> performances are mainly located on the browser side. I have actually a
> Zeppelin instance with quite a lot of notebooks and many paragraphs inside,
> and I find that Zeppelin refresh and rendering is really slow. It means for
> example that the spark job is finished, but the Web UI shows that it is 20%
> completed, and it takes a few minutes to reflect the real spark status.
>
> So, I extracted one notebook, and imported it in another Zeppelin on the
> same server. Actually, I have this:
>
> -          A Zeppelin 0.8.0 which comes with Hortonworks data platform
>
> -          A standalone Zeppelin 0.8.2 on the same server, with the same
> configuration for memory
>
>
>
> And then, my notebook imported in the standalone instance runs really
> fastly. One difference I also have between the 2 instances is that the
> standalone one has its notebook on local filesystem, while the HDP one has
> its notebook on hdfs. For sure, hdfs access is slower than the local fs
> one. Can this also be a possible origin of this issue?
>
>
>
> Thanks for your help,
>
>
>
> Stéphane
>


-- 
Best Regards

Jeff Zhang