Zeppelin jobs get stuck in Pending state after every few days

[Aaryan Reddy <aa...@gmail.com>]

Hello All,

We are using Hortonworks distribution of Hadoop stack HDP 2.6.5, which
comes with Zeppelin 0.7.3 . Every few days zeppelin jobs get into pending
state .

* All the users at that point experience the issue
* looking on the zeppelin server node , it seems no new processes (pids)
are getting spun up by zeppelin user .
* ulimits for zeppelin user seems to not restrict it .

Restarting zeppelin server gives us few more days , but since this is
happening randomly and every few days it gets annoying . Any suggestion as
to how to handle this .

Zeppelin configuration :
heap  | min : 4g max: 8g
# of users : around 15-20 active users

Thank You,
Aaryan

Re: Zeppelin jobs get stuck in Pending state after every few days

[moon <mo...@zepl.com>]

Thanks for sharing the problem.
If you can share stacktrace (using jstack) on ZeppelinServer process when
you experience the problem, that'll be really helpful.

Thanks,
moon

On Fri, Feb 15, 2019 at 9:14 AM Aaryan Reddy <aa...@gmail.com>
wrote:

> Hello All,
>
> We are using Hortonworks distribution of Hadoop stack HDP 2.6.5, which
> comes with Zeppelin 0.7.3 . Every few days zeppelin jobs get into pending
> state .
>
> * All the users at that point experience the issue
> * looking on the zeppelin server node , it seems no new processes (pids)
> are getting spun up by zeppelin user .
> * ulimits for zeppelin user seems to not restrict it .
>
> Restarting zeppelin server gives us few more days , but since this is
> happening randomly and every few days it gets annoying . Any suggestion as
> to how to handle this .
>
> Zeppelin configuration :
> heap  | min : 4g max: 8g
> # of users : around 15-20 active users
>
> Thank You,
> Aaryan
>