intercepter does not work after pod restart

[James Yu <cy...@gmail.com>]

I deploy zeppelin in Kubernetes environment and found zeppelin does not
behave as expected in the following cases:
(1) when I delete the interceptor pod with "kubectl delete", the pod does
not restart itself
(2) when I delete the interceptor container with "docker rm -f", the pod
restarts itself, but zeppelin reports "Internal error processing interpret"
error

I tried these cases to see if zeppelin is able to heal itself in production
environment.
For case (1), the pod will definitely restart when deployed as deployment
or at least managed by replication set controller.
For case (2),  since the restarted pod has the same, I have not idea why
zeppelin reports error. Perhaps zeppelin can access interpreter through
service instead of using pod straightaway?


This is a UTF-8 formatted mail
-----------------------------------------------
James C.-C.Yu
+886988713275
+8615618429976

Re: intercepter does not work after pod restart

[moon soo Lee <mo...@apache.org>]

Regarding (1),

If an Interpreter is stateless (for example jdbc interpreter), they can be
silently restarted by Deployment.

However, if an Interpreter is stateful (like spark, python) and restarted
silently, user may experience unexpected behavior or result. It can be
quite confusing. So when an Interpreter is restarted itself by Deployment,
it should let the user know that it is restarted and all states may be
lost. But currently, the user does not have visibility through Zeppelin UI
to know whether the interpreter is restarted or not.

One reason why the interpreter is running as a Pod at the moment is, when
it is removed, Zeppelin UI will show some error to the user, so the user
can know the state is lost.

It would be great if it is restarted automatically and UI lets users know
about it.


Regarding (2),

When the interpreter is started and the connection is made, there are some
initialization processes  (invoke open()) initiated by Zeppelin Server.
When the Interpreter container is restarted and Zeppelin Server does not
aware of it, so the initialization process is not started on the new
container, there could be some error.  Not 100% sure if this is the case.

Thanks,
moon

On Sun, May 31, 2020 at 8:37 AM James Yu <cy...@gmail.com> wrote:

> I deploy zeppelin in Kubernetes environment and found zeppelin does not
> behave as expected in the following cases:
> (1) when I delete the interceptor pod with "kubectl delete", the pod does
> not restart itself
> (2) when I delete the interceptor container with "docker rm -f", the pod
> restarts itself, but zeppelin reports "Internal error processing interpret"
> error
>
> I tried these cases to see if zeppelin is able to heal itself in
> production environment.
> For case (1), the pod will definitely restart when deployed as deployment
> or at least managed by replication set controller.
> For case (2),  since the restarted pod has the same, I have not idea why
> zeppelin reports error. Perhaps zeppelin can access interpreter through
> service instead of using pod straightaway?
>
>
> This is a UTF-8 formatted mail
> -----------------------------------------------
> James C.-C.Yu
> +886988713275
> +8615618429976
>