Running spark interpreter in 0.10.0 docker image fails to delete files

[br...@brutex.de]

Hi,

I have set up Zeppelin to use docker to run interpreters using
<property>
  <name>zeppelin.run.mode</name>
  <value>docker</value>
  <description>'auto|local|k8s|docker'</description>
</property>

<property>
  <name>zeppelin.docker.container.image</name>
  <value> apache/zeppelin:0.10.0 </value>
  <description>Docker image for interpreters</description>
</property>

This works so far, as interpreter images are being launched and used,
however I run into an issue with the Spark Interpreter.
In the 0.10.0 docker image, the DockerInterpreter tries to delete all
interpreters, that are not relevant for Spark. There I do get a permission
denied message. The Dockerfile uses USER 1000 and the Zeppelin files in the
image are owned by root, thus a delete fails.

This happens in
org/apache/zeppelin/interpreter/launcher/DockerInterpreterProcess.java
private void rmInContainer(String containerId, String path)

Please advice.
Thanks.

Cheers
Brian


Log snippet:
[...]
rm: cannot remove
'/opt/zeppelin/interpreter/angular/zeppelin-angular-0.10.0.jar'
 INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:591) - : Permission denied

 INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove
'/opt/zeppelin/interpreter/angular/META-INF/LICENSE': Permission denied
rm: cannot remove '/opt/zeppelin/interpreter/angular/META-INF/DEPENDENCIES':
Permission denied

 INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove
'/opt/zeppelin/interpreter/angular/META-INF/NOTICE': Permission denied

 INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove
'/opt/zeppelin/interpreter/flink-cmd/interpreter-setting.json': Permissi
on denied
rm: cannot remove
'/opt/zeppelin/interpreter/flink-cmd/zeppelin-flink-cmd-0.10.0.jar'
 INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:591) - : Permission denied

 INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove
'/opt/zeppelin/interpreter/flink-cmd/META-INF/LICENSE': Permission denied
rm: cannot remove
'/opt/zeppelin/interpreter/flink-cmd/META-INF/DEPENDENCIES': Permission
denied
INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove
'/opt/zeppelin/interpreter/flink-cmd/META-INF/NOTICE': Permission denied

 INFO [2021-11-02 09:29:29,031] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:581) - exec container
commmand: mkdir /opt/zeppelin -p
 INFO [2021-11-02 09:29:29,082] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:581) - exec container
commmand: mkdir /opt/zeppelin/conf -p
 WARN [2021-11-02 09:29:29,114] ({SchedulerFactory3}
DockerInterpreterProcess.java[copyRunFileToContainer]:452) - /etc/krb5.conf
file not found, Did not upload the krb5.conf to the container!
 INFO [2021-11-02 09:29:29,114] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:581) - exec container
commmand: mkdir /opt/zeppelin/bin -p
 INFO [2021-11-02 09:29:29,177] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:581) - exec container
commmand: mkdir /opt/zeppelin/interpreter/spark -p
 INFO [2021-11-02 09:29:40,185] ({SchedulerFactory3}
DockerInterpreterProcess.java[execInContainer]:581) - exec container
commmand: mkdir /tmp/zeppelin-tar -p
 WARN [2021-11-02 09:29:40,404] ({SchedulerFactory3}
NotebookServer.java[onStatusChange]:1986) - Job 20180530-101750_1491737301
is finished, status: ERROR, exception: null, result: %text
java.lang.IllegalStateException: Container
8d64e1b1c58327771aa08678f555f95b3d0bbca37cb7b7662220594a4f39f2da is not
running.
        at
com.spotify.docker.client.DefaultDockerClient.execCreate(DefaultDockerClient
.java:1650)
        at
org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.execInCont
ainer(DockerInterpreterProcess.java:584)
        at
org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.mkdirInCon
tainer(DockerInterpreterProcess.java:563)
        at
org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.deployToCo
ntainer(DockerInterpreterProcess.java:539)
        at
org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.copyRunFil
eToContainer(DockerInterpreterProcess.java:533)
        at
org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.start(Dock
erInterpreterProcess.java:237)
        at
org.apache.zeppelin.interpreter.ManagedInterpreterGroup.getOrCreateInterpret
erProcess(ManagedInterpreterGroup.java:68)
        at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpre
terProcess(RemoteInterpreter.java:104)
        at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(Rem
oteInterpreter.java:154)
        at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpre
ter.java:126)
        at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteI
nterpreter.java:271)
        at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:440)
        at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:71)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
        at
org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.jav
a:132)
        at
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.
java:182)
        at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java
:515)
        at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTa
sk.run(ScheduledThreadPoolExecutor.java:304)
        at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
or.java:1128)
        at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
tor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)