[jira] [Created] (ZEPPELIN-2076) Problem resolving interpreter dependencies behind firewall

["Dries De Maeyer (JIRA)" <ji...@apache.org>]

Dries De Maeyer created ZEPPELIN-2076:
-----------------------------------------

             Summary: Problem resolving interpreter dependencies behind firewall
                 Key: ZEPPELIN-2076
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2076
             Project: Zeppelin
          Issue Type: Bug
          Components: Interpreters
    Affects Versions: 0.7.0
         Environment: RHEL 6.0
            Reporter: Dries De Maeyer


My zeppelin is running behind a corporate firewall. 

When I create a spark interpreter which has external dependencies, the interpreter cannot be started due to the fact that it cannot resolve the dependencies. It will look for the dependencies in external repositories to check for newer versions. After n-times retrying the resolution of the newer dependencies (which will always timeout (see exception)) it will just copy the dependencies to the local-repo folder for the specific interpreter when they are in the local repository. This means however that restarting the zeppelin takes more than half an hour because he will retry resolving the dependencies. 


Current workaround is to download the external dependencies using maven 2 so that they can be copied into the local repo folder. Restarting the interpreter and manually copying the dependencies overrides the automatic resolution.

I tried
 - adding additional external repositories with proxy settings
 - adding the proxy settings to the zeppelin startup process (but these are probably not used by the interpreter itself)
 - a lot of waiting (which helps but is very unfortunate if you have to restart)

Beneath the exception logging as in the .log files:
DEBUG [2017-02-07 17:58:52,923] ({pool-50-thread-1} HttpConnection.java[open]:692) - Open connection to oss.sonatype.org:443
DEBUG [2017-02-07 17:59:55,932] ({pool-50-thread-1} HttpMethodDirector.java[executeWithRetry]:404) - Closing the connection.
 INFO [2017-02-07 17:59:55,932] ({pool-50-thread-1} HttpMethodDirector.java[executeWithRetry]:439) - I/O exception (java.net.ConnectException) caught when processing request: Connection timed out (Connection timed out)
DEBUG [2017-02-07 17:59:55,932] ({pool-50-thread-1} HttpMethodDirector.java[executeWithRetry]:443) - Connection timed out (Connection timed out)
java.net.ConnectException: Connection timed out (Connection timed out)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
        at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:472)
        at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:153)
        at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:82)
        at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:127)
        at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
        at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
        at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
        at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
        at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.execute(AbstractHttpClientWagon.java:446)
        at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:561)
        at org.apache.maven.wagon.StreamWagon.getInputStream(StreamWagon.java:116)
        at org.apache.maven.wagon.StreamWagon.getIfNewer(StreamWagon.java:88)
        at org.apache.maven.wagon.StreamWagon.get(StreamWagon.java:61)
        at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.run(WagonRepositoryConnector.java:615)
        at org.sonatype.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:60)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)