Using Ignite within within networks without internet egress

[Haithem Turki <tu...@gmail.com>]

Hello,

When deploying Ignite on YARN in a network without outbound internet
access, I run into the following issue on startup:

Exception in thread "main" java.lang.RuntimeException: Failed update ignite.
at
org.apache.ignite.yarn.IgniteProvider.updateIgnite(IgniteProvider.java:243)
at org.apache.ignite.yarn.IgniteProvider.getIgnite(IgniteProvider.java:93)
at
org.apache.ignite.yarn.IgniteYarnClient.getIgnite(IgniteYarnClient.java:194)
at org.apache.ignite.yarn.IgniteYarnClient.main(IgniteYarnClient.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:997)
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:933)
at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:851)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1301)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
at
org.apache.ignite.yarn.IgniteProvider.updateIgnite(IgniteProvider.java:220)
... 9 more

Is there a supported way of disabling the download?

As a separate bug, even with internet access, you run into the following
issue during startup:

Exception in thread "main" java.io.FileNotFoundException: File
ignite-releases/gridgain-community-fabric-1.5.22.zip does not exist

at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534)

at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747)

at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524)

at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:424)

at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:340)

at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1944)

at
org.apache.ignite.yarn.utils.IgniteYarnUtils.copyLocalToHdfs(IgniteYarnUtils.java:86)

at org.apache.ignite.yarn.IgniteProvider.getIgnite(IgniteProvider.java:105)

at
org.apache.ignite.yarn.IgniteYarnClient.getIgnite(IgniteYarnClient.java:194)

at org.apache.ignite.yarn.IgniteYarnClient.main(IgniteYarnClient.java:84)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)


Renaming the downloaded jar from
ignite-releases/gridgain-professional-fabric-1.5.22.zip to
gnite-releases/gridgain-community-fabric-1.5.22.zip seems to solve the
issue.

- Haithem

Re: Using Ignite within within networks without internet egress

[Nikolai Tikhonov <nt...@apache.org>]

Hi, Haithem Turki!

If your yarn cluster running in network without internet, you can use
IGNITE_PATH property. The property allows to use apache ignite build from
hdfs. Also error message isn't clear, I've created ticket [1] for this
issue.

1. https://issues.apache.org/jira/browse/IGNITE-3268

On Tue, May 31, 2016 at 11:58 PM, Haithem Turki <tu...@gmail.com>
wrote:

> Hello,
>
> When deploying Ignite on YARN in a network without outbound internet
> access, I run into the following issue on startup:
>
> Exception in thread "main" java.lang.RuntimeException: Failed update
> ignite.
> at
> org.apache.ignite.yarn.IgniteProvider.updateIgnite(IgniteProvider.java:243)
> at org.apache.ignite.yarn.IgniteProvider.getIgnite(IgniteProvider.java:93)
> at
> org.apache.ignite.yarn.IgniteYarnClient.getIgnite(IgniteYarnClient.java:194)
> at org.apache.ignite.yarn.IgniteYarnClient.main(IgniteYarnClient.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.net.ConnectException: Connection timed out
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at java.net.Socket.connect(Socket.java:528)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
> at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
> at sun.net.www.http.HttpClient.New(HttpClient.java:308)
> at sun.net.www.http.HttpClient.New(HttpClient.java:326)
> at
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:997)
> at
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:933)
> at
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:851)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1301)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
> at
> org.apache.ignite.yarn.IgniteProvider.updateIgnite(IgniteProvider.java:220)
> ... 9 more
>
> Is there a supported way of disabling the download?
>
> As a separate bug, even with internet access, you run into the following
> issue during startup:
>
> Exception in thread "main" java.io.FileNotFoundException: File
> ignite-releases/gridgain-community-fabric-1.5.22.zip does not exist
>
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534)
>
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747)
>
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524)
>
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:424)
>
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:340)
>
> at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1944)
>
> at
> org.apache.ignite.yarn.utils.IgniteYarnUtils.copyLocalToHdfs(IgniteYarnUtils.java:86)
>
> at org.apache.ignite.yarn.IgniteProvider.getIgnite(IgniteProvider.java:105)
>
> at
> org.apache.ignite.yarn.IgniteYarnClient.getIgnite(IgniteYarnClient.java:194)
>
> at org.apache.ignite.yarn.IgniteYarnClient.main(IgniteYarnClient.java:84)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>
>
> Renaming the downloaded jar from
> ignite-releases/gridgain-professional-fabric-1.5.22.zip to
> gnite-releases/gridgain-community-fabric-1.5.22.zip seems to solve the
> issue.
>
> - Haithem
>