You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Gokulnath Chidambaram <c....@gmail.com> on 2019/11/07 11:27:20 UTC
Kubernetes- Failed to retrieve Ignite pods IP addresses.
Hello,
I am trying to install apache ignite:2.7.6 in kubernetes cluster (deployed
in aws).
I created
1.service account
2. role access
3.rolebinding
4.deployment.
4. added namesapce (bean property) in xml configuration file.
I am getting the following error message.
11:18:39,724][INFO][main][PartitionsEvictManager] Evict partition permits=2
[11:18:44,719][INFO][main][ClientListenerProcessor] Client connector
processor has started on TCP port 10800
[11:18:46,418][INFO][main][GridTcpRestProtocol] Command protocol
successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211]
[11:18:48,617][WARNING][jvm-pause-detector-worker][IgniteKernal] Possible
too long JVM pause: 850 milliseconds.
[11:18:52,420][INFO][main][GridJettyRestProtocol] Command protocol
successfully started [name=Jetty REST, host=/0.0.0.0, port=8080]
[11:18:53,721][INFO][main][IgniteKernal] Non-loopback local IPs:
10.42.5.132
[11:18:53,721][INFO][main][IgniteKernal] Enabled local MACs: 363E412C208F
[11:18:54,625][INFO][main][TcpDiscoverySpi] Connection check threshold is
calculated: 10000
[11:18:55,024][INFO][main][TcpDiscoverySpi] Successfully bound to TCP port
[port=47500, localHost=0.0.0.0/0.0.0.0,
locNodeId=6bc33d41-6dba-422b-832d-ed8cae326a00]
[11:19:03,423][SEVERE][main][TcpDiscoverySpi] Failed to get registered
addresses from IP finder on start (retrying every 2000ms; change
'reconnectDelay' to configure the frequency of retries).
class org.apache.ignite.spi.IgniteSpiException: Failed to retrieve Ignite
pods IP addresses.
at
org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:172)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1848)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1049)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:910)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:391)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2020)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)
at org.apache.ignite.Ignition.start(Ignition.java:348)
at
org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)
Caused by: java.io.IOException: Server returned HTTP response code: 403 for
URL:
https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/dataobjns/endpoints/ignite
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
at
org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:153)
... 20 more
[11:19:15,821][WARNING][jvm-pause-detector-worker][IgniteKernal] Possible
too long JVM pause: 500 milliseconds.
any help is appreciated.
Re: Kubernetes- Failed to retrieve Ignite pods IP addresses.
Posted by Gokulnath Chidambaram <c....@gmail.com>.
Thanks Andrei. the solution described in
https://apacheignite.readme.io/docs/rbac-authorization works with
minorchange in Role access. If we have restricted access, Role and
Rolebinding works.
Thanks again for your help,
-gokul
On Thu, Nov 7, 2019 at 5:09 PM Andrei Aleksandrov <ae...@gmail.com>
wrote:
> Hi,
>
> Using the search I found that current issue was resolved in next thread:
>
>
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-on-RBAC-enabled-K8s-cluster-td22165.html
> I guess that you should also read next article:
>
> https://apacheignite.readme.io/docs/rbac-authorization
>
> BR,
> Andrei
>
> 11/7/2019 2:27 PM, Gokulnath Chidambaram пишет:
>
> Hello,
>
> I am trying to install apache ignite:2.7.6 in kubernetes cluster (deployed
> in aws).
> I created
>
> 1.service account
> 2. role access
> 3.rolebinding
> 4.deployment.
> 4. added namesapce (bean property) in xml configuration file.
>
> I am getting the following error message.
>
> 11:18:39,724][INFO][main][PartitionsEvictManager] Evict partition
> permits=2
> [11:18:44,719][INFO][main][ClientListenerProcessor] Client connector
> processor has started on TCP port 10800
> [11:18:46,418][INFO][main][GridTcpRestProtocol] Command protocol
> successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211]
> [11:18:48,617][WARNING][jvm-pause-detector-worker][IgniteKernal] Possible
> too long JVM pause: 850 milliseconds.
> [11:18:52,420][INFO][main][GridJettyRestProtocol] Command protocol
> successfully started [name=Jetty REST, host=/0.0.0.0, port=8080]
> [11:18:53,721][INFO][main][IgniteKernal] Non-loopback local IPs:
> 10.42.5.132
> [11:18:53,721][INFO][main][IgniteKernal] Enabled local MACs: 363E412C208F
> [11:18:54,625][INFO][main][TcpDiscoverySpi] Connection check threshold is
> calculated: 10000
> [11:18:55,024][INFO][main][TcpDiscoverySpi] Successfully bound to TCP port
> [port=47500, localHost=0.0.0.0/0.0.0.0,
> locNodeId=6bc33d41-6dba-422b-832d-ed8cae326a00]
> [11:19:03,423][SEVERE][main][TcpDiscoverySpi] Failed to get registered
> addresses from IP finder on start (retrying every 2000ms; change
> 'reconnectDelay' to configure the frequency of retries).
> class org.apache.ignite.spi.IgniteSpiException: Failed to retrieve Ignite
> pods IP addresses.
> at
> org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:172)
>
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
>
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1848)
>
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1049)
>
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:910)
>
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:391)
>
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2020)
>
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
>
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
>
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
>
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
>
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
>
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
> at
> org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)
>
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)
> at org.apache.ignite.Ignition.start(Ignition.java:348)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)
>
> Caused by: java.io.IOException: Server returned HTTP response code: 403
> for URL:
> https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/dataobjns/endpoints/ignite
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
>
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
>
> at
> sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
>
> at
> org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:153)
>
> ... 20 more
> [11:19:15,821][WARNING][jvm-pause-detector-worker][IgniteKernal] Possible
> too long JVM pause: 500 milliseconds.
>
> any help is appreciated.
>
>
Re: Kubernetes- Failed to retrieve Ignite pods IP addresses.
Posted by Andrei Aleksandrov <ae...@gmail.com>.
Hi,
Using the search I found that current issue was resolved in next thread:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-on-RBAC-enabled-K8s-cluster-td22165.html
I guess that you should also read next article:
https://apacheignite.readme.io/docs/rbac-authorization
BR,
Andrei
11/7/2019 2:27 PM, Gokulnath Chidambaram пишет:
> Hello,
>
> I am trying to install apache ignite:2.7.6 in kubernetes cluster
> (deployed in aws).
> I created
>
> 1.service account
> 2. role access
> 3.rolebinding
> 4.deployment.
> 4. added namesapce (bean property) in xml configuration file.
>
> I am getting the following error message.
>
> 11:18:39,724][INFO][main][PartitionsEvictManager] Evict partition
> permits=2
> [11:18:44,719][INFO][main][ClientListenerProcessor] Client connector
> processor has started on TCP port 10800
> [11:18:46,418][INFO][main][GridTcpRestProtocol] Command protocol
> successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0
> <http://0.0.0.0/0.0.0.0>, port=11211]
> [11:18:48,617][WARNING][jvm-pause-detector-worker][IgniteKernal]
> Possible too long JVM pause: 850 milliseconds.
> [11:18:52,420][INFO][main][GridJettyRestProtocol] Command protocol
> successfully started [name=Jetty REST, host=/0.0.0.0 <http://0.0.0.0>,
> port=8080]
> [11:18:53,721][INFO][main][IgniteKernal] Non-loopback local IPs:
> 10.42.5.132
> [11:18:53,721][INFO][main][IgniteKernal] Enabled local MACs: 363E412C208F
> [11:18:54,625][INFO][main][TcpDiscoverySpi] Connection check threshold
> is calculated: 10000
> [11:18:55,024][INFO][main][TcpDiscoverySpi] Successfully bound to TCP
> port [port=47500, localHost=0.0.0.0/0.0.0.0 <http://0.0.0.0/0.0.0.0>,
> locNodeId=6bc33d41-6dba-422b-832d-ed8cae326a00]
> [11:19:03,423][SEVERE][main][TcpDiscoverySpi] Failed to get registered
> addresses from IP finder on start (retrying every 2000ms; change
> 'reconnectDelay' to configure the frequency of retries).
> class org.apache.ignite.spi.IgniteSpiException: Failed to retrieve
> Ignite pods IP addresses.
> at
> org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:172)
>
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
>
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1848)
>
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1049)
>
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:910)
>
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:391)
>
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2020)
>
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
>
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
>
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
>
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
>
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
>
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
> at
> org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076)
>
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700)
> at org.apache.ignite.Ignition.start(Ignition.java:348)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)
>
> Caused by: java.io.IOException: Server returned HTTP response code:
> 403 for URL:
> https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/dataobjns/endpoints/ignite
>
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
>
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
>
> at
> sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
>
> at
> org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:153)
>
> ... 20 more
> [11:19:15,821][WARNING][jvm-pause-detector-worker][IgniteKernal]
> Possible too long JVM pause: 500 milliseconds.
>
> any help is appreciated.