export ignite data

[Anil <an...@gmail.com>]

HI,

we have export functionality which reads the ignite data (around 1L
records) and add to file.

i have implemented using jdbc driver to get the fetch size of 100. when i
run parallel exports , client is getting restarted with following log

2017-05-15 22:36:50 342 ERROR TcpDiscoverySpi:495 - Failed to send message:
TcpDiscoveryClientHeartbeatMessage [super=TcpDiscoveryAbstractMessage
[sndNodeId=null, id=6e6089f0c51-17f17130-b771-42d2-88d6-3be03fda1389,
verifierNodeId=null, topVer=0, pendingIdx=0, failedNodes=null,
isClient=true]]
java.net.SocketException: Socket is closed
        at java.net.Socket.getSendBufferSize(Socket.java:1215)
        at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.socketStream(TcpDiscoverySpi.java:1254)
        at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.writeToSocket(TcpDiscoverySpi.java:1366)
        at
org.apache.ignite.spi.discovery.tcp.ClientImpl$SocketWriter.body(ClientImpl.java:1095)
        at
org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
2017-05-15 22:37:04 903 WARN  TcpDiscoverySpi:480 - Client node was
reconnected after it was already considered failed by the server topology
(this could happen after all servers restarted or due to a long network
outage between the client and servers). All continuous queries and remote
event listeners created by this client will be unsubscribed, consider
listening to EVT_CLIENT_NODE_RECONNECTED event to restore them.
2017-05-15 22:37:05 094 INFO  GridDiscoveryManager:475 - Client node
reconnected to topology: TcpDiscoveryNode
[id=3749aab0-5048-4b33-933e-7c03ff6d921b, addrs=[0:0:0:0:0:0:0:1%lo,
127.0.0.1, 172.16.41.6], sockAddrs=[/0:0:0:0:0:0:0:1%lo:0, /127.0.0.1:0,
aswb-ignite-client-stage-hnpqd/172.16.41.6:0], discPort=0, order=381,
intOrder=0, lastExchangeTime=1494910250111, loc=true,
ver=1.9.0#20170302-sha1:a8169d0a, isClient=true]
2017-05-15 22:37:05 094 INFO  GridDiscoveryManager:475 - Topology snapshot
[ver=381, servers=8, clients=5, CPUs=72, heap=160.0GB]

this could be because of long running query. Is there any better way of
implementing export like feature using ignite ? Please advise. Thanks.


Thanks.

Re: export ignite data

[dkarachentsev <dk...@gridgain.com>]

Hi,

Probably the reason of client reconnection was a long GC pause on it.
Try to use scan query [1], it's liter than SQL, and check that you have
enough of heap.

[1] https://apacheignite.readme.io/v1.8/docs/cache-queries#scan-queries

Thanks!
-Dmitry



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/export-ignite-data-tp12873p12877.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.