Long-running Job to Extract Data Timesout After ~ 60 Hours

[Fred Habash <fm...@gmail.com>]

We tried to extract large volume of data from a 42-node cluster about three
times and in all attempts, client sessions aborts after ~ 60 hours.

Here's what we see in in the client logs

I have reviewed the multiple timeout settings in C*, but none seemed to
relate to the 60 hrs limit.

What is potentially enforcing the ~ 60-hr timeout?

19:38:55.545 [cluster1-nio-worker-0] DEBUG
com.datastax.driver.core.Connection - Connection[
cassandra.prod.int.MyBiz.com/10.242.145.21:9042-1, inFlight=0,
closed=false] heartbeat query succeeded
19:39:15.804 [main] ERROR com.MyBiz.getdata.main.FetchTrackingData -
com.MyBiz.getdata.main.FetchTrackingData:main:main Issue with getting
connection or clusterCassandra timeout during read query at consistency
LOCAL_ONE (1 responses were required but only 0 replica responded)
com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout
during read query at consistency LOCAL_ONE (1 responses were required but
only 0 replica responded)
        at
com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:88)
        at
com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:25)
        at
com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
        at
com.datastax.driver.core.ArrayBackedResultSet$MultiPage.prepareNextRow(ArrayBackedResultSet.java:308)
        at
com.datastax.driver.core.ArrayBackedResultSet$MultiPage.isExhausted(ArrayBackedResultSet.java:265)
        at
com.datastax.driver.core.ArrayBackedResultSet$1.hasNext(ArrayBackedResultSet.java:136)
        at
com.MyBiz.getdata.main.FetchTrackingData.main(FetchTrackingData.java:118)
Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException:
Cassandra timeout during read query at consistency LOCAL_ONE (1 responses
were required but only 0 replica responded)
        at
com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:115)


----------------------------------------
Thank you