You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@cassandra.apache.org by Serega Sheypak <se...@gmail.com> on 2015/06/17 12:20:30 UTC

spark-sql estimates Cassandra table with 3 rows as 8 TB of data, Cassandra 2.1, DSE 4.7

Hi, spark-sql estimated input for Cassandra table with 3 rows as 8 TB.
sometimes it's estimated as -167B.
I run it on laptop, I don't have 8 TB space for the data.

We use DSE 4.7 with bundled spark and spark-sql-thriftserver

Here is the stat for a dummy select foo from bar where bar three rows and
several columns


   - *Total task time across all tasks: *7.6 min
   - *Input: *8388608.0 TB

I don't have so much TB on my macbook pro. I would like to, but I dont :(