[jira] [Commented] (CASSANDRA-6311) Add CqlRecordReader to take advantage of native CQL pagination

["Jonathan Ellis (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13832811#comment-13832811 ] 

Jonathan Ellis commented on CASSANDRA-6311:
-------------------------------------------

Summary of discussion in chat:

The user is responsible for providing a valid CQL statement, including token bind variables.

The IF API needs to change, probably to {{<Long, Row>}} where Row is a Java Driver Row (http://www.datastax.com/drivers/java/2.0/apidocs/com/datastax/driver/core/Row.html) and Long is a per-Task row ID.  (Precedent: DBInputFormat also uses a Long ID -- https://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/db/DBInputFormat.html.)

We can either use the metadata from the java driver to continue to estimate progress based on partitions, or switch to estimating progress by CQL row count if there is a way to get that from the server easily.

> Add CqlRecordReader to take advantage of native CQL pagination
> --------------------------------------------------------------
>
>                 Key: CASSANDRA-6311
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6311
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Hadoop
>            Reporter: Alex Liu
>            Assignee: Alex Liu
>             Fix For: 2.0.4
>
>         Attachments: 6311-v3-2.0-branch.txt, 6331-2.0-branch.txt, 6331-v2-2.0-branch.txt
>
>
> Since the latest Cql pagination is done and it should be more efficient, so we need update CqlPagingRecordReader to use it instead of the custom thrift paging.



--
This message was sent by Atlassian JIRA
(v6.1#6144)