You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kudu.apache.org by "Kevin J McCarthy (Jira)" <ji...@apache.org> on 2020/08/03 14:32:00 UTC

[jira] [Commented] (KUDU-3177) Expose snapshotTimestampMicros to Spark Read Options

    [ https://issues.apache.org/jira/browse/KUDU-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170074#comment-17170074 ] 

Kevin J McCarthy commented on KUDU-3177:
----------------------------------------

[https://gerrit.cloudera.org/#/c/16276/]

> Expose snapshotTimestampMicros to Spark Read Options
> ----------------------------------------------------
>
>                 Key: KUDU-3177
>                 URL: https://issues.apache.org/jira/browse/KUDU-3177
>             Project: Kudu
>          Issue Type: Improvement
>          Components: spark
>            Reporter: Kevin J McCarthy
>            Priority: Major
>              Labels: beginner
>
> If a spark application needs to read from the same table multiple times and that table has new records that may come in during the life of the application, you may get inconsistent scan results unless you persist the DataFrame. I'd like to expose snapshotTimestampMicros to the spark read options so I can set a timestamp before the first scan and use that for READ_AT_SNAPSHOT to keep all scans on the same table consistent throughout the run of the application. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)