[jira] [Comment Edited] (PHOENIX-3176) Rows will be skipped which are having future timestamp in row_timestamp column

["Samarth Jain (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418114#comment-15418114 ] 

Samarth Jain edited comment on PHOENIX-3176 at 8/11/16 11:27 PM:
-----------------------------------------------------------------

Wrote a test with ROW_TIMESTAMP column following UpsertSelectAutoCommitIT#testUpsertSelectDoesntSeeUpsertedData. The test passes with and without the patch. The time range on the scan doesn't take into account the time at which the table was resolved. 

Here is the test that I wrote:
{code}
@Test
    public void testUpsertSelectDoesntSeeUpsertedDataWithRowTimestampColumn() throws Exception {
        Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
        props.setProperty(QueryServices.MUTATE_BATCH_SIZE_ATTRIB, Integer.toString(3));
        props.setProperty(QueryServices.SCAN_CACHE_SIZE_ATTRIB, Integer.toString(3));
        props.setProperty(QueryServices.SCAN_RESULT_CHUNK_SIZE, Integer.toString(3));
        Connection conn = DriverManager.getConnection(getUrl(), props);
        conn.setAutoCommit(true);
        conn.createStatement().execute("CREATE SEQUENCE keys");
        String tableName = generateRandomString();
        conn.createStatement().execute(
            "CREATE TABLE " + tableName + " (pk BIGINT PRIMARY KEY ROW_TIMESTAMP, val INTEGER)");

        conn.createStatement().execute(
            "UPSERT INTO " + tableName + " VALUES (NEXT VALUE FOR keys,1)");
        for (int i=0; i<6; i++) {
            Statement stmt = conn.createStatement();
            int upsertCount = stmt.executeUpdate(
                "UPSERT INTO " + tableName + " SELECT NEXT VALUE FOR keys, val FROM " + tableName);
            assertEquals((int)Math.pow(2, i), upsertCount);
        }
        conn.close();
    }
{code}


was (Author: samarthjain):
Wrote a test with ROW_TIMESTAMP column following UpsertSelectAutoCommitIT#testUpsertSelectDoesntSeeUpsertedData. The test passes with and without the patch. The time range on the scan doesn't take into account the time at which the table was resolved. 

Here is the test that I wrote:
@Test
    public void testUpsertSelectDoesntSeeUpsertedDataWithRowTimestampColumn() throws Exception {
        Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
        props.setProperty(QueryServices.MUTATE_BATCH_SIZE_ATTRIB, Integer.toString(3));
        props.setProperty(QueryServices.SCAN_CACHE_SIZE_ATTRIB, Integer.toString(3));
        props.setProperty(QueryServices.SCAN_RESULT_CHUNK_SIZE, Integer.toString(3));
        Connection conn = DriverManager.getConnection(getUrl(), props);
        conn.setAutoCommit(true);
        conn.createStatement().execute("CREATE SEQUENCE keys");
        String tableName = generateRandomString();
        conn.createStatement().execute(
            "CREATE TABLE " + tableName + " (pk BIGINT PRIMARY KEY ROW_TIMESTAMP, val INTEGER)");

        conn.createStatement().execute(
            "UPSERT INTO " + tableName + " VALUES (NEXT VALUE FOR keys,1)");
        for (int i=0; i<6; i++) {
            Statement stmt = conn.createStatement();
            int upsertCount = stmt.executeUpdate(
                "UPSERT INTO " + tableName + " SELECT NEXT VALUE FOR keys, val FROM " + tableName);
            assertEquals((int)Math.pow(2, i), upsertCount);
        }
        conn.close();
    }
{code}

> Rows will be skipped which are having future timestamp in row_timestamp column
> ------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3176
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3176
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.6.0
>            Reporter: Ankit Singhal
>             Fix For: 4.8.1
>
>         Attachments: PHOENIX-3176.patch
>
>
> Rows will be skipped when row_timestamp have future timestamp
> {code}
> : jdbc:phoenix:localhost> CREATE TABLE historian.data (
> . . . . . . . . . . . . .> assetid unsigned_int not null,
> . . . . . . . . . . . . .> metricid unsigned_int not null,
> . . . . . . . . . . . . .> ts timestamp not null,
> . . . . . . . . . . . . .> val double
> . . . . . . . . . . . . .> CONSTRAINT pk PRIMARY KEY (assetid, metricid, ts row_timestamp))
> . . . . . . . . . . . . .> IMMUTABLE_ROWS=true;
> No rows affected (1.283 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data values(1,2,'2015-01-01',1.2);
> 1 row affected (0.047 seconds)
> 0: jdbc:phoenix:localhost> upsert into historian.data values(1,2,'2018-01-01',1.2);
> 1 row affected (0.005 seconds)
> 0: jdbc:phoenix:localhost> select * from historian.data;
> +----------+-----------+--------------------------+------+
> | ASSETID  | METRICID  |            TS            | VAL  |
> +----------+-----------+--------------------------+------+
> | 1        | 2         | 2015-01-01 00:00:00.000  | 1.2  |
> +----------+-----------+--------------------------+------+
> 1 row selected (0.04 seconds)
> 0: jdbc:phoenix:localhost> select count(*) from historian.data;
> +-----------+
> | COUNT(1)  |
> +-----------+
> | 1         |
> +-----------+
> 1 row selected (0.013 seconds)
> {code}
> Explain plan, where scan range is capped to compile time.
> {code}
> | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER HISTORIAN.DATA  |
> |     ROW TIMESTAMP FILTER [0, 1470901929982)                  |
> |     SERVER FILTER BY FIRST KEY ONLY                          |
> |     SERVER AGGREGATE INTO SINGLE ROW                         |
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)