You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Gokcen Iskender (Jira)" <ji...@apache.org> on 2020/04/14 23:01:00 UTC

[jira] [Updated] (PHOENIX-5841) When data columns get TTLed, we need inline index validation to publish a metric for this

     [ https://issues.apache.org/jira/browse/PHOENIX-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gokcen Iskender updated PHOENIX-5841:
-------------------------------------
    Description: 
We do index writes as full row writes. This means all columns keep get re-written to index and they have a current timestamp.

However, if there is a column that did not get updated for a long time (like Created_By type of columns that don't change) index inline validation marks these as "Index has extra columns". We need to publish an extra metric to distinguish these cases since they are expected to be not matching rows.

  was:
We do index writes as full row writes. This means all columns keep get re-written to index and they have a current timestamp.

However, if there is a column that did not get updated for a long time (like Created_By type of columns that don't change) the regular scan that IndexScrutiny uses doesn't return this column even though we might see it in the raw scan.

IndexScrutiny mark these rows as invalid because data table had TTL columns and index table have it.

Index inline validation also marks these as "Index has extra columns" if the column got TTLed.


> When data columns get TTLed, we need inline index validation to publish a metric for this
> -----------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5841
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5841
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Gokcen Iskender
>            Priority: Major
>
> We do index writes as full row writes. This means all columns keep get re-written to index and they have a current timestamp.
> However, if there is a column that did not get updated for a long time (like Created_By type of columns that don't change) index inline validation marks these as "Index has extra columns". We need to publish an extra metric to distinguish these cases since they are expected to be not matching rows.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)