[jira] [Commented] (HIVE-14498) Freshness period for query rewriting using materialized views

["Eugene Koifman (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286312#comment-16286312 ] 

Eugene Koifman commented on HIVE-14498:
---------------------------------------

could 
{noformat}
s = "select ctc_database, ctc_table, ctc_id, ctc_timestamp from COMPLETED_TXN_COMPONENTS where ctc_txnid = " + txnid;
840	        // Update registry with modifications
841	        LOG.debug("Going to register table modification in invalidation cache <" + s + ">");
842	        rs = stmt.executeQuery(s);
843	        if (rs.next()) {
844	          MaterializationsInvalidationCache.get().notifyTableModification(
845	              rs.getString(1), rs.getString(2), rs.getLong(3),
846	              rs.getTimestamp(4, Calendar.getInstance(TimeZone.getTimeZone("UTC"))).getTime());
847	        }
{noformat}
be done after dbConn.commit();?
I would prefer to keep the db transaction as short as possible.  The info produced by this query should be "stable" since it has 
{noformat}
where ctc_txnid = " + txnid;.
{noformat}


> Freshness period for query rewriting using materialized views
> -------------------------------------------------------------
>
>                 Key: HIVE-14498
>                 URL: https://issues.apache.org/jira/browse/HIVE-14498
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Materialized views
>    Affects Versions: 2.2.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>         Attachments: HIVE-14498.patch
>
>
> Once we have query rewriting in place (HIVE-14496), one of the main issues is data freshness in the materialized views.
> Since we will not support view maintenance at first, we could include a HiveConf property to configure a max freshness period (_n timeunits_). If a query comes, and the materialized view has been populated (by create, refresh, etc.) for a longer period than _n_, then we should not use it for rewriting the query.
> Optionally, we could print a warning for the user indicating that the materialized was not used because it was not fresh.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)