You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/10/05 01:55:22 UTC
[jira] [Commented] (TRAFODION-2263) Move Similarity check for hive
tables to master executor root operator
[ https://issues.apache.org/jira/browse/TRAFODION-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547356#comment-15547356 ]
ASF GitHub Bot commented on TRAFODION-2263:
-------------------------------------------
GitHub user anoopsharma00 opened a pull request:
https://github.com/apache/incubator-trafodion/pull/746
jira TRAFODION-2263 Similarity check changes and obsolete code removal.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/anoopsharma00/incubator-trafodion ansharma_simcheck_br
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/746.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #746
----
commit 074428ed74a9b90a3c131007e7ecdeac6720ea99
Author: Anoop Sharma <an...@esgyn.com>
Date: 2016-10-03T18:55:39Z
similarity check changes, commit #1
commit 667cf65f80ef97df5a5148756d2d04978c64ee4e
Author: Anoop Sharma <an...@esgyn.com>
Date: 2016-10-04T19:36:59Z
Merge remote branch 'origin/master' into ansharma_simcheck_br
commit 5eca1883817ce0af756e976c6563250cdaa73979
Author: Anoop Sharma <an...@esgyn.com>
Date: 2016-10-05T01:25:24Z
Merge remote branch 'origin/master' into ansharma_simcheck_br
commit 3d41ee0f0bfac5af8db1f485b5369787a382c97e
Author: Anoop Sharma <an...@esgyn.com>
Date: 2016-10-05T01:53:19Z
jira TRAFODION-2263 Similarity check changes and obsolete code removal.
-- similarity check for tables now done at root operator before start of
query execution. Done only for hive tables by detecting data
dir modification time change.
-- the older leaf data mod check is still available and
can be enabled by a cqd. This is temporary and will be removed
once the newer code has stabilized.
-- data mod mismatch now also returns the runtime dir which caused it.
-- removed obsolete code related to query recompilation from pre-AQR days
-- removed obsolete code related to runtime query execution in cli
-- removed obsolete code related to runtime table name resolution,
visibility checks and recompile late name list handling
-- removed other dead or ifdefd out code
----
> Move Similarity check for hive tables to master executor root operator
> ----------------------------------------------------------------------
>
> Key: TRAFODION-2263
> URL: https://issues.apache.org/jira/browse/TRAFODION-2263
> Project: Apache Trafodion
> Issue Type: Improvement
> Reporter: Anoop Sharma
> Assignee: Anoop Sharma
> Priority: Minor
>
> Query plans for hive tables include location of data directories/files
> and the plan depends on it.
> If data changes in a hive file/dir after a query has been compiled and
> before it is executed, then that change is detected at runtime by
> comparing compile time and runtime dir timestamps.
> A mismatch causes that query to recompile using AQR and a new
> plan is generated with latest dir location.
> This detection is currently being done at leaf operators (scan, insert) when
> that operator is executed.
> This jira is to move that check (called Similarity Check) to the root
> operator before starting query execution. That way plan consistency
> checks will be done before running the query and will not depend
> on the actual operator being executed. It will also result in checks not
> being done at various processes and operators like an ESP process.
> All checks will be done at one place in master executor and code will not
> be spread across multiple operators as it is now.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)