[jira] [Created] (DRILL-4832) Metadata Cache Pruning is not taking place in some rare scenarios

["Rahul Challapalli (JIRA)" <ji...@apache.org>]

Rahul Challapalli created DRILL-4832:
----------------------------------------

             Summary: Metadata Cache Pruning is not taking place in some rare scenarios
                 Key: DRILL-4832
                 URL: https://issues.apache.org/jira/browse/DRILL-4832
             Project: Apache Drill
          Issue Type: Bug
          Components: Query Planning & Optimization
            Reporter: Rahul Challapalli


git.commit.id.abbrev=f476eb5

The below set of queries is one test case.  I usually run 10 instances of the same test case in parallel and we read the cache file from the appropriate sub-directory in almost all the cases. 
But only once out of 25 such runs, I observed that we are reading the cache from the root level instead of the sub-directory for the "l_3level" table.

{code}
refresh table metadata l_3level;
refresh table metadata c_1level;
refresh table metadata o_2level;
explain plan for select
  l.l_orderkey,
  sum(l.l_extendedprice * (1 - l.l_discount)) as revenue,
  o.o_orderdate,
  o.o_shippriority
from
  c_1level c,
  o_2level o,
  l_3level l
where
  c.c_mktsegment = 'HOUSEHOLD'
  and c.c_custkey = o.o_custkey
  and l.l_orderkey = o.o_orderkey
  and l.dir0 = 1 and l.dir1 = 'three' and l.dir2 = '2015-7-12'
  and o.dir0 = '1991' and o.dir1 = 'feb'
  and o.o_orderdate < date '1995-03-25'
  and l.l_shipdate > date '1995-03-25'
group by
  l.l_orderkey,
  o.o_orderdate,
  o.o_shippriority

00-00    Screen
00-01      Project(l_orderkey=[$0], o_orderdate=[$1], o_shippriority=[$2])
00-02        Project(l_orderkey=[$8], o_orderdate=[$6], o_shippriority=[$7])
00-03          HashJoin(condition=[=($1, $2)], joinType=[inner])
00-05            SelectionVectorRemover
00-08              Filter(condition=[=($0, 'HOUSEHOLD')])
00-11                Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/drill/testdata/metadata_caching_pp/c_1level/1991/customer.parquet], ReadEntryWithPath [path=/drill/testdata/metadata_caching_pp/c_1level/1992/customer.parquet]], selectionRoot=/drill/testdata/metadata_caching_pp/c_1level, numFiles=2, usedMetadataFile=true, cacheFileRoot=/drill/testdata/metadata_caching_pp/c_1level, columns=[`c_mktsegment`, `c_custkey`]]])
00-04            HashJoin(condition=[=($6, $1)], joinType=[inner])
00-07              SelectionVectorRemover
00-10                Filter(condition=[AND(=($2, '1991'), =($3, 'feb'), <($4, 1995-03-25))])
00-13                  Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/drill/testdata/metadata_caching_pp/o_2level/1991/feb/orders.parquet]], selectionRoot=/drill/testdata/metadata_caching_pp/o_2level, numFiles=1, usedMetadataFile=true, cacheFileRoot=/drill/testdata/metadata_caching_pp/o_2level/1991/feb, columns=[`o_custkey`, `o_orderkey`, `dir0`, `dir1`, `o_orderdate`, `o_shippriority`]]])
00-06              Project(l_orderkey=[$0], dir00=[$1], dir10=[$2], dir2=[$3], l_shipdate=[$4])
00-09                SelectionVectorRemover
00-12                  Filter(condition=[AND(=($1, 1), =($2, 'three'), =($3, '2015-7-12'), >($4, 1995-03-25))])
00-14                    Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/drill/testdata/metadata_caching_pp/l_3level/1/three/2015-7-12/40.parquet]], selectionRoot=/drill/testdata/metadata_caching_pp/l_3level, numFiles=1, usedMetadataFile=true, cacheFileRoot=/drill/testdata/metadata_caching_pp/l_3level, columns=[`l_orderkey`, `dir0`, `dir1`, `dir2`, `l_shipdate`]]])
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)