[jira] [Closed] (TRAFODION-1900) Optimize MDAM scans with small scanner

["Eric Owhadi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TRAFODION-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Owhadi closed TRAFODION-1900.
----------------------------------

> Optimize MDAM scans with small scanner
> --------------------------------------
>
>                 Key: TRAFODION-1900
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-1900
>             Project: Apache Trafodion
>          Issue Type: Improvement
>          Components: sql-exe
>            Reporter: Eric Owhadi
>            Assignee: Eric Owhadi
>             Fix For: 2.0-incubating
>
>
> When doing MDAM scans, we are performing interlaced scan for PROBE and for real scan. The probes always return only 1 row, then we close the scanner immediately, therefore should use always small scanner. I will make it conditional on the existing CQD HBASE_SMALL_SCANNER (ether SYSTEM or ON). In addition, caching of blocks retrieved by probe should always we at least receiving one succesfull cache hit on the next MDAM scan, therefore forcing caching ON for MDAM prob is a good idea. Again, will make this forcing conditional on HBASE_SMALL_SCANNER SYSTEM or ON.
> Then for the real scan part of MDAM, I will use the following heuristic: If previous scan fitted in one hbase block, then it is likelly than next will also fit in one hbase block, therefore enable small scanner for next scan. Again all this only if CQD above is ON or SYSTEM.
> Results of using small scanner on MDAM when it make sense showed a 1.39X speed improvement...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)