[jira] [Resolved] (TRAFODION-2645) MDAM costing overestimates I/O cost

["David Wayne Birdsall (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TRAFODION-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Wayne Birdsall resolved TRAFODION-2645.
---------------------------------------------
       Resolution: Fixed
    Fix Version/s: 2.3

It turns out that the original code had the right spirit. It is indeed the subset interaction (with its message interaction between a Trafodion process and the HBase RegionServer) that is most expensive. In the new MDAM costing code rewrite this is modeled more clearly.

The code is turned off by default for the moment. To test it, set CQD MDAM_COSTING_REWRITE 'ON'. When we can calibrate the new costing code on large systems, we can change the default to 'ON'.

> MDAM costing overestimates I/O cost
> -----------------------------------
>
>                 Key: TRAFODION-2645
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-2645
>             Project: Apache Trafodion
>          Issue Type: Bug
>          Components: sql-cmp
>    Affects Versions: 2.2-incubating
>         Environment: All
>            Reporter: David Wayne Birdsall
>            Assignee: David Wayne Birdsall
>            Priority: Major
>             Fix For: 2.3
>
>
> The method MDAMCostWA::compute (optimizer/ScanOptimizer.cpp) has logic to calculate the total cost of an MDAM plan and compare it to the cost of a single subset plan. In the case of a Trafodion table, the cost logic for MDAM cost has an error. It is adding the total number of probes (totalRqsts) to the number of seeks (totalSeeks) in order to calculate I/O costs.
> TotalSeeks models direct access disk seeks. These happen at the block level and are an I/O cost.
> TotalRqsts models the number of probes, which is a CPU cost. A probe is a direct access within a block that has already been read into memory. This is a vastly smaller unit of cost than an I/O operation.
> Note that TotalRqsts is already modeled as a message cost since it is added to TotalRows.
> The effect of the bug is to vastly penalize MDAM plans having large numbers of probes. This will show up the most on the largest tables.
> The fix is simply not to add totalRqsts to totalSeeks. However, in making this fix, it is likely that other MDAM-related parameters will need tuning. So this change needs to be done carefully.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)