[jira] [Updated] (DERBY-6211) Make Optimizer trace logic pluggable.

["Rick Hillegas (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DERBY-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rick Hillegas updated DERBY-6211:
---------------------------------

    Attachment: derby-6211-09-aa-addTests.diff

Attaching derby-6211-09-aa-addTests.diff. This patch adds tests for xml-based optimizer tracing. I am running the full tests now.

This patch makes a couple changes:

1) Wraps a privilege block around the file writes performed by optimizer tracing.

2) Wraps a privilege block around the file reading performed by the XmlVTI.

3) Changes xml-based optimizer tracing:

a) Removes the verbose summary element. After other changes to the summary element, I don't think that the verbose element adds much value any more.

b) Uses OptimizerPlan.toString() to format plan summaries. This should help enforce the contract that summary lines can be used as optimizer overrides.

4) Adds tests for xml-based optimizer tracing. More extensive tests would be useful but I haven't figured out a way to ensure cross-platform stability for the results of more complicated tests. I am hoping that these tests turn out to be stable when run as part of the full regression test suite.

While experimenting with these tests, I tripped across some interesting behaviors:

A) An optimizer trace (either xml-based or otherwise) can show two separate optimizer sessions for a query. The first session happens when you prepare the query. The second happens when the execution logic decides that it needs to re-prepare the query you just prepared. I saw this behavior for a query against tables which I had just stuffed with many rows. Maybe statistics were being automatically calculated between the initial preparation and the actual execution of the query.

B) The order in which plan shapes appear in the optimizer trace is not deterministic. This may be an instability in the planCost table function. But the instability may be in the optimizer itself. Maybe the optimizer considers plans in an unstable order. That would be weird and troubling.


Touches the following files:

------------

M       java/engine/org/apache/derby/impl/sql/compile/OptimizerTracer.java

(1)

------------

M       java/engine/org/apache/derby/vti/XmlVTI.java

(2)

------------

M       java/engine/org/apache/derby/impl/sql/compile/XMLOptTrace.java

(3)

------------

A       java/testing/org/apache/derbyTesting/functionTests/tests/lang/RSMDWrapper.java
M       java/testing/org/apache/derbyTesting/functionTests/tests/lang/_Suite.java
A       java/testing/org/apache/derbyTesting/functionTests/tests/lang/XMLOptimizerTraceTest.java

(4). Along the way, I checked in a slightly amended version of the RSMDWrapper table function attached to DERBY-4926. This creates a table function out of the ResultSetMetaData returned by a query; it can be useful in tests which need to verify the shapes of ResultSets.

                
> Make Optimizer trace logic pluggable.
> -------------------------------------
>
>                 Key: DERBY-6211
>                 URL: https://issues.apache.org/jira/browse/DERBY-6211
>             Project: Derby
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 10.11.0.0
>            Reporter: Rick Hillegas
>            Assignee: Rick Hillegas
>              Labels: derby_triage10_11
>         Attachments: derby-6211-01-aa-createPlugin.diff, derby-6211-02-aa-cleanup.diff, derby-6211-02-ab-cleanup.diff, derby-6211-03-aa-customTracer.diff, derby-6211-04-aa-moveOptimizerTracerToEngineJar.diff, derby-6211-05-aa-xmlOptimizerTracer.diff, derby-6211-06-ab-packageProtect-XMLOptTrace.diff, derby-6211-07-aa-useSchemaQualifiedNamesInSummaries.diff, derby-6211-07-ab-useSchemaQualifiedNamesInSummaries.diff, derby-6211-08-aa-fixNPE.diff, derby-6211-09-aa-addTests.diff
>
>
> Right now the trace logic in the optimizer is hard-coded to produce a stream of diagnostics. It would be good to be able to plug alternative trace logic into the optimizer. This would make the following possible:
> 1) Plug in trace logic which produces formats which are easier to study and which can be analyzed mechanically. E.g., xml formatted output.
> 2) Plug in trace logic which can be used during unit testing to verify that the optimizer has picked the right plan. Over time this might make it easier to migrate canon-based tests to assertion-based tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira