[jira] [Updated] (CALCITE-604) Invoke metadata providers via a class generated at runtime, rather than reflection

["Julian Hyde (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CALCITE-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Julian Hyde updated CALCITE-604:
--------------------------------
    Description: 
Currently we dispatch to metadata providers using reflection. With this change, we generate a dispatcher class and compile using Janino.

Currently calls to metadata providers are dispatched via reflection (Method.invoke, see ReflectiveRelMetadataProvider, ChainedRelMetadataProvider and CachingInvocationHandler), and I suspect that planning a complex query (especially with tracing enabled) makes millions of calls. Reflection has a performance problem; lots of boxing and unboxing and creating temporary arrays and bound objects.

One option we considered to improve performance was moving to MethodHandle. (JDK 1.7 introduced http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html, a more efficient way to invoke methods. Access control etc. is done when the handle is created, not each invocation.) However, MethodHandle.invoke can dispatch only within one provider.

So, we should generate a dispatcher class, and compile and instantiate at run time using Janino.

For each metadata interface we will need to add a handler interface that has the same methods but two extra arguments (a sub-class of RelNode, and a RelMetadataQuery). Previously these arguments were passed by calling "bind" on an UnboundMetadata.

We can also make the dispatcher cache the results within a metadata call. This should yield a performance improvement when, say, a join requires many kinds of metadata (rowCount, averageRowSize, selectivity) and they all rely on the rowCount of the input, and that rowCount is expensive to compute.

  was:
JDK 1.7 introduced http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html, a more efficient way to invoke methods. Access control etc. is done when the handle is created, not each invocation.

Currently calls to metadata providers are dispatched via reflection (Method.invoke, see ReflectiveRelMetadataProvider, ChainedRelMetadataProvider and CachingInvocationHandler), and I suspect that planning a complex query (especially with tracing enabled) makes millions of calls. Reflection has a performance problem; lots of boxing and unboxing and creating temporary arrays and bound objects.

One option we considered to improve performance was moving to MethodHandle. However, MethodHandle.invoke can dispatch only within one provider.

So, we should generate a handler class, and compile and instantiate at run time using Janino.

For each metadata interface we will need to add a handler interface that has the same methods but two extra arguments (a sub-class of RelNode, and a RelMetadataQuery). Previously these arguments were passed by calling "bind" on an UnboundMetadata.


> Invoke metadata providers via a class generated at runtime, rather than reflection
> ----------------------------------------------------------------------------------
>
>                 Key: CALCITE-604
>                 URL: https://issues.apache.org/jira/browse/CALCITE-604
>             Project: Calcite
>          Issue Type: Bug
>            Reporter: Julian Hyde
>            Assignee: Julian Hyde
>
> Currently we dispatch to metadata providers using reflection. With this change, we generate a dispatcher class and compile using Janino.
> Currently calls to metadata providers are dispatched via reflection (Method.invoke, see ReflectiveRelMetadataProvider, ChainedRelMetadataProvider and CachingInvocationHandler), and I suspect that planning a complex query (especially with tracing enabled) makes millions of calls. Reflection has a performance problem; lots of boxing and unboxing and creating temporary arrays and bound objects.
> One option we considered to improve performance was moving to MethodHandle. (JDK 1.7 introduced http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.html, a more efficient way to invoke methods. Access control etc. is done when the handle is created, not each invocation.) However, MethodHandle.invoke can dispatch only within one provider.
> So, we should generate a dispatcher class, and compile and instantiate at run time using Janino.
> For each metadata interface we will need to add a handler interface that has the same methods but two extra arguments (a sub-class of RelNode, and a RelMetadataQuery). Previously these arguments were passed by calling "bind" on an UnboundMetadata.
> We can also make the dispatcher cache the results within a metadata call. This should yield a performance improvement when, say, a join requires many kinds of metadata (rowCount, averageRowSize, selectivity) and they all rely on the rowCount of the input, and that rowCount is expensive to compute.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)