calcite+graal native-image

[Egor Ryashin <ri...@gmail.com>]

Hi all,

I’ve been trying to build a native executable with GraalVM and got stuck with this exception when running the executable:

java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CollationHandler
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:180)
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:138)
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:72)
	at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3951)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
	at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:189)

I found a ticket that probably relates to the issue https://issues.apache.org/jira/browse/CALCITE-4539 <https://issues.apache.org/jira/browse/CALCITE-4539> 
I also found that substrait.io <http://substrait.io/> (substrait-java) already compiles Calcite dependency to a native executable. Right now, I’m confused, is it possible to compile Calcite to a working native executable or not? It would be great if somebody could give some insight on how to do that. I tried to apply the substrait-java configs and used the GraalVM trace agent generated configs too but I still get the above exception nonetheless.

Thanks

Re: calcite+graal native-image

[Jacques Nadeau <ja...@apache.org>]

I definitely suggest that you look at how Substrait compiles. There are a
number of customizations on top of vanilla Calcite that are done. Key
things to observe include:
* Configuration of runtime/compile time classes at build. [1]
* Registration of addition reflection, etc items [2]
* Use of proxying metadata handler (as opposed to code generating one) [3]

The specific problem you are hitting looks like the third bullet above
based on the stacktrace.

Note that anywhere Calcite uses Janino/code generation has to have an
alternative path to function. This means that if you want to use other
parts of Calcite, you'll have to supplement for this requirement (for
example, I believe constant reduction uses janino and thus would need an
alternative path).

[1]
https://github.com/substrait-io/substrait-java/blob/main/isthmus/build.gradle.kts#L42
[2]
https://github.com/substrait-io/substrait-java/blob/main/isthmus/src/main/java/io/substrait/isthmus/RegisterAtRuntime.java
[3]
https://github.com/substrait-io/substrait-java/blob/main/isthmus/src/main/java/io/substrait/isthmus/SqlConverterBase.java#L63

On Tue, Jul 19, 2022 at 11:05 AM Egor Ryashin <ri...@gmail.com>
wrote:

> Hi all,
>
> I’ve been trying to build a native executable with GraalVM and got stuck
> with this exception when running the executable:
>
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CollationHandler
>         at
> org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:180)
>         at
> org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:138)
>         at
> org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:72)
>         at
> com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
>         at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
>         at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
>         at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
>         at
> com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
>         at com.google.common.cache.LocalCache.get(LocalCache.java:3951)
>         at
> com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
>         at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
>         at
> org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:189)
>
> I found a ticket that probably relates to the issue
> https://issues.apache.org/jira/browse/CALCITE-4539 <
> https://issues.apache.org/jira/browse/CALCITE-4539>
> I also found that substrait.io <http://substrait.io/> (substrait-java)
> already compiles Calcite dependency to a native executable. Right now, I’m
> confused, is it possible to compile Calcite to a working native executable
> or not? It would be great if somebody could give some insight on how to do
> that. I tried to apply the substrait-java configs and used the GraalVM
> trace agent generated configs too but I still get the above exception
> nonetheless.
>
> Thanks
>
>